GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-130806: emit ResourceWarning if GzipFile unclosed by cmaloney · Pull Request #130905 · python/cpython

Conversation

@cmaloney

@cmaloney cmaloney commented Mar 6, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

This may indicate accidental data loss, the crc and size fields of the GzipFile are only written on close.

Ways to make sure all data is written:

  1. Use the file-like object as a “With Statement Context Manager”.
    • All objects which inherit from IOBase support this.
    • BufferedIOBase, BufferedWriter, and GzipFile all support this.
    • Ensures .close() is called in both exception and regular cases.
  2. Ensure .close() is always called which flushes data before closing.
  3. If the underlying stream needs to be kept open, use .detach()

Since at least 3.12 flushing has been necessary in GzipFile (see gh-105808 which was a release blocker), this makes that more visible. Users have been encountering as they upgrade to 3.12 (ex. gh-129726).

  • Issue: Emit ResourceWarning when GzipFile is deleted with unwritten data #130806

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@cmaloney
pythongh-130806: emit ResourceWarning if GzipFile unclosed
8bf1895
This may indicate accidental data loss.

Ways to make sure all data is written:
1. Use the [file-like object](https://docs.python.org/3/glossary.html#term-file-object) as a [“With Statement Context Manager”](https://docs.python.org/3/reference/datamodel.html#context-managers).
   - All objects which [inherit](https://docs.python.org/3/tutorial/classes.html#inheritance) from [IOBase](https://docs.python.org/3/library/io.html#io.IOBase) support this.
   - [`BufferedIOBase`](https://docs.python.org/3/library/io.html#io.BufferedIOBase), [`BufferedWriter`](https://docs.python.org/3/library/io.html#io.BufferedWriter), and [`GzipFile`](https://docs.python.org/3/library/gzip.html#gzip.GzipFile) all support this.
   - Ensures `.close()` is called in both exception and regular cases.
1. Ensure [`.close()`](https://docs.python.org/3/library/io.html#io.IOBase.close) is always called which flushes data before closing.
1. If the underlying stream need to be kept open, use [`.detach()`](https://docs.python.org/3/library/io.html#io.BufferedIOBase.detach)

Since 3.12 flushing has been necessary in GzipFile (see pythongh-105808 which was a release blocker), this makes that more visible. Users have been encountering as they upgrade to 3.12 (ex. pythongh-129726).

There are a number of cases of unclosed file-like objects being deleted in CPython libraries and the test suite. This issue includes resolving those cases where the new ResourceWarning is emitted.
@bedevere-app bedevere-app Bot added the awaiting review label Mar 6, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Mar 6, 2025
Emit ResourceWarning when GzipFile is deleted with unwritten data #130806
Closed
vstinner
vstinner reviewed Mar 6, 2025
View reviewed changes
Comment thread Lib/gzip.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/gzip.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@cmaloney @vstinner
Apply suggestions from code review
da8a021
Co-authored-by: Victor Stinner <[email protected]>
vstinner
vstinner approved these changes Mar 7, 2025
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

cmaloney reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Mar 7, 2025
@vstinner

vstinner commented Mar 7, 2025

Copy link
Copy Markdown
Member

cc @serhiy-storchaka @pitrou

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
vstinner merged commit 93089c0 into python:main Mar 13, 2025
@bedevere-app bedevere-app Bot removed the awaiting merge label Mar 13, 2025
@vstinner

vstinner commented Mar 13, 2025

Copy link
Copy Markdown
Member

Merged, thank you.

cmaloney reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@cmaloney
cmaloney deleted the gzipwarning branch March 14, 2025 07:41
plashchynski pushed a commit to plashchynski/cpython that referenced this pull request Mar 17, 2025
@cmaloney @vstinner @plashchynski
pythongh-130806: Emit ResourceWarning if GzipFile unclosed (python#13…
0158b99
…0905)

This may indicate accidental data loss.

Ways to make sure all data is written:
1. Use the [file-like object](https://docs.python.org/3/glossary.html#term-file-object) as a [“With Statement Context Manager”](https://docs.python.org/3/reference/datamodel.html#context-managers).
   - All objects which [inherit](https://docs.python.org/3/tutorial/classes.html#inheritance) from [IOBase](https://docs.python.org/3/library/io.html#io.IOBase) support this.
   - [`BufferedIOBase`](https://docs.python.org/3/library/io.html#io.BufferedIOBase), [`BufferedWriter`](https://docs.python.org/3/library/io.html#io.BufferedWriter), and [`GzipFile`](https://docs.python.org/3/library/gzip.html#gzip.GzipFile) all support this.
   - Ensures `.close()` is called in both exception and regular cases.

2. Ensure [`.close()`](https://docs.python.org/3/library/io.html#io.IOBase.close) is always called which flushes data before closing.

3. If the underlying stream need to be kept open, use [`.detach()`](https://docs.python.org/3/library/io.html#io.BufferedIOBase.detach)

Since 3.12 flushing has been necessary in GzipFile (see pythongh-105808 which was a release blocker), this makes that more visible. Users have been encountering as they upgrade to 3.12 (ex. pythongh-129726).

There are a number of cases of unclosed file-like objects being deleted in CPython libraries and the test suite. This issue includes resolving those cases where the new ResourceWarning is emitted.

Co-authored-by: Victor Stinner <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@cmaloney @vstinner
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.