Description

cmaloney

opened on Mar 3, 2025

Issue body actions

Feature or enhancement

Proposal:

This may indicate accidental data loss.

Ways to make sure all data is written:

1. Use the file-like object as a “With Statement Context Manager”.
   • All objects which inherit from IOBase support this.
   • BufferedIOBase, BufferedWriter, and GzipFile all support this.
   • Ensures .close() is called in both exception and regular cases.
2. Ensure .close() is always called which flushes data before closing.
3. If the underlying stream need to be kept open, use .detach()

Since 3.12 flushing has been necessary in GzipFile (see gh-105808 which was a release blocker), this makes that more visible. Users have been encountering as they upgrade to 3.12 (ex. gh-129726).

There are a number of cases of unclosed file-like objects being deleted in CPython libraries and the test suite. This issue includes resolving those cases where the new ResourceWarning is emitted.

cc: @vstinner

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

#129726 (comment)

Linked PRs

• gh-130806: emit ResourceWarning if GzipFile unclosed #130905

Activity

1. cmaloney

   added

   type-featureA feature request or enhancementA feature request or enhancement

   on Mar 3, 2025
2. encukou

   added

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   on Mar 4, 2025
3. cmaloney

   added a commit that references this issue on Mar 6, 2025

   pythongh-130806: emit ResourceWarning if GzipFile unclosed

   8bf1895
4. cmaloney commented on Mar 13, 2025

   

   cmaloney

   on Mar 13, 2025

   ContributorAuthor

   More actions

   Working on getting BufferedWriter warning to work, having a hard time as it involves adding a tp_finalize, and that interacts with the existing tp_dealloc which calls _dealloc_warn on itself and the underlying raw (often FileIO), which is how the FileIO close warning is emitted currently. I got it working well for _pyio but not sure how to get everything to fit for _io.Buffered{Writer,Random}. What I currently have. At the moment struggling to figure out whats the right tweaks, particularly for tests which seem to have often evolved separately from the original need, and this changes behavior (ex. _pyio tests using _io by accident, checks for no warning and unraisable exceptions, etc). Wondering if with tp_finalize as it currently works could replace the existing _dealloc_warn / FileIO system with a simpler one...

   While working on that, found the docs around tp_finalize pointed to the deprecated C APIs PyErr_Fetch + PyErr_Restore, made an issue + PR to update to PyErr_GetRaisedException and PyErr_SetRaisedException: gh-131117

   cc: @vstinner
5. vstinner

   added a commit that references this issue on Mar 13, 2025

   gh-130806: Emit ResourceWarning if GzipFile unclosed (#130905)

   93089c0
6. plashchynski

   added a commit that references this issue on Mar 17, 2025

   pythongh-130806: Emit ResourceWarning if GzipFile unclosed (python#13…

   0158b99
7. cmaloney

   changed the title [-]Emit ResourceWarning when GzipFile or BufferedWriter are deleted with unwritten data[/-] [+]Emit ResourceWarning when GzipFile is deleted with unwritten data[/+] on Mar 20, 2025
8. cmaloney commented on Mar 20, 2025

   

   cmaloney

   on Mar 20, 2025

   ContributorAuthor

   More actions

   Reducing the issue to just GzipFile, after a couple attempts at getting BufferedWriter working, have yet to find a reliable way. Is really neat how _dealloc_warn works with the tp_finalize in IOBase.
9. cmaloney

   closed this as completedon Mar 20, 2025

Issue actions