GeezerVison is showing github.com — ValueError: I/O operation on closed file. in ZipFile destructor · Issue #81954 · python/cpython
ValueError: I/O operation on closed file. in ZipFile destructor #81954Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2019-08-06.09:35:50.904>
labels = ['3.10', '3.9', 'type-crash', '3.11']
title = 'ValueError: I/O operation on closed file. in ZipFile destructor'
updated_at = <Date 2022-01-23.20:02:33.133>
user = 'https://github.com/joernheissler' bugs.python.org fields:
activity = <Date 2022-01-23.20:02:33.133>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2019-08-06.09:35:50.904>
creator = 'joernheissler'
dependencies = []
files = []
hgrepos = []
issue_num = 37773
keywords = ['3.8regression']
message_count = 10.0
messages = ['349104', '349105', '389594', '389605', '389611', '389689', '389690', '389691', '396649', '396657']
nosy_count = 6.0
nosy_names = ['pitrou', 'vstinner', 'serhiy.storchaka', 'joernheissler', 'xtreak', 'andrei.avk']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue37773'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11'] Linked PRs
Activityjoernheissler commented on Aug 6, 2019 MannequinAuthor More actions When running this code:
from zipfile import ZipFile
import io
def foo():
pass
data = io.BytesIO()
zf = ZipFile(data, "w")I get this message:
Exception ignored in: <function ZipFile.__del__ at 0x7f9005caa160>
Traceback (most recent call last):
File "/home/user/git/oss/cpython/Lib/zipfile.py", line 1800, in __del__
File "/home/user/git/oss/cpython/Lib/zipfile.py", line 1817, in close
ValueError: I/O operation on closed file. Comment out def foo: pass, and there is no error.
It looks like the bug was introduced with commit ada319b (bpo-32388). tirkarthi commented on Aug 6, 2019 This looks like a regression in 3.8 so I have added 3.8 regression tag. joernheissler commented on Mar 27, 2021 MannequinAuthor More actions Still reproducible in cpython 3.10.0a3 (debian unstable) and 3.10.0a6 (pyenv). serhiy-storchaka commented on Mar 27, 2021 That's what happened. Function foo creates a reference loop. It has reference to the module dict, and the dict has reference to the function. The dict has also references to BytesIO and ZipFile objects. At shutdown stage the garbage collector is called. It founds the reference loop which contains the module dict, function foo and BytesIO and ZipFile objects. It tries to break the loop by calling the destructor of arbitrary object in the loop. In that case it is the BytesIO object. It does not help, but finally the destructor of the ZipFile object is called (either directly or after breaking the loop and destroying the module dict). The ZipFile destructor tries to close the BytesIO object, but it is already closed in its destructor.
It is a dangerous situation which potentially can cause data corruption (if the underlying stream is closed before flushing buffers). Clearing the module dict before invoking garbage collecting would solve this issue. Making the garbage collector more clever and and avoid destroying objects if it does not break the loop would help too, but it is more complex problem and such change could not be backported to 3.8.
Victor, were there any changes in the garbage collector or interpreter shutdown code in 3.8? joernheissler commented on Mar 27, 2021 MannequinAuthor More actions Thanks Serhiy for the explanation, it's making sense now.
Guess whatever I did back then (no idea what I was working on) was basically a mistake; I should have closed my ZipFile properly, e.g. by using context managers. So maybe it's not really a cpython bug.
I ran a git-bisect back then and came up with https://hg.python.org/lookup/ada319bb6d0ebcc68d3e0ef2b4279ea061877ac8 as cause. vstinner commented on Mar 29, 2021
Victor, were there any changes in the garbage collector or interpreter shutdown code in 3.8?
I took some notes there: https://pythondev.readthedocs.io/finalization.html
I proposed bpo-42671 "Make the Python finalization more deterministic" but it will break some use cases, so I'm not comfortable with this idea yet :-( vstinner commented on Mar 29, 2021 IMO ZipFile destructor should emit a ResourceWarning if it's not closed explicitly.
Nothing in the Python specification gives you any warranty that destructors will be ever called... Depending on the Python implementation and depending on many things, you never know when a destructor is called.
Just call the close() method explicitly. vstinner commented on Mar 29, 2021 FYI TextIOWrapper has a similar issue: bpo-17852. There was an attempt to fix the issue in 2017, but it had to be reverted. akulakov commented on Jun 28, 2021 In my testing on 3.9.1 and the current main branch, I found:
- I can reproduce this issue if a function or a lambda or class with a method are defined.
- There is no error if a class is defined with no methods. (I'm not sure why).
The test code runs without errors if the following check is done before seeking the fp in close():
here:
https://github.com/python/cpython/blob/main/Lib/zipfile.py#L1823
The fix also requires adding closed property to zipfile._Tellable
With these two small changes, all test_zipfile tests pass.
I'm not sure this is the proper fix, because this is a complex issue..
I will go ahead and put up a PR if this seems like a good approach. akulakov commented on Jun 28, 2021 I forgot to add, to fix the tests I also had to add closed=False attr to test_zipfile.Tellable. serhiy-storchaka commented on May 18, 2026 This issue is not specific for zipfile. You will get similar symptoms for buffered files with something in the buffer, and for any object with non-trivial destructor. This is how the garbage collector works. More smart garbage collector could see that the BytesIO should not be destroyed before the ZipFile which refers to it. It could also see that destroying the BytesIO and the ZipFile will not help to break the loop cause by the function. But this is a complex issue.
Recommendation: always use ZipFile as the content manager, or ensure that close() is always called (in the finally block if necessary). Do not rely on the garbage collector, it can destroy object in wrong order. Reacted by Victor Stinner and Cody Maloney cmaloney commented on Jun 27, 2026 I Created #152354 to raise ResourceWarning on this case in the same way that GzipFile and TextIOWrapper do which should make the source(s) more straightforward to find with tracemalloc. vstinner commented on Aug 7, 2026 The zipfile module now emits a ResourceWarning if a zipfile object is not closed explicitly and it's open in write mode. I close the issue. added a commit that references this issue on Oct 23, 2026gh-81954: Eagerly import warnings module to failure of ZipFile.__del_…
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|