GeezerVison is showing github.com — Python 3.13.3 runs __iter__ twice in a list comprehension · Issue #132711 · python/cpython
Python 3.13.3 runs __iter__ twice in a list comprehension #132711Labels 3.13only security fixesonly security fixes3.14bugs and security fixesbugs and security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)pendingThe issue will be closed if no feedback is providedThe issue will be closed if no feedback is providedtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error DescriptionBug report
Bug description:
I can't upload comprehensionTestbed2.py because I am told the .py type is not allowed so I here is the script I used to reproduce this issue:
===========================================
# TestClass: Class that interacts with the DVD data file.
class TestClass:
# Initialize the file timestamp.
def __init__(self):
pass
# The iterator method.
def __iter__(self):
print('S1')
self._index = 0
return self
# The next method.
def __next__(self):
print('S5')
if self._index == 5:
raise StopIteration
value = self._index
self._index += 1
return value
# Test code.
print('#1')
for n in TestClass():
print(n)
print('#2')
numbers = [n for n in TestClass()]
print(numbers)==============================================
Here is the output of the Python script above:
D:\Files\Testing> comprehensionTestbed2.py
#1
S1
S5
0
S5
1
S5
2
S5
3
S5
4
S5
#2
S1
S1 <----- The __iter__ method is invoked twice in the list comprehension when using 3.13.3 (and 3.13.2).
S5
S5
S5
S5
S5
S5
[0, 1, 2, 3, 4]
This does not happen with python 3.12.9 and is a real problem when the iter method creates a mutex.
Edward
CPython versions tested on:
3.13
Operating systems tested on:
Windows Activitypicnixz commented on Apr 18, 2025 picnixz commented on Apr 18, 2025 Ah, could it be a duplicate of #127682? efimov-mikhail commented on Apr 19, 2025
Ah, could it be a duplicate of #127682?
I think so, this is a duplicate. fb2078 commented on Jun 19, 2025 This was fixed in 3.13.4 but is back in 3.13.5:
D:\Files\Testing> more comprehensionTestbed2.py
# TestClass: Class that interacts with the DVD data file.
class TestClass:
# Initialize the file timestamp.
def __init__(self):
pass
# The iterator method.
def __iter__(self):
print('S1')
self._index = 0
return self
# The next method.
def __next__(self):
print('S5')
if self._index == 5:
raise StopIteration
value = self._index
self._index += 1
return value
# Test code.
print(' #1')
for n in TestClass():
print(n)
print(' #2')
numbers = [n for n in TestClass()]
print(numbers)
D:\Files\Testing> comprehensionTestbed2.py
#1
S1
S5
0
S5
1
S5
2
S5
3
S5
4
S5
#2
S1
S1
S5
S5
S5
S5
S5
S5
[0, 1, 2, 3, 4]
I was in the process of switching from 3.12.9 to 3.13.4 but then I noticed 3.13.5 had been released so I tested it.
This is important to me because in production code the __iter__ method creates a lock and the script will hang when __iter__ is called the second time. Of course, I could get around that by using a counter but why is __iter__ being called more than once when I does not need to be.
Edward efimov-mikhail commented on Jun 19, 2025 fb2078 commented on Jun 19, 2025 Understood.
For my benefit, is the __iter__ method always called twice or only sometimes when in a comprehension?
Edward
…efimov-mikhail commented on Jun 20, 2025
For my benefit, is the __iter__ method always called twice or only sometimes when in a comprehension?
Only in comprehensions.
JFR, the most correct way for writing related code is to always define __iter__ method for iterator objects this way:
def __iter__(self):
return selffb2078 commented on Jun 21, 2025 That works but it seems that the __iter__ method is not really supposed do anything but just return self.
Is it possible that the issue of __iter__ being called more than once will be fixed in 3.14 or later version?
Edward
…fb2078 commented on Jun 21, 2025 Sorry, I pushed the Send too soon. I wanted to thank you for your responses and all who were involved with this issue.
Edward
…
________________________________
From: Edward Waugh ***@***.***>
Sent: Friday, June 20, 2025 6:30 PM
To: python/cpython ***@***.***>; python/cpython ***@***.***>
Cc: Author ***@***.***>
Subject: Re: [python/cpython] Python 3.13.3 runs __iter__ twice in a list comprehension (Issue #132711)
That works but it seems that the __iter__ method is not really supposed do anything but just return self.
Is it possible that the issue of __iter__ being called more than once will be fixed in 3.14 or later version?
Edward Reacted by Mikhail Efimov efimov-mikhail commented on Jun 21, 2025
That works but it seems that the iter method is not really supposed do anything but just return self.
Yes, __iter__ method should be idempotent without side effects, so calling it twice gives the same result as calling it one time.
Is it possible that the issue of __iter__ being called more than once will be fixed in 3.14 or later version?
It's already fixed on 3.14 branch, you can test on fresh beta: https://discuss.python.org/t/python-3-14-0-beta-3-is-here/95843
MetadataMetadata3.13only security fixesonly security fixes3.14bugs and security fixesbugs and security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)pendingThe issue will be closed if no feedback is providedThe issue will be closed if no feedback is providedtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error DevelopmentNo branches or pull requestsIssue actions
|