Description

markshannon

opened on Jan 6, 2026

Issue body actions

Bug report

Bug description:

>>> def f(s):
...     return (x for x in s)
...     
>>> f(1)
<generator object f.<locals>.<genexpr> at 0xe8529f655c70>

According to the docs (and in earlier versions) the correct behavior is:

>>> def f(s):
...     return (x for x in s)
...     
>>> f(1)
TypeError: 'int' object is not iterable

The current behavior on main is arguably more logical as it is consistent with generators, but Hyrum's law dictates we need to be backwards compatible.

https://docs.python.org/3/reference/expressions.html#generator-expressions

This issue has a bit of history.
To fix #125038 we moved the GET_ITER instruction into the generator body. This broke stuff, so we added a supposedly redundant GET_ITER to the callsite, but that also caused problems #127682.
Eventually, for 3.14 we added an additional check in FOR_ITER that the iterator is indeed an iterator.
This adds a little bit of unnecessary overhead and is an obstacle to optimization.

On main, we use "virtual iterators" and have kept the GET_ITER in the body of the generator. This is efficient and optimizes nicely, but doesn't conform to the docs.

We can keep both the nice performance and conform to the docs, by keeping the GET_ITER in the generator function, but moving it into generator creation and out of the generator body.
Instead of:

RETURN_GENERATOR
...
LOAD__FAST 0
GET_ITER

we have:

LOAD_FAST 0
GET_ITER
RETURN_GENERATOR
...

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

• GH-143493: Conform to spec for generator expressions while supporting virtual iterators #143569
• gh-143493: fix cleanup on errors in codegen_comprehension #156374
• [3.15] gh-143493: fix cleanup on errors in codegen_comprehension (GH-156374) #156962

Activity

1. markshannon

   self-assigned this

   on Jan 6, 2026
2. markshannon

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   3.15bugs and security fixesbugs and security fixes

   on Jan 6, 2026
3. Yashp002 commented on Jan 6, 2026

   

   Yashp002

   on Jan 6, 2026

   Contributor

   More actions

   just so I understand the proposed bytecode flow correctly: the plan is to have GET_ITER execute before RETURN_GENERATOR, which restores the eager TypeError while preserving the optimized frame behavior for the rest of the generator body.

   If that's the case, I'd be happy to write a regression test case for this eager failure behavior to help verify the fix once it lands.
4. picnixz

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   on Jan 6, 2026
5. picnixz commented on Jan 6, 2026

   

   picnixz

   on Jan 6, 2026

   Member

   More actions

   If that's the case, I'd be happy to write a regression test case for this eager failure behavior to help verify the fix once it lands.

   If a fix lands, the regression test would be written at the same time. We don't separate tests from fixes. I also don't think it's an issue for newcomers considering the long history of how generator expressions behave.
6. markshannon

   changed the title [-]Behavior of generator expressions doesn't match the docs when passes a non-iterable sequence.[/-] [+]Behavior of generator expressions doesn't match the docs when passed a non-iterable[/+] on Jan 7, 2026
7. sobolevn commented on Jan 9, 2026

   

   sobolevn

   on Jan 9, 2026

   Member

   More actions

   cc @efimov-mikhail
8. markshannon

   added a commit that references this issue on Jan 16, 2026

   GH-143493: Conform to spec for generator expressions while supporting…

   ae53da5
9. thunder-coding

   added a commit that references this issue on Feb 15, 2026

   pythonGH-143493: Conform to spec for generator expressions while supp…

   f2185b1
10. markshannon

    closed this as completedon Mar 9, 2026
11. iritkatriel

    added a commit that references this issue on Sep 4, 2026

    gh-143493: fix cleanup on errors in codegen_comprehension (#156374)

    e56f86f
12. clin1234

    added a commit that references this issue on Sep 12, 2026

    pythongh-143493: fix cleanup on errors in codegen_comprehension (pyth…

    3f3f3cd
13. encukou

    added a commit that references this issue on Oct 13, 2026

    [3.15] gh-143493: fix cleanup on errors in codegen_comprehension (GH-…

    8870e26

Issue actions