Description

assignee = None
closed_at = None
created_at = <Date 2019-05-18.13:53:21.143>
labels = ['3.10', '3.9', 'type-crash', '3.11']
title = 'Calling "functions" used to implement generators/comps easily cause crash'
updated_at = <Date 2021-09-19.21:54:42.882>
user = 'https://github.com/mr-nfamous'

activity = <Date 2021-09-19.21:54:42.882>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2019-05-18.13:53:21.143>
creator = 'bup'
dependencies = []
files = []
hgrepos = []
issue_num = 36956
keywords = []
message_count = 5.0
messages = ['342797', '346807', '346808', '346809', '402171']
nosy_count = 4.0
nosy_names = ['rhettinger', 'bup', 'Jeffrey.Kintscher', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue36956'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

Activity

1. mr-nfamous commented on May 18, 2019

   

   mr-nfamousmannequin

   on May 18, 2019

   MannequinAuthor

   More actions

   As far as I know, generators, set comprehensions, list comprehensions, and dict comprehensions, (along with their asynchronous variants) are implemented by first calling the GET_(A)ITER opcode and then building and calling a function that acepts the resulting iterator as its sole argument.

   Assigning the code object used to make that function (or using it in the types.FunctionType constructor) and then calling it with a non-iterator argument will obviously cause a crash since the FOR_ITER opcode rightly expects that it will never have to deal with non-iterators and calls tp_iternext without checking if it exists.

   The 4-liner demonstrates the crash:

   if 1:
     fn = lambda: None
     gi = (i for i in ())
     fn.__code__ = gi.gi_code
     [*fn("abc")]
2. mr-nfamous

   added

   3.7 (EOL)end of lifeend of life

   3.8 (EOL)end of lifeend of life

   type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

   on May 18, 2019
3. huntertur

   added

   3.9 (EOL)end of lifeend of life

   on Jun 16, 2019
4. websurfer5 commented on Jun 28, 2019

   

   websurfer5mannequin

   on Jun 28, 2019

   Mannequin

   More actions

   What is the preferred behavior? Raise an exception of some kind?
5. websurfer5 commented on Jun 28, 2019

   

   websurfer5mannequin

   on Jun 28, 2019

   Mannequin

   More actions

   Specifically, the crash occurs in Python/ceval.c function _PyEval_EvalFrameDefault() in the TARGET(FOR_ITER) case at line 3198 (master branch):

   PyObject *next = (*iter->ob_type->tp_iternext)(iter);

   It segfaults because tp_iternext is NULL.
6. rhettinger commented on Jun 28, 2019

   

   rhettinger

   on Jun 28, 2019

   Contributor

   More actions

   ISTM this report is close to that covered by the traditional stance that word code hacks aren't protected from crashes because trying to do can be complicated, can be expensive, and isn't always possible.

   I'm not really sure where this falls on the grey scale with "normal, everyday python shouldn't crash" on one end and "with ctypes or code hacks, all bets are off" on the other end.

   Offhand, the posted snippet seems closer to "messing with the internals" but perhaps small cheap check can be added when the __code__ variable is updated.
7. iritkatriel commented on Sep 19, 2021

   

   iritkatriel

   on Sep 19, 2021

   Member

   More actions

   Reproduced on 3.11.
8. iritkatriel

   added

   3.10 (EOL)end of lifeend of life

   3.11only security fixesonly security fixes

   and removed

   3.7 (EOL)end of lifeend of life

   3.8 (EOL)end of lifeend of life

   on Sep 19, 2021
9. ezio-melotti

   transferred this issue fromon Apr 10, 2022
10. iritkatriel

    added

    3.13only security fixesonly security fixes

    and removed

    3.11only security fixesonly security fixes

    3.10 (EOL)end of lifeend of life

    3.9 (EOL)end of lifeend of life

    on Nov 7, 2023
11. iritkatriel

    added a commit that references this issue on Nov 7, 2023

    pythongh-81137: deprecate assignment of code object to a function of …

    c8b0f8d
12. iritkatriel

    closed this as completedin #111823on Nov 7, 2023
13. iritkatriel

    added a commit that references this issue on Nov 7, 2023

    gh-81137: deprecate assignment of code object to a function of a mism…

    2f9cb7e
14. hugovk

    added a commit that references this issue on Nov 8, 2023

    pythongh-81137: deprecate assignment of code object to a function of …

    143cf0f
15. aisk

    added a commit that references this issue on Feb 11, 2024

    pythongh-81137: deprecate assignment of code object to a function of …

    abeaf80
16. Glyphack

    added a commit that references this issue on Sep 2, 2024

    pythongh-81137: deprecate assignment of code object to a function of …

    4bc7879
17. davidt

    added a commit that references this issue on Nov 5, 2025

    Fix DeprecationWarning while spying on generator functions.

    d549103

Issue actions