Description

ionite34

opened on Feb 9, 2023

Issue body actions

Crash report

Example from @chilaxan

corrupt = iter(lambda:0, 0)

class Cstr:
    def __hash__(self):
        return hash('iter')
    def __eq__(self, other):
        [*corrupt]
        return other == 'iter'

builtins = __builtins__.__dict__ if hasattr(__builtins__, '__dict__') else __builtins__
oiter = builtins['iter']
del builtins['iter']
builtins[Cstr()] = oiter

print(corrupt.__reduce__())

Expected result

This should return a valid __reduce__ tuple of the exhausted iterator. Instead behavior is inconsistent between segmentation faults, SystemErrors, and sometimes returning the iterator without being exhausted.

Error messages

• 3.11, windows, PYTHONMALLOC=debug
• 3.12.0a4, windows, PYTHONMALLOC=debug

Windows fatal exception: access violation
> exit code -1073741819 (0xC0000005)

• 3.12.04a4, windows, compiled with debug mode

    print(corrupt.__reduce__())
          ^^^^^^^^^^^^^^^^^^^^
SystemError: NULL object passed to Py_BuildValue

• 3.11, ubuntu

(<built-in function iter>, (<function  at 0x7fb772c3c4a0>, 0))
> terminated by signal SIGSEGV (Address boundary error)

• 3.12.0a4, ubuntu

(<built-in function iter>, (<function  at 0x7f3480d71f80>, 0))

• 3.12.0a4, ubuntu, PYTHONMALLOC=debug

Fatal Python error: Segmentation fault

Linked PRs

• gh-101765: Fix SystemError / segmentation fault in iter __reduce__ when internal access of builtins.__dict__ exhausts the iterator #101769
• [3.11] gh-101765: Fix SystemError / segmentation fault in iter __reduce__ when internal access of builtins.__dict__ exhausts the iterator (GH-101769) #102228
• [3.10] gh-101765: Fix SystemError / segmentation fault in iter __reduce__ when internal access of builtins.__dict__ exhausts the iterator (GH-101769) #102229
• gh-101765: Fix refcount issues in list and unicode pickling #102265
• [3.11] gh-101765: Fix refcount issues in list and unicode pickling (G… #102268
• [3.10] gh-101765: Fix refcount issues in list and unicode pickling (G… #102269
• gh-101765: unicodeobject: use Py_XDECREF correctly #102283
• [3.11] gh-101765: unicodeobject: use Py_XDECREF correctly (GH-102283) #102285
• [3.10] gh-101765: unicodeobject: use Py_XDECREF correctly (GH-102283) #102286

Activity

1. ionite34

   added

   type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

   on Feb 9, 2023
2. godlygeek commented on Feb 9, 2023

   

   godlygeek

   on Feb 9, 2023

   Contributor

   More actions

   This (horrific) reproducer provokes undefined defined behavior from this statement:

   cpython/Objects/iterobject.c

   Lines 243 to 244 in f1f3af7

   	return Py_BuildValue("N(OO)", _PyEval_GetBuiltin(&_Py_ID(iter)),
   	it->it_callable, it->it_sentinel);

   The call to _PyEval_GetBuiltin to find the iter builtin is calling Cstr.__eq__, which exhausts the iterator, causing the Py_CLEAR in calliter_iternext to be executed, setting it->it_callable and it->it_sentinel to NULL. But the order of evaluation of arguments in a function call isn't specified, and modifying an argument by evaluating another argument is a bug.

   On some platforms, it->it_callable and it->it_sentinel are being evaluated before _PyEval_GetBuiltin, and so Py_BuildValue is being passed pointers to objects that got freed inside the _PyEval_GetBuiltin call.

   Reacted by Ionite
3. ionite34

   changed the title [-]`iter.__reduce__` can segfault if accessing `__builtins__.__dict__['iter']` exhausts the iter object[/-] [+]iter `__reduce__` can segfault if accessing `__builtins__.__dict__['iter']` mutates the iter object[/+] on Feb 10, 2023
4. arhadthedev

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   on Feb 10, 2023
5. ionite34

   added a commit that references this issue on Feb 10, 2023

   Add iter reduce tests for issue python#101765

   c5abb14
6. JelleZijlstra

   added a commit that references this issue on Feb 24, 2023

   gh-101765: Fix SystemError / segmentation fault in iter `__reduce__` …

   54dfa14
7. ionite34

   added 3 commits that reference this issue on Feb 25, 2023

   [3.11] pythongh-101765: Fix SystemError / segmentation fault in iter …

   a9cd908

   [3.11] pythongh-101765: Fix SystemError / segmentation fault in iter …

   297d01e

   [3.10] pythongh-101765: Fix SystemError / segmentation fault in iter …

   031382c
8. JelleZijlstra

   added 10 commits that reference this issue on Feb 25, 2023

   [3.11] gh-101765: Fix SystemError / segmentation fault in iter `__red…

   5d46122

   [3.10] gh-101765: Fix SystemError / segmentation fault in iter `__red…

   9f472f8

   pythongh-101765: Fix refcount issues in list and unicode pickling

   b8ad59e

   gh-101765: Fix refcount issues in list and unicode pickling (#102265)

   d71edbd

   [3.11] pythongh-101765: Fix refcount issues in list and unicode pickl…

   e97ddc6

   [3.10] pythongh-101765: Fix refcount issues in list and unicode pickl…

   fca504e

   [3.10] gh-101765: Fix refcount issues in list and unicode pickling (G…

   6fa6c2a

   [3.11] gh-101765: Fix refcount issues in list and unicode pickling (G…

   b36c498

   pythongh-101765: unicodeobject: use Py_XDECREF correctly

   a064bfa

   gh-101765: unicodeobject: use Py_XDECREF correctly (#102283)

   8d0f09b
9. miss-islington

   added 4 commits that reference this issue on Feb 26, 2023

   pythongh-101765: unicodeobject: use Py_XDECREF correctly (pythonGH-10…

   2ebc1f6

   pythongh-101765: unicodeobject: use Py_XDECREF correctly (pythonGH-10…

   cda755b

   gh-101765: unicodeobject: use Py_XDECREF correctly (GH-102283)

   64d3715

   gh-101765: unicodeobject: use Py_XDECREF correctly (GH-102283)

   4cc3636
10. carljm

    added a commit that references this issue on Feb 28, 2023

    Merge branch 'main' into inlinecomp2

    73dc0ed
11. ionite34

    closed this as completedon Mar 11, 2023
12. JelleZijlstra

    added a commit that references this issue on Sep 10, 2024

    pythongh-101765: Fix SystemError / segmentation fault in iter `__redu…

    473a891

Issue actions