Description

ericsnowcurrently

opened on May 20, 2024

Issue body actions

Bug report

Bug description:

@1st1 encountered some crashes due to this.

CPython versions tested on:

3.12, 3.13

Operating systems tested on:

No response

Linked PRs

• [3.12] gh-119213: Fix getargs.c to store state in InterpreterState... #119194
• gh-119213: Fix getargs.c to store state in InterpreterState... #119195
• gh-119213: Be More Careful About _PyArg_Parser.kwtuple Across Interpreters #119331
• [3.13] gh-119213: Be More Careful About _PyArg_Parser.kwtuple Across Interpreters (gh-119331) #119410
• [3.12] gh-119213: Be More Careful About _PyArg_Parser.kwtuple Across Interpreters (gh-119331) #119425

Activity

1. ericsnowcurrently

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   topic-subinterpreters

   3.12only security fixesonly security fixes

   3.13only security fixesonly security fixes

   3.14bugs and security fixesbugs and security fixes

   on May 20, 2024
2. github-project-automation

   added this to Subinterpreterson May 20, 2024
3. github-project-automation

   moved this to Todo in Subinterpreterson May 20, 2024
4. ericsnowcurrently commented on May 21, 2024

   

   ericsnowcurrently

   on May 21, 2024

   MemberAuthor

   More actions

   Here's what I've found:

   • there's a bug (in 3.13/main) where _PyRuntime.getargs.static_parsers is only ever set to the last node added by _parser_init()
   • for non-builtin modules (even for Py_BUILD_CORE_MODULE) _PyArg_Parser.kwtuple is allocated on the heap under the current interpreter

   The first bug masks the problem on 3.13/main, so thankfully @1st1 was testing on 3.12.

   The second bug is solvable by either always statically allocating the kwtuple or by always temporarily switching to the main interpreter when allocating from the heap.

   Now that I can reliably reproduce the crash I'll have a fix up soon.

   Reacted by Erlend E. Aasland
5. ericsnowcurrently commented on May 21, 2024

   

   ericsnowcurrently

   on May 21, 2024

   MemberAuthor

   More actions

   One other note:

   If I destroy the interpreter before runtime finalization I hit an assertion about negative refcount:

   ./Include/object.h:1040: _Py_NegativeRefcount: Assertion failed: object has negative ref count
   <object at 0x7f6922fd5800 is freed>
   Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
   Python runtime state: finalizing (tstate=0x000055d542ee5cb0)
   
   Current thread 0x00007f692773b100 (most recent call first):
     <no Python frame>
   Aborted (core dumped)
   

   If I do not destroy the subinterpreter before runtime fini then it crashes due to the dangling pointer:

   Debug memory block at address p=0x7f06a7ad4620: API ''
       71492436437630461 bytes originally requested
       The 7 pad bytes at p-7 are not all FORBIDDENBYTE (0xfd):
           at p-7: 0x00 *** OUCH
           at p-6: 0x00 *** OUCH
           at p-5: 0x00 *** OUCH
           at p-4: 0x00 *** OUCH
           at p-3: 0x00 *** OUCH
           at p-2: 0x00 *** OUCH
           at p-1: 0x00 *** OUCH
       Because memory is corrupted at the start, the count of bytes requested
          may be bogus, and checking the trailing pad bytes may segfault.
       The 8 pad bytes at tail=0xfe7d04a5ab441d are Segmentation fault (core dumped)
   
6. ericsnowcurrently

   added a commit that references this issue on May 22, 2024

   gh-119213: Be More Careful About _PyArg_Parser.kwtuple Across Interpr…

   8186500
7. miss-islington

   added a commit that references this issue on May 22, 2024

   pythongh-119213: Be More Careful About _PyArg_Parser.kwtuple Across I…

   05a00aa
8. ericsnowcurrently

   added 2 commits that reference this issue on May 22, 2024

   [3.13] gh-119213: Be More Careful About _PyArg_Parser.kwtuple Across …

   bfd9c3e

   pythongh-119213: Be More Careful About _PyArg_Parser.kwtuple Across I…

   c4bf08b
9. ericsnowcurrently commented on May 22, 2024

   

   ericsnowcurrently

   on May 22, 2024

   MemberAuthor

   More actions

   FTR, original change: gh-95860
10. ericsnowcurrently

    added a commit that references this issue on May 22, 2024

    [3.12] gh-119213: Be More Careful About _PyArg_Parser.kwtuple Across …

    0d5fe2c
11. ericsnowcurrently commented on May 23, 2024

    

    ericsnowcurrently

    on May 23, 2024

    MemberAuthor

    More actions

    Thanks for identifying the bug here, @1st1. Let me know if you notice any problems with the fix.
12. ericsnowcurrently

    closed this as completedon May 23, 2024
13. github-project-automation

    moved this from Todo to Done in Subinterpreterson May 23, 2024
14. estyxx

    added a commit that references this issue on Jul 17, 2024

    pythongh-119213: Be More Careful About _PyArg_Parser.kwtuple Across I…

    712aaf2

Issue actions