Description

devdanzin

opened on Nov 16, 2024

Issue body actions

Crash report

What happened?

It's possible to segfault or abort a no-gil interpreter running with PYTHON_GIL=0 by calling atexit functions from threads:

from threading import Thread
import atexit

def g():
    pass

def f():
    for x in range(100):
        atexit.register(g)
        atexit._clear()
        atexit.register(g)

for x in range(100):
    Thread(target=f, args=()).start()

Found using fusil by @vstinner.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.0a1+ experimental free-threading build (heads/main-dirty:612ac283b81, Nov 16 2024, 01:37:56) [GCC 11.4.0]

Linked PRs

• gh-126907: Lock the atexit state on the free-threaded build #126908
• gh-126907: Use a list for atexit callbacks #127935

Activity

1. devdanzin

   added

   type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

   on Nov 16, 2024
2. kumaraditya303 commented on Nov 16, 2024

   

   kumaraditya303

   on Nov 16, 2024

   Contributor

   More actions

   Please also post the stack trace preferably with a debug build for ease with crashers.
3. picnixz

   added

   extension-modulesC modules in the Modules dirC modules in the Modules dir

   3.13only security fixesonly security fixes

   topic-free-threading

   3.14bugs and security fixesbugs and security fixes

   on Nov 16, 2024
4. ZeroIntensity commented on Nov 16, 2024

   

   ZeroIntensity

   on Nov 16, 2024

   Member

   More actions

   At a glance, atexit just needs some locks on the interpreter state field. I can do it.
5. ZeroIntensity

   self-assigned this

   on Nov 16, 2024
6. devdanzin commented on Nov 16, 2024

   

   devdanzin

   on Nov 16, 2024

   MemberAuthor

   More actions

   Please also post the stack trace preferably with a debug build for ease with crashers.

   Sorry, will do from now on.

   I get this backtrace for a slightly modified code (because original code wasn't triggering under gdb):

   #0  atexit_delete_cb (state=state@entry=0x555555cfd8f8 <_PyRuntime+140088>, i=i@entry=0)
       at ./Modules/atexitmodule.c:58
   #1  0x000055555598d962 in atexit_cleanup (state=0x555555cfd8f8 <_PyRuntime+140088>)
       at ./Modules/atexitmodule.c:75
   #2  0x000055555598d9b6 in atexit_clear (module=<optimized out>, unused=<optimized out>)
       at ./Modules/atexitmodule.c:249
   #3  0x0000555555702405 in cfunction_vectorcall_NOARGS (
       func=<built-in method _clear of module object at remote 0x200007752c0>, args=<optimized out>,
       nargsf=<optimized out>, kwnames=<optimized out>) at Objects/methodobject.c:495
   #4  0x000055555567cc55 in _PyObject_VectorcallTstate (tstate=0x555555dc8fe0,
       callable=<built-in method _clear of module object at remote 0x200007752c0>, args=0x7ffff7c42b48,
       nargsf=9223372036854775808, kwnames=0x0) at ./Include/internal/pycore_call.h:167
   #5  0x000055555567cd74 in PyObject_Vectorcall (
       callable=callable@entry=<built-in method _clear of module object at remote 0x200007752c0>,
       args=args@entry=0x7ffff7c42b48, nargsf=<optimized out>, kwnames=kwnames@entry=0x0)
       at Objects/call.c:327
   #6  0x00005555558441f7 in _PyEval_EvalFrameDefault (tstate=tstate@entry=0x555555dc8fe0,
       frame=<optimized out>, throwflag=throwflag@entry=0) at Python/generated_cases.c.h:955
   #7  0x0000555555872978 in _PyEval_EvalFrame (throwflag=0, frame=<optimized out>, tstate=0x555555dc8fe0)
       at ./Include/internal/pycore_ceval.h:116
   #8  _PyEval_Vector (tstate=<optimized out>, func=0x20000a8b530, locals=locals@entry=0x0,
       args=0x7ffff7c42cd8, argcount=1, kwnames=<optimized out>) at Python/ceval.c:1901
   #9  0x000055555567c622 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>,
       nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:413
   #10 0x00005555556816b4 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0x7ffff7c42cd8,
       callable=<function at remote 0x20000a8b530>, tstate=0x555555dc8fe0)
       at ./Include/internal/pycore_call.h:167

   Code that caused it:

   from threading import Thread
   import atexit
   
   def g():
       pass
   
   def f():
       for x in range(100):
           atexit.register(g)
           atexit._clear()
           atexit.register(g)
           atexit.unregister(g)
           atexit._run_exitfuncs()
   
   for x in range(100):
       Thread(target=f, args=()).start()

   Also possible to get this error message:

   Debug memory block at address p=0x20002110f90: API '�'
       9789596714021935601 bytes originally requested
       The 7 pad bytes at p-7 are not all FORBIDDENBYTE (0xfd):
           at p-7: 0xdd *** OUCH
           at p-6: 0xdd *** OUCH
           at p-5: 0xdd *** OUCH
           at p-4: 0xdd *** OUCH
           at p-3: 0xdd *** OUCH
           at p-2: 0xdd *** OUCH
           at p-1: 0xdd *** OUCH
       Because memory is corrupted at the start, the count of bytes requested
          may be bogus, and checking the trailing pad bytes may segfault.
       The 8 pad bytes at tail=0x87dba4580bace981 are 
   Thread 84 "python" received signal SIGSEGV, Segmentation fault.
   [Switching to Thread 0x7ffff6430640 (LWP 828561)]
   

   In that case the backtrace is:

   #0  0x000055555570fa78 in _PyObject_DebugDumpAddress (p=p@entry=0x20002110f90) at Objects/obmalloc.c:3016
   #1  0x000055555570fde6 in _PyMem_DebugCheckAddress (
       func=func@entry=0x555555a6c860 <__func__.8> "_PyMem_DebugRawFree", api=<optimized out>,
       p=p@entry=0x20002110f90) at Objects/obmalloc.c:2956
   #2  0x000055555571390a in _PyMem_DebugRawFree (ctx=ctx@entry=0x555555cdb928 <_PyRuntime+872>,
       p=p@entry=0x20002110f90) at Objects/obmalloc.c:2762
   #3  0x0000555555713986 in _PyMem_DebugFree (ctx=0x555555cdb928 <_PyRuntime+872>, ptr=0x20002110f90)
       at Objects/obmalloc.c:2904
   #4  0x000055555572c740 in PyMem_Free (ptr=ptr@entry=0x20002110f90) at Objects/obmalloc.c:1018
   #5  0x000055555598d93c in atexit_delete_cb (state=state@entry=0x555555cfd8f8 <_PyRuntime+140088>,
       i=i@entry=1) at ./Modules/atexitmodule.c:61
   #6  0x000055555598dc94 in atexit_unregister (module=<optimized out>,
       func=<function at remote 0x20000a8dbb0>) at ./Modules/atexitmodule.c:291
   #7  0x00005555557021ef in cfunction_vectorcall_O (
       func=<built-in method unregister of module object at remote 0x200007752c0>, args=<optimized out>,
       nargsf=<optimized out>, kwnames=<optimized out>) at Objects/methodobject.c:523
   #8  0x000055555567cc55 in _PyObject_VectorcallTstate (tstate=0x555555dc8fe0,
       callable=<built-in method unregister of module object at remote 0x200007752c0>, args=0x7ffff642fb48,
       nargsf=9223372036854775809, kwnames=0x0) at ./Include/internal/pycore_call.h:167
   #9  0x000055555567cd74 in PyObject_Vectorcall (
       callable=callable@entry=<built-in method unregister of module object at remote 0x200007752c0>,
       args=args@entry=0x7ffff642fb48, nargsf=<optimized out>, kwnames=kwnames@entry=0x0)
       at Objects/call.c:327
   #10 0x00005555558441f7 in _PyEval_EvalFrameDefault (tstate=tstate@entry=0x555555dc8fe0,
       frame=<optimized out>, throwflag=throwflag@entry=0) at Python/generated_cases.c.h:955
   #11 0x0000555555872978 in _PyEval_EvalFrame (throwflag=0, frame=<optimized out>, tstate=0x555555dc8fe0)
       at ./Include/internal/pycore_ceval.h:116

   Reacted by Peter Bierma
7. ZeroIntensity commented on Nov 16, 2024

   

   ZeroIntensity

   on Nov 16, 2024

   Member

   More actions

   Eh, stack traces are generally not that helpful on the free-threaded build (mimalloc screws up most error messages), especially in a data race.
8. kumaraditya303

   added a commit that references this issue on Dec 16, 2024

   gh-126907: make `atexit` thread safe in free-threading (#127935)

   3b76682
9. kumaraditya303

   closed this as completedon Dec 16, 2024
10. kumaraditya303

    added 2 commits that reference this issue on Dec 17, 2024

    pythongh-126907: make `atexit` thread safe in free-threading (python#…

    559e0a1

    pythongh-126907: make `atexit` thread safe in free-threading (python#…

    454d143
11. srinivasreddy

    added a commit that references this issue on Jan 8, 2025

    pythongh-126907: make `atexit` thread safe in free-threading (python#…

    458bf0f

Issue actions