GeezerVison is showing github.com — Crash in `_io_TextIOWrapper_write_impl` when running `tracemalloc.clear_traces` in threads · Issue #144763 · python/cpython
Crash in _io_TextIOWrapper_write_impl when running tracemalloc.clear_traces in threads #144763DescriptionCrash report
What happened?
Running the code below will non-deterministically cause a segfault or abort in _io_TextIOWrapper_write_impl in the !PyList_CheckExact(self->pending_bytes) assertion.
from sys import stdout
from threading import Thread
from time import sleep
from tracemalloc import clear_traces, start, get_traceback_limit
def print_to_file(text):
get_traceback_limit()
stdout.write("")
clear_traces()
sleep(0.04)
clear_traces()
alive = []
start()
for i in range(300):
alive.append(Thread(target=print_to_file, args=("a" * 800,)))
for t in alive:
t.start()
print(object)Segfault backtrace
Thread 143 "Thread-142 (pri" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7bfec37d66c0 (LWP 1476230)]
0x000055555622f434 in _Py_TYPE_impl (ob=0x0) at ./Include/object.h:313
313 return ob->ob_type;
#0 0x000055555622f434 in _Py_TYPE_impl (ob=0x0) at ./Include/object.h:313
#1 _Py_IS_TYPE_impl (ob=0x0, type=<optimized out>) at ./Include/object.h:328
#2 _io_TextIOWrapper_write_impl (self=0x7bffb4361110, text=<optimized out>) at ./Modules/_io/textio.c:1765
#3 _io_TextIOWrapper_write (self=0x7bffb4361110, arg=0x5555569275c8 <_PyRuntime+96968>) at ./Modules/_io/clinic/textio.c.h:803
#4 0x0000555555ecb860 in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at Python/generated_cases.c.h:2499
#5 0x0000555555e96854 in _PyEval_EvalFrame (tstate=0x7e8ff7576210, frame=0x7e8ff7404328, throwflag=0) at ./Include/internal/pycore_ceval.h:118
#6 _PyEval_Vector (tstate=0x7e8ff7576210, func=0x7bffb449b7d0, locals=0x0, args=0x7bff33fcf520, argcount=1, kwnames=0x0) at Python/ceval.c:2125
#7 0x0000555555ac3110 in _PyObject_VectorcallTstate (tstate=tstate@entry=0x7e8ff7576210, callable=callable@entry=0x7bffb449b7d0, args=0x7bff33fcf520, nargsf=nargsf@entry=1,
kwnames=kwnames@entry=0x0) at ./Include/internal/pycore_call.h:136
#8 0x0000555555ac080a in method_vectorcall (method=0x7bff440d0070, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/classobject.c:74
#9 0x0000555555f582b3 in _PyObject_VectorcallTstate (tstate=0x7e8ff7576210, callable=0x7bff440d0070, args=0x7bff440e00e7, nargsf=17042036080667, kwnames=0x0)
at ./Include/internal/pycore_call.h:136
#10 context_run (self=0x7bffb4d82770, args=0x7bfec37d47b0, nargs=1, kwnames=0x0) at Python/context.c:727
#11 0x0000555555e997da in _PyCallMethodDescriptorFastWithKeywords_StackRefSteal (callable=callable@entry=..., meth=0x555556740560 <PyContext_methods+160>, self=self@entry=0x7bffb4d82770,
arguments=0x7e8ff74042f8, total_args=total_args@entry=2) at Python/ceval.c:913
#12 0x0000555555ec9cad in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at Python/generated_cases.c.h:3852
#13 0x0000555555e96854 in _PyEval_EvalFrame (tstate=0x7e8ff7576210, frame=0x7e8ff7404220, throwflag=0) at ./Include/internal/pycore_ceval.h:118
#14 _PyEval_Vector (tstate=0x7e8ff7576210, func=0x7bffb449b8b0, locals=0x0, args=0x7bff33fcd120, argcount=1, kwnames=0x0) at Python/ceval.c:2125
#15 0x0000555555ac3110 in _PyObject_VectorcallTstate (tstate=tstate@entry=0x7e8ff7576210, callable=callable@entry=0x7bffb449b8b0, args=0x7bff33fcd120, nargsf=nargsf@entry=1,
kwnames=kwnames@entry=0x0) at ./Include/internal/pycore_call.h:136
#16 0x0000555555ac080a in method_vectorcall (method=0x7bffb457ff70, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/classobject.c:74
#17 0x0000555555ab955f in _PyVectorcall_Call (tstate=<optimized out>, func=<optimized out>, callable=<optimized out>, tuple=0x55555693a1f8 <_PyRuntime+173816>, kwargs=<optimized out>)
at Objects/call.c:273
#18 0x00005555562b0b59 in thread_run (boot_raw=boot_raw@entry=0x7c6ff6fe83e0) at ./Modules/_threadmodule.c:387
#19 0x00005555560f8a26 in pythread_wrapper (arg=0x7c3ff70fa0a0) at Python/thread_pthread.h:234
#20 0x000055555589b34b in asan_thread_start(void*) ()
#21 0x00007ffff7ca3d64 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:448
#22 0x00007ffff7d373fc in __GI___clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
Abort backtrace
python: ./Modules/_io/textio.c:1765: PyObject *_io_TextIOWrapper_write_impl(textio *, PyObject *): Assertion `!PyList_CheckExact(self->pending_bytes)' failed.
Thread 65 "Thread-64 (prin" received signal SIGABRT, Aborted.
[Switching to Thread 0x7bff14a596c0 (LWP 1478715)]
#0 __pthread_kill_implementation (threadid=<optimized out>, signo=6, no_tid=0) at ./nptl/pthread_kill.c:44
#1 __pthread_kill_internal (threadid=<optimized out>, signo=6) at ./nptl/pthread_kill.c:89
#2 __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:100
#3 0x00007ffff7c45e2e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4 0x00007ffff7c28888 in __GI_abort () at ./stdlib/abort.c:77
#5 0x00007ffff7c287f0 in __assert_fail_base (fmt=<optimized out>, assertion=<optimized out>, file=<optimized out>, line=<optimized out>, function=<optimized out>) at ./assert/assert.c:118
#6 0x00007ffff7c3c19f in __assert_fail (assertion=<optimized out>, file=<optimized out>, line=<optimized out>, function=<optimized out>) at ./assert/assert.c:127
#7 0x000055555622f965 in _io_TextIOWrapper_write_impl (self=0x7bffb4361110, text=<optimized out>) at ./Modules/_io/textio.c:1765
#8 _io_TextIOWrapper_write (self=0x7bffb4361110, arg=0x5555569275c8 <_PyRuntime+96968>) at ./Modules/_io/clinic/textio.c.h:803
#9 0x0000555555ecb860 in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at Python/generated_cases.c.h:2499
#10 0x0000555555e96854 in _PyEval_EvalFrame (tstate=0x7e8ff741d210, frame=0x7e8ff718e328, throwflag=0) at ./Include/internal/pycore_ceval.h:118
#11 _PyEval_Vector (tstate=0x7e8ff741d210, func=0x7bffb449b7d0, locals=0x0, args=0x7bff98169a20, argcount=1, kwnames=0x0) at Python/ceval.c:2125
#12 0x0000555555ac3110 in _PyObject_VectorcallTstate (tstate=tstate@entry=0x7e8ff741d210, callable=callable@entry=0x7bffb449b7d0, args=0x7bff98169a20, nargsf=nargsf@entry=1,
kwnames=kwnames@entry=0x0) at ./Include/internal/pycore_call.h:136
#13 0x0000555555ac080a in method_vectorcall (method=0x7bffda0d0070, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/classobject.c:74
#14 0x0000555555f582b3 in _PyObject_VectorcallTstate (tstate=0x7e8ff741d210, callable=0x7bffda0d0070, args=0x7bff14a596c0, nargsf=22, kwnames=0x0) at ./Include/internal/pycore_call.h:136
#15 context_run (self=0x7bffb4d80850, args=0x7bff14a577b0, nargs=1, kwnames=0x0) at Python/context.c:727
#16 0x0000555555e997da in _PyCallMethodDescriptorFastWithKeywords_StackRefSteal (callable=callable@entry=..., meth=0x555556740560 <PyContext_methods+160>, self=self@entry=0x7bffb4d80850,
arguments=0x7e8ff718e2f8, total_args=total_args@entry=2) at Python/ceval.c:913
#17 0x0000555555ec9cad in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at Python/generated_cases.c.h:3852
#18 0x0000555555e96854 in _PyEval_EvalFrame (tstate=0x7e8ff741d210, frame=0x7e8ff718e220, throwflag=0) at ./Include/internal/pycore_ceval.h:118
#19 _PyEval_Vector (tstate=0x7e8ff741d210, func=0x7bffb449b8b0, locals=0x0, args=0x7bff98168120, argcount=1, kwnames=0x0) at Python/ceval.c:2125
#20 0x0000555555ac3110 in _PyObject_VectorcallTstate (tstate=tstate@entry=0x7e8ff741d210, callable=callable@entry=0x7bffb449b8b0, args=0x7bff98168120, nargsf=nargsf@entry=1,
kwnames=kwnames@entry=0x0) at ./Include/internal/pycore_call.h:136
#21 0x0000555555ac080a in method_vectorcall (method=0x7bffb44e2a10, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/classobject.c:74
#22 0x0000555555ab955f in _PyVectorcall_Call (tstate=<optimized out>, func=<optimized out>, callable=<optimized out>, tuple=0x55555693a1f8 <_PyRuntime+173816>, kwargs=<optimized out>)
at Objects/call.c:273
#23 0x00005555562b0b59 in thread_run (boot_raw=boot_raw@entry=0x7c6ff6fe61c0) at ./Modules/_threadmodule.c:387
#24 0x00005555560f8a26 in pythread_wrapper (arg=0x7c3ff70db460) at Python/thread_pthread.h:234
#25 0x000055555589b34b in asan_thread_start(void*) ()
#26 0x00007ffff7ca3d64 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:448
#27 0x00007ffff7d373fc in __GI___clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78)
Affected code:
|
else if (!PyList_CheckExact(self->pending_bytes)) { |
|
PyObject *list = PyList_New(2); |
|
if (list == NULL) { |
|
Py_DECREF(b); |
|
return NULL; |
|
} |
|
// Since Python 3.12, allocating GC object won't trigger GC and release |
|
// GIL. See https://github.com/python/cpython/issues/97922 |
|
assert(!PyList_CheckExact(self->pending_bytes)); |
|
PyList_SET_ITEM(list, 0, self->pending_bytes); |
|
PyList_SET_ITEM(list, 1, b); |
|
self->pending_bytes = list; |
|
} |
Found using fusil by @vstinner.
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
Python 3.15.0a6+ free-threading build (heads/main:b488f338cf0, Feb 12 2026, 17:28:06) [Clang 21.1.2 (2ubuntu6)]
Linked PRs
Activityvstinner commented on Feb 13, 2026 _io.TextIOWrapper methods are protected by critical sections. Example with write:
/*[clinic input]
@critical_section
_io.TextIOWrapper.write
It's used to make these methods atomic. For example, it's safe to use self->pending_bytes without holding a dedicated lock on self->pending_bytes.
Then comes tracemalloc which injects hooks on memory allocations. _io.TextIOWrapper.write() calls _io__Buffered_flush() which calls PyMemoryView_FromBuffer() which calls indirectly PyObject_Malloc(). tracemalloc has a hook on PyObject_Malloc() and calls its tracemalloc_alloc() function which calls tracemalloc TABLES_LOCK(). If another thread is already holding the thread, TABLES_LOCK() can call _PyThreadState_Detach() which suspends the current critical section: TextIOWrapper critical section!
So while tracemalloc is running, it's possible that it unlocks the TextIOWrapper critical section and another thread can call a TextIOWrapper method concurrently!
In short, it's possible that _io_TextIOWrapper_write() runs in two threads in parallel, and so it's no longer safe to use self->pending_bytes without a dedicated lock.
Example from Modules/_io/textio.c:
else if (!PyList_CheckExact(self->pending_bytes)) {
PyObject *list = PyList_New(2);
if (list == NULL) {
Py_DECREF(b);
return NULL;
}
// Since Python 3.12, allocating GC object won't trigger GC and release
// GIL. See https://github.com/python/cpython/issues/97922
assert(!PyList_CheckExact(self->pending_bytes));
PyList_SET_ITEM(list, 0, self->pending_bytes);
PyList_SET_ITEM(list, 1, b);
self->pending_bytes = list;
}The comment is interesting. When using tracemalloc, the comment is no longer true: allocation memory can release the GIL and suspend the critical section.
I'm not sure how to solve this problem. vstinner commented on Feb 13, 2026 Simpler reproducer:
from sys import stdout
from threading import Thread
from time import sleep
import tracemalloc
def print_to_file():
stdout.write(".")
sleep(0.04)
alive = []
tracemalloc.start()
for i in range(300):
alive.append(Thread(target=print_to_file))
for t in alive:
t.start()
stdout.write("long string in main thread\n")vstinner commented on Feb 13, 2026 Ah, to be clear: this issue only affects free-threaded builds of Python. vstinner commented on Feb 18, 2026 The issue is fixed in the main branch. Thanks for your bug report! vstinner commented on Feb 18, 2026 I fail to reproduce the issue in the 3.13 branch: it's not affected by the issue. In 3.13, TABLES_LOCK() is implemented as PyThread_acquire_lock(tables_lock, 1): it doesn't detach the thread state.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|