GeezerVison is showing github.com — struct.Struct crashes (seg fault) under concurrent re-initialization and unpack in free-threading builds · Issue #146020 · python/cpython
struct.Struct crashes (seg fault) under concurrent re-initialization and unpack in free-threading builds #146020DescriptionCrash report
What happened?
Description
struct.Struct.__init__() can be called on an already-initialized Struct with a different format, which frees and replaces s_codes without any lock. Concurrent calls to unpack() or pack() read s_codes without any lock, producing a seg fault.
_struct.c declares Py_MOD_GIL_NOT_USED but has zero @critical_section annotations and no atomic operations on s_codes, s_size, or s_len.
Reproducer Code
import sys
import threading
import warnings
if sys._is_gil_enabled():
sys.exit("SKIP: requires --disable-gil build")
s = struct.Struct('I')
def reader():
for _ in range(5_000_000):
try:
s.unpack(b'\x00\x00\x00\x00')
except Exception:
pass
def reinit():
with warnings.catch_warnings():
warnings.simplefilter("ignore", FutureWarning)
for _ in range(1_000_000):
s.__init__('i') # different format -> frees s_codes
s.__init__('I') # back again -> frees s_codes again
readers = [threading.Thread(target=reader) for _ in range(4)]
writers = [threading.Thread(target=reinit) for _ in range(4)]
for t in writers + readers:
t.start()
for t in writers + readers:
t.join()
print("Completed without crash.")
Test Output
[1] 24871 segmentation fault ./python3 /tmp/test_struct_race.py
Note
Re-initialization of Struct is already deprecated (FutureWarning) and will be removed in a future version, which would close this race. However
I was able to trigger the crash today.
CPython versions tested on:
3.15
Operating systems tested on:
macOS
Output from running 'python -VV' on the command line:
Python 3.15.0a7+ free-threading build (heads/main:e167e06f8c6, Mar 15 2026, 09:14:39) [Clang 17.0.0 (clang-1700.6.4.2)] ActivityZeroIntensity commented on Mar 16, 2026 Python doesn't support concurrently calling __init__ on an object. I assume you're running a fuzzer? sampsonc commented on Mar 16, 2026 Hi! I didn't use a fuzzer - I wrote the reproduction manually after reading the code and noticing _struct.c had zero critical sections despite Py_MOD_GIL_NOT_USED. sampsonc commented on Mar 16, 2026 If concurrent init is explicitly unsupported, then the crash is technically out of scope? My question is whether the broader lack of critical sections in _struct.c is itself a concern worth tracking. picnixz commented on Mar 16, 2026 cc @skirpichev I think you worked on the constructors of Structures ZeroIntensity commented on Mar 16, 2026
If concurrent init is explicitly unsupported, then the crash is technically out of scope? My question is whether the broader lack of critical sections in _struct.c is itself a concern worth tracking.
To my knowledge, all objects have no locking in __init__, because there is almost always only a single thread with access to the object anyway. Concurrently calling __init__ is not something that anyone does in practice, and since adding locking would significantly hurt performance, we don't support it. colesbury commented on Mar 16, 2026 I expect we will revisit this in the future, but for now we typically do not make concurrent __init__ calls thread-safe for the free threading. Making __init__ calls thread-safe often adds a lot of complexity or performance issues for something that people are unlikely to encounter in practice.
If you're looking for bugs, please exclude this case. skirpichev commented on Mar 16, 2026 I'm going to close this as a duplicate of the fixed issue #143715.
Note, that OP's reproducer filter out FutureWarning's, looking like:
/home/sk/src/cpython/xxx.py:22: FutureWarning: Re-initialization of Struct by calling the __init__() method will not work in future Python versions
s.__init__('i') # different format -> frees s_codes
/home/sk/src/cpython/xxx.py:23: FutureWarning: Re-initialization of Struct by calling the __init__() method will not work in future Python versions
s.__init__('I') # back again -> frees s_codes again
Completed without crash.
When the Struct became immutable, this scenario will be impossible. For now, the solution is: not do things like this.
I wrote the reproduction manually after reading the code and noticing _struct.c had zero critical sections despite Py_MOD_GIL_NOT_USED.
The point is that the reproducer is artificial, no coming from real world.
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|