GeezerVison is showing github.com — gcmodule: GIL drop during collection in debug mode may result in unexpected behavior with multiple threads · Issue #112564 · python/cpython
gcmodule: GIL drop during collection in debug mode may result in unexpected behavior with multiple threads #112564Labels interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or errortype-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump DescriptionBug report
Bug description:
By default, garbage collection is performed without dropping the GIL. However, when debug mode is enabled using gc.set_debug, functions PySys_WriteStderr and PySys_FormatStderr are used to print debug information, potentially leading to GIL drop. In a multi-threaded application, this behavior can result in the execution of code in other threads while the garbage collector (GC) is in the process of collection.
gc.DEBUG_STATS seems to be safe, only possibly disrupting GC's debug info output and hypothetically causing issues for callbacks in gc.callbacks if they exist.
gc.DEBUG_COLLECTABLE can cause segmentation fault.
Code causing segmentation fault
import gc
import threading
from time import sleep
from weakref import ref
class Object:
pass
RANGE = 10000
def main_thread():
for _ in range(RANGE):
gc.collect()
sleep(0)
def side_thread_segmentation_fault():
for _ in range(RANGE):
references = []
for _ in range(20):
_object = Object()
_list = [_object]
_object._list = _list
references.append(ref(_object))
collected = gc.collect()
if collected == 0 and any(_object() for _object in references):
# GC started in the main thread and has dropped the GIL
# somewhere in the middle of `gc_collect_main`
for reference in reversed(references):
if reference() is not None:
_object = reference()
_list = _object._list
_list.remove(_object)
del _object._list
sleep(0)
sleep(0)
gc.set_debug(gc.DEBUG_STATS | gc.DEBUG_COLLECTABLE)
t = threading.Thread(target=side_thread_segmentation_fault)
t.start()
main_thread()
t.join()
Can PySys_WriteStderr and PySys_FormatStderr be replaced with functions that don't drop the GIL?
CPython versions tested on:
3.12
Operating systems tested on:
Linux
Linked PRs
Activitycorona10 commented on Dec 3, 2023 From my macOS with the latest commit, there was no issue
gc: collectable <function 0x1030fb600>
gc: collectable <function 0x1030fb7e0>
gc: collectable <tuple 0x103120a60>
gc: collectable <cell 0x103120a30>
gc: done, 1203 unreachable, 0 uncollectable, 0.0043s elapsed
olutra commented on Dec 3, 2023 I saved the code causing segmentation fault from the bug description to the file segfault.py and ran it a couple of times
for i in {1..20}
do
./python segfault.py 2>/dev/null && echo $?
doneHere, ./python is the binary built today from the main branch.
The result is:
0
0
0
0
Segmentation fault (core dumped)
0
0
0
0
0
0
Segmentation fault (core dumped)
0
0
0
0
0
Segmentation fault (core dumped)
Segmentation fault (core dumped)
0 If I run the script manually without redirecting stderr to /dev/null (./python segfault.py), it segfaults 10 out of 10 times. I believe this is due to the faster writing to /dev/null compared to printing to the terminal. @corona10, the same might be true for your machine; if it's fast enough, it could collect all objects within a time less than the switch interval. In that case, increasing the number of objects from 20 to 50 (or perhaps 70, 100, or even more) can help reproduce the bug. On my machine, changing the number of objects from 20 to 50 results in segfault 17 out of 20 times in the bash for loop with redirecting stderr. BHUVANSH855 commented on Sep 24, 2026 I tested this on a local CPython 3.14.6 build and haven't been able to reproduce the crash so far. I ran a 50M-iteration stress test covering the affected keyword-call path with argument corruption checks, and all iterations completed successfully.
I'll keep narrowing down the conditions needed to trigger the failure. olutra commented on Sep 25, 2026 Here is an improved version of the reproduction script that tests various execution parameters. From my testing, modifying sys.setswitchinterval() seems to have the most significant impact on triggering the race condition.
My current setup:
OS: Fedora Linux 44
Kernel: 7.2.7-200.fc44.x86_64 (64-bit)
Processor: Intel Core Ultra 7 155H
I tested this across multiple Python runtimes, including the system Python v3.14.7 and several versions managed via uv (v3.9 through v3.13). Interestingly, Python 3.9 and 3.10 last slightly longer before reproducing the failure compared to newer versions.
Also, the issue doesn't appear on free-threaded builds of Python.
Code
import gc
import sys
import threading
from time import sleep
from weakref import ref
class CircularReferenceObject:
def __init__(self):
self._list = [self]
def remove_circular_reference(self):
self._list.remove(self)
del self._list
def main_thread(*, number_of_collections):
for _ in range(number_of_collections):
gc.collect()
sleep(0)
def side_thread_segmentation_fault(*, number_of_collections, number_of_objects):
for _ in range(number_of_collections):
weak_refs = []
for _ in range(number_of_objects):
_object = CircularReferenceObject()
weak_refs.append(ref(_object))
collected = gc.collect()
if collected == 0 and any(_object() for _object in weak_refs):
# GC started in the main thread and has dropped the GIL
# somewhere in the middle of `gc_collect_main`
for reference in reversed(weak_refs):
if reference() is not None:
_object = reference()
_object.remove_circular_reference()
sleep(0)
sleep(0)
gc.set_debug(gc.DEBUG_STATS | gc.DEBUG_COLLECTABLE)
for number_of_collections in (1_000, 5_000, 10_000, 20_000):
for number_of_objects in (20, 50, 100, 200, 500, 1_000, 2_000):
for switch_interval in (1, 1e-3, 1e-6, 1e-9, 1e-12):
print(
f"\tNumber of objects: {number_of_objects}\n"
f"\tNumber of collections: {number_of_collections}\n"
f"\tSwitch interbal: {switch_interval}\n"
)
sys.setswitchinterval(switch_interval)
t = threading.Thread(
target=side_thread_segmentation_fault,
kwargs={
"number_of_collections": number_of_collections,
"number_of_objects": number_of_objects,
},
)
t.start()
main_thread(number_of_collections=number_of_collections)
t.join()
johng commented on Oct 25, 2026 I've raised a fix for this and added a reliable reproduction leveraging Stderr so that we can free the object during the debug loop
MetadataMetadatainterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or errortype-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump DevelopmentNo branches or pull requestsIssue actions
|