GeezerVison is showing github.com — PyGILStateEnsure() (Probably) Crashes If Called After Finalization · Issue #124619 · python/cpython
PyGILStateEnsure() (Probably) Crashes If Called After Finalization #124619DescriptionBug report
Bug description:
I'm pretty sure PyGILState_Ensure() will crash if called after finalization, particularly in the case where the thread doesn't already have a thread state active. I noticed this while looking at the function for something else.
CC @gpshead
CPython versions tested on:
CPython main branch
Operating systems tested on:
No response ActivitySccucci84 commented on Sep 1, 2026 Hitting what looks like this same crash in real-world (non-synthetic) code, via PyGObject + GDBus, on Python 3.14.7 — so this is still unfixed on stable 3.14, not just the 3.14.0a7 build from #132948.
Reproducer: powerprofilesctl get (a short-lived Python script shipped with power-profiles-daemon, using PyGObject to talk to power-profiles-daemon over D-Bus). It queries the current profile, prints it, and exits. Invoked frequently (once per status-bar poll interval) it segfaults intermittently with SIGSEGV / SEGV_MAPERR.
coredumpctl info backtrace of the crashing thread (a GDBus worker thread, not the main thread):
Stack trace of thread 284370 (gdbus):
#0 0x00007fa616eaa612 n/a (libpython3.14.so.1.0 + 0x2aa612)
#1 0x00007fa616e393ad PyEval_RestoreThread (libpython3.14.so.1.0 + 0x2393ad)
#2 0x00007fa616f2a66a PyGILState_Ensure (libpython3.14.so.1.0 + 0x32a66a)
#3 0x00007fa616a6156e n/a (_gi.cpython-314-x86_64-linux-gnu.so + 0x1156e)
#4 0x00007fa615fd2cd9 n/a (libglib-2.0.so.0 + 0x35cd9)
#5 0x00007fa615f5ca90 g_object_unref (libgobject-2.0.so.0 + 0x22a90)
#6 0x00007fa614920720 n/a (libgio-2.0.so.0 + 0x10c720)
...
#11 0x00007fa616001217 g_main_loop_run (libglib-2.0.so.0 + 0x64217)
Frame #0 is unresolved (no debug symbols on this build), but given PyEval_RestoreThread → PyGILState_Ensure directly above it, it's almost certainly the same _PyThreadState_Attach/bind_gilstate_tstate path as the original report.
At the same instant, the main thread is already inside exit():
Stack trace of thread 284367 (main):
#2 n/a (libc.so.6 + 0x40fbf)
#3 exit (libc.so.6 + 0x4109e)
#4 n/a (libc.so.6 + 0x27788)
#5 __libc_start_main (libc.so.6 + 0x278b9)
Mechanism appears to be: the main thread finishes and calls exit(), kicking off interpreter finalization, while a GDBus-owned worker thread is mid-g_object_unref on a proxy/connection object and its GObject finalizer calls back into Python (PyGObject's _gi), which calls PyGILState_Ensure() on a thread state that finalization has already begun tearing down.
Reproduced twice on this machine so far (identical stack both times), a few days apart, under ordinary desktop use — no synthetic thread-spam required, just a program that talks to D-Bus from a background thread and exits promptly afterward. Given power-profiles-daemon, NetworkManager's Python bindings, and plenty of other GLib/GDBus-using Python tools follow this same "spin up async D-Bus, do one thing, exit" pattern, this is likely to affect more than just this one tool as more distros ship 3.14. russfellows commented on Oct 30, 2026 We hit this in a Rust/PyO3 extension (russfellows/s3dlio#302) and narrowed it to a pure C reproducer with no Rust or PyO3 involved. The results add something to this issue: 3.13.8+ survives this pattern and 3.14 does not, so it is a regression in 3.14, not only a long-standing API hazard.
Reproducer
N native (non-Python) threads loop on PyGILState_Ensure() / small Python work / PyGILState_Release(), with a short random sleep between attaches, while the main thread returns and the interpreter finalizes. Many objects with __del__ keep bytecode running during finalization.
attacher.c (build with gcc -O2 -shared -fPIC -I$(python3 -c 'import sysconfig;print(sysconfig.get_path("include"))') attacher.c -o attacher.so):
// Pure-CPython repro for s3dlio #302 (no s3dlio, no Rust): N native (non-Python) threads
// repeatedly do PyGILState_Ensure / light Python work / PyGILState_Release, like tokio workers that
// Python::attach after an await. Every Ensure on a thread that has no thread state creates a NEW one.
#include <Python.h>
#include <pthread.h>
#include <stdio.h>
#include <stdatomic.h>
#include <unistd.h>
#include <stdlib.h>
#if PY_VERSION_HEX < 0x030D0000
#define FIN() _Py_IsFinalizing()
#else
#define FIN() Py_IsFinalizing()
#endif
static atomic_long attached_while_finalizing = 0;
static int check_first = 0; static unsigned max_sleep = 3000;
void set_sleep(unsigned v) { max_sleep = v; }
void set_check(int v) { check_first = v; }
static atomic_long attaches = 0;
static atomic_long began_final = 0, began_final_returned = 0;
static void *run(void *arg) {
unsigned seed = (unsigned)(size_t)&seed ^ (unsigned)pthread_self();
for (;;) {
if (check_first && FIN()) { usleep(1000); continue; } // what pyo3 Python::attach does (best-effort, racy)
int bf = FIN();
if (bf) atomic_fetch_add(&began_final, 1);
PyGILState_STATE g = PyGILState_Ensure();
if (bf) atomic_fetch_add(&began_final_returned, 1);
if (FIN()) atomic_fetch_add(&attached_while_finalizing, 1);
PyObject *b = PyBytes_FromStringAndSize("x", 1); // light Python work, like Py::new(BytesView)
Py_XDECREF(b);
atomic_fetch_add(&attaches, 1);
PyGILState_Release(g);
usleep(max_sleep ? 1 + (unsigned)rand_r(&seed) % max_sleep : 0); // I/O-like gap: threads are often NOT inside Ensure when finalization starts
}
return NULL;
}
static void report(void){ fprintf(stderr,"C-REPORT attaches=%ld attached_while_finalizing=%ld ensure_called_while_finalizing=%ld ensure_RETURNED_for_those=%ld\n", atomic_load(&attaches), atomic_load(&attached_while_finalizing), atomic_load(&began_final), atomic_load(&began_final_returned)); }
int start(int n) {
atexit(report);
for (int i = 0; i < n; i++) { pthread_t t; pthread_create(&t, NULL, run, NULL); pthread_detach(t); }
return n;
}
long finalizing_attaches(void) { return atomic_load(&attached_while_finalizing); }
long total_attaches(void) { return atomic_load(&attaches); }drv.py (run: python3 drv.py ./attacher.so 64, where 64 is the number of native threads):
import ctypes, sys, time, os
lib = ctypes.CDLL(os.path.join(os.path.dirname(os.path.abspath(__file__)), sys.argv[1]))
class D:
def __del__(self): # bytecode runs during finalization -> the eval breaker honours GIL-drop requests
x = [1, 2, 3]; x.append(4)
KEEP = [D() for _ in range(int(sys.argv[3]) if len(sys.argv) > 3 else 50000)]
lib.set_sleep(int(os.environ.get("SLEEP","3000")))
lib.set_check(int(os.environ.get("CHECK","0")))
lib.start(int(sys.argv[2]) if len(sys.argv) > 2 else 8)
time.sleep(0.5)
print("READY", lib.total_attaches(), flush=True)Results (Linux x86_64 glibc, default GIL build, 64 native threads, 30 runs each)
| CPython |
abnormal exits |
| 3.14.10 |
23/30 and 25/30 in two batches: double free or corruption (out), SIGSEGV, Fatal Python error: PyInterpreterState_Delete: remaining threads |
| 3.13.18 |
0/30 |
| 3.13.7, 3.12.16 |
0/30 (from our investigation's runs) |
Analysis (3.14.10 vs 3.13.18 sources)
_Py_Finalize marks only the thread states that already exist. _PyThreadState_RemoveExcept() takes the list of thread states, and only those get _PyThreadState_SetShuttingDown() (pylifecycle.c ~2092). The comment at ~2118 notes the gap: "XXX Make sure we are preventing the creating of any new thread states".- New thread states are still created. A native thread calling
PyGILState_Ensure() during finalization has no thread state, so new_threadstate() creates a fresh one with no finalization check. On a default (GIL) build _PyEval_StopTheWorldAll is a no-op.
- In 3.14,
take_gil parks only flagged thread states. It decides via _PyThreadState_MustExit(tstate), which checks the thread state's own shutting-down flag. The fresh thread state is not flagged, so the thread proceeds and later races PyInterpreterState_Delete / zapthreads. That race produces the three failure modes above:
- a SIGSEGV in
_PyThreadState_Attach while zapthreads frees the thread state;
- a double free when the thread's own
PyGILState_Release frees it too;
- "remaining threads" when the thread state is created after zapthreads ran.
- 3.13.8+ asks the runtime instead. Its
_PyThreadState_MustExit checks globally whether the runtime is finalizing, so the late thread state is parked too (pthread_exit & PyThread_exit_thread from PyEval_RestoreThread etc. are harmful #87135 backport), and there is no crash.
The thread-state-based check in 3.14 appears to have lost the "late native thread" case that 3.13.8+ handles. Refusing (or parking) thread-state creation in PyGILState_Ensure once finalization has started would close it.
Workaround on our side: no native thread attaches during shutdown; Python objects are built only on Python threads. Happy to test a patch.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|