Description

Naserume

opened on Jul 1, 2026

Issue body actions

Bug description:

In the free-threaded build, sys._current_exceptions() iterates every thread state under stop-the-world + HEAD_LOCK and reads t->thread_id.

cpython/Python/pystate.c

Lines 2808 to 2817 in ecdef17

	_PyEval_StopTheWorldAll(runtime);
	HEAD_LOCK(runtime);
	PyInterpreterState *i;
	for (i = runtime->interpreters.head; i != NULL; i = i->next) {
	_Py_FOR_EACH_TSTATE_UNLOCKED(i, t) {
	_PyErr_StackItem *err_info = _PyErr_GetTopmostException(t);
	if (err_info == NULL) {
	continue;
	}
	PyObject *id = PyLong_FromUnsignedLong(t->thread_id);

But a newly started thread sets tstate->thread_id in bind_tstate() without holding HEAD_LOCK. Its PyThreadState is already on the interpreter's thread list by this point.

cpython/Python/pystate.c

Lines 164 to 178 in ecdef17

	bind_tstate(PyThreadState *tstate)
	{
	assert(tstate != NULL);
	assert(tstate_is_alive(tstate) && !tstate->_status.bound);
	assert(!tstate->_status.unbound); // just in case
	assert(!tstate->_status.bound_gilstate);
	assert(tstate != gilstate_get());
	assert(!tstate->_status.active);
	assert(tstate->thread_id == 0);
	assert(tstate->native_thread_id == 0);
	// Currently we don't necessarily store the thread state
	// in thread-local storage (e.g. per-interpreter).
	tstate->thread_id = PyThread_get_thread_ident();

A thread that is still attaching is not yet stop-the-world-stoppable. _PyEval_StopTheWorldAll() therefore does not pause it, and its unlocked write to tstate->thread_id races the stop-the-world reader in _PyThread_CurrentExceptions.

Reproducer:

import sys
import threading

stop = threading.Event()

def reader():                      # keep stop-the-world almost always active
    for _ in range(20000):
        sys._current_exceptions()

def churn():                       # continuously attach new threads (bind_tstate)
    for _ in range(20000):
        t = threading.Thread(target=lambda: None)
        t.start()
        t.join()

readers  = [threading.Thread(target=reader) for _ in range(8)]
churners = [threading.Thread(target=churn)  for _ in range(4)]
for t in readers + churners: t.start()
for t in readers + churners: t.join()

TSAN Report:

WARNING: ThreadSanitizer: data race (pid=94157)
  Read of size 8 at 0x0001081180b0 by thread T1:
    #0 _PyThread_CurrentExceptions pystate.c:2817
    #1 sys__current_exceptions sysmodule.c.h:1180
    #2 cfunction_vectorcall_NOARGS methodobject.c:508
    #3 PyObject_Vectorcall call.c:327
    #4 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:768
    #5 _PyEval_EvalFrameDefault generated_cases.c.h:1906
...
  Previous write of size 8 by thread T9:
    #0 bind_tstate pystate.c:178
    #1 _PyThreadState_Bind pystate.c:2660
    #2 thread_run _threadmodule.c:384
    #3 pythread_wrapper thread_pthread.h:234

SUMMARY: ThreadSanitizer: data race pystate.c:2817 in _PyThread_CurrentExceptions

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-152741: Fix data race on PyThreadState.thread_id during thread bind #152755
• gh-152741: Publish thread states to the interpreter's thread list only after they are fully initialized #156202

Activity

1. Naserume

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Jul 1, 2026
2. Naserume

   changed the title [-]Data race between `sys._current_exceptions()` and a concurrently attaching thread (MacOS ARM)[/-] [+]Data race between `sys._current_exceptions()` and a concurrently attaching thread[/+] on Jul 1, 2026
3. zainnadeem786 commented on Jul 1, 2026

   

   zainnadeem786

   on Jul 1, 2026

   Contributor

   More actions

   I took a closer look at this on the latest main (564c58c718bc) using a free-threaded Windows debug build.

   I couldn't independently reproduce the reported ThreadSanitizer warning because TSAN isn't available with the Windows/MSVC free-threaded build, so I focused on tracing the implementation and verifying the reported code paths.

   From what I found, the synchronization concern described in the report appears consistent with the current implementation.

   _PyThreadState_New() publishes the new thread state into interp->threads.head while holding HEAD_LOCK, but the child thread later calls _PyThreadState_Bind(), where bind_tstate() initializes tstate->thread_id without HEAD_LOCK.

   Meanwhile, _PyThread_CurrentExceptions() performs stop-the-world, acquires HEAD_LOCK, iterates the published thread-state list, and reads t->thread_id. Since bind_tstate() runs before _PyThreadState_Attach(), my understanding is that an attaching thread is not yet fully covered by stop-the-world, so the write to thread_id may race with those readers.

   While tracing this path, I also noticed that _PyThread_CurrentFrames() and PyThreadState_SetAsyncExc() use the same thread_id publication/read pattern, so the discussion may extend beyond _current_exceptions() alone.

   I also considered a few possible implementation approaches (synchronizing publication under HEAD_LOCK, atomic publication/access, or excluding published-but-unbound thread states from these APIs), but each seems to have different lifecycle implications, so I don't want to guess at the intended design.

   Before attempting a PR, I'd appreciate some guidance from the runtime/free-threading maintainers on the intended invariant here.

   Should published-but-not-yet-bound thread states be visible to APIs such as _current_exceptions() and _current_frames(), or is the expectation that thread_id publication itself should be synchronized (for example via HEAD_LOCK or atomic access)?

   If there's a preferred direction, I'd be happy to investigate further and work on a patch.
4. tonghuaroot commented on Jul 1, 2026

   

   tonghuaroot

   on Jul 1, 2026

   Contributor

   More actions

   Confirmed on a free-threaded ThreadSanitizer build (--with-thread-sanitizer --disable-gil --with-pydebug); the reproducer fires on essentially every run:

   WARNING: ThreadSanitizer: data race
     Read  of size 8 ... _PyThread_CurrentExceptions  Python/pystate.c:2817
     Write of size 8 ... bind_tstate                  Python/pystate.c:178
   

   bind_tstate writes thread_id after the PyThreadState is on interp->threads.head but before the thread attaches, so stop-the-world can't pause it and the store races sys._current_exceptions(). _PyThread_CurrentFrames reads the same field but I couldn't make it race — its if (frame == NULL) continue; gate means the thread only reaches the read after PyEval_AcquireThread has synchronized through tstate->state.

   Suggested labels: topic-free-threading, interpreter-core (on top of type-bug).

   Fix is small — store thread_id/native_thread_id with a relaxed atomic in bind_tstate and read them with a relaxed atomic load in the thread-list readers, matching tstate->state. Opened #152755 (TSan-clean across 30+ runs).
5. StanFromIreland

   added

   extension-modulesC modules in the Modules dirC modules in the Modules dir

   topic-free-threading

   on Jul 1, 2026
6. kumaraditya303

   added a commit that references this issue on Aug 21, 2026

   pythongh-152741: Publish thread states to the interpreter's thread li…

   6213ab4
7. kumaraditya303

   added a commit that references this issue on Sep 9, 2026

   Merge branch 'main' into pythongh-152741-defer-tstate-publish

   ea15c1d
8. kumaraditya303

   added a commit that references this issue on Nov 9, 2026

   Merge remote-tracking branch 'upstream/main' into pythongh-152741-def…

   5ba1e0f

Issue actions