GeezerVison is showing github.com — `ThreadPoolExecutor` tasks see the context of whichever `submit()` call started their worker thread · Issue #159964 · python/cpython
ThreadPoolExecutor tasks see the context of whichever submit() call started their worker thread #159964
DescriptionBug report
Bug description:
When sys.flags.thread_inherit_context is true (the default on the free-threaded build), ThreadPoolExecutor workers copy the context of the submit() call that happened to start them, and every later task on that worker sees that copy:
import contextlib
import contextvars
from concurrent.futures import ThreadPoolExecutor
precision = contextvars.ContextVar("precision", default=8)
@contextlib.contextmanager
def printoptions(p):
token = precision.set(p)
try:
yield
finally:
precision.reset(token)
def fmt(x):
return f"{x:.{precision.get()}f}"
pool = ThreadPoolExecutor(max_workers=1)
with printoptions(2):
print("with precision=2: ", fmt(3.14159), pool.submit(fmt, 3.14159).result())
print("no with block: ", fmt(3.14159), pool.submit(fmt, 3.14159).result())
with printoptions(5):
print("with precision=5: ", fmt(3.14159), pool.submit(fmt, 3.14159).result())On 3.15.0t:
with precision=2: 3.14 3.14
no with block: 3.14159000 3.14
with precision=5: 3.14159 3.14
The pool column is stuck at precision=2 because its only worker was started inside the first with block. With more than one worker, the value a task sees depends on which worker runs it.
sentry-sdk, opentelemetry-instrumentation-threading, and dask all patch or subclass ThreadPoolExecutor to get this behavior.
See also https://discuss.python.org/t/enabling-thread-inherit-context-and-context-aware-warnings-by-default-on-both-builds/108205 and gh-154562.
CPython versions tested on:
3.14, 3.15, CPython main branch
Operating systems tested on:
macOS
Linked PRs
Activityngoldbaum commented on Oct 20, 2026 ContributorAuthor More actions nascheme commented on Oct 20, 2026 I think you could argue that this is a bug. For each task to see the submitting thread’s context, the pool would need to capture that context with contextvars.copy_context() at submission time and execute the task using that copy’s run() method. Currently, when thread_inherit_context is true, contextvar bindings are copied when the pool starts a worker thread, and that worker keeps using its context for subsequent tasks.
Edit: also note that copy_context() and using Context.run() would add a little overhead. However the ContextVars mechanism is designed specifically to be efficient to be used like this. Since asyncio already uses run() similarly, I think it's hard to argue it's too expensive for thread based work. ngoldbaum commented on Oct 21, 2026 ContributorAuthor More actions I’d like to fix this during the core dev sprint. ngoldbaum commented on Oct 21, 2026 ContributorAuthor More actions Unfortunately there are significant downstream uses that making this change would break. In particular:
from concurrent.futures import ThreadPoolExecutor
from asgiref.local import Local
local = Local()
def task():
if not hasattr(local, "connection"):
local.connection = object()
print("opened a connection")
pool = ThreadPoolExecutor(max_workers=1)
for _ in range(3):
pool.submit(task).result()Today this prints "opened a connection" once, because each worker keeps a single context across tasks. With a naive fix for the reproducer in the issue, each task runs in a fresh copy of the submitter's context, and it prints three times. asgiref.local.Local is documented as thread-local in sync threads, and asgiref already works around thread_inherit_context for new threads (django/asgiref#564), but that can't help here. Django keeps its cache backends in a Local, so with this change Django would create a new cache backend for every pool task on the free-threaded build (or with -X thread_inherit_context=1).
So sadly the easy fix I think requires Guido's time machine. It's possible to fix this in asgiref but I'm not sure if they'd be open to that.
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|