GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — Race on `_PyRuntime.signals.unhandled_keyboard_interrupt` when calling `eval()` concurrently in free-threading mode · Issue #128130 · python/cpython

Race on `_PyRuntime.signals.unhandled_keyboard_interrupt` when calling `eval()` concurrently in free-threading mode #128130

New issue

Copy link

New issue

Copy link

Closed

Race on _PyRuntime.signals.unhandled_keyboard_interrupt when calling eval() concurrently in free-threading mode#128130

Copy link

Labels

3.13only security fixesonly security fixes3.14bugs and security fixesbugs and security fixesextension-modulesC modules in the Modules dirC modules in the Modules dirtopic-free-threadingtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Description

hawkinsp

opened

on Dec 20, 2024

Issue body actions

Bug report

Bug description:

Run the following code under Python 3.13.1t with thread-sanitizer enabled:

from collections import namedtuple
import concurrent.futures
import functools
import threading

num_threads = 8


def closure(b):
  b.wait()
  namedtuple('mytuple', ['f'])

with concurrent.futures.ThreadPoolExecutor(max_workers=num_threads) as executor:
  for _ in range(100):
    b = threading.Barrier(num_threads)
    for _ in range(num_threads):
      executor.submit(functools.partial(closure, b))

Get:

WARNING: ThreadSanitizer: data race (pid=682761)
  Write of size 4 at 0x56312bb5f168 by thread T6:
    #0 run_eval_code_obj /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1318:53 (python3.13+0x4a02a6) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #1 run_mod /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1422:19 (python3.13+0x49fa95) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #2 _PyRun_StringFlagsWithName /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1221:15 (python3.13+0x49cfe4) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #3 PyRun_StringFlags /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1233:12 (python3.13+0x49cfe4)
    #4 builtin_eval_impl /usr/local/google/home/phawkins/p/cpython/Python/bltinmodule.c:1027:18 (python3.13+0x3d8189) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #5 builtin_eval /usr/local/google/home/phawkins/p/cpython/Python/clinic/bltinmodule.c.h:467:20 (python3.13+0x3d8189)
    #6 cfunction_vectorcall_FASTCALL_KEYWORDS /usr/local/google/home/phawkins/p/cpython/Objects/methodobject.c:441:24 (python3.13+0x28a010) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #7 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eafaa) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #8 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eafaa)
    #9 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e24fb) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #10 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #11 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #12 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #13 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x571bb2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #14 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x571bb2)
    #15 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #16 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #17 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #18 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #19 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #20 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #21 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #22 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #23 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
    #24 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #25 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #26 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #27 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #28 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)

  Previous write of size 4 at 0x56312bb5f168 by thread T8:
    #0 run_eval_code_obj /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1318:53 (python3.13+0x4a02a6) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #1 run_mod /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1422:19 (python3.13+0x49fa95) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #2 _PyRun_StringFlagsWithName /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1221:15 (python3.13+0x49cfe4) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #3 PyRun_StringFlags /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1233:12 (python3.13+0x49cfe4)
    #4 builtin_eval_impl /usr/local/google/home/phawkins/p/cpython/Python/bltinmodule.c:1027:18 (python3.13+0x3d8189) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #5 builtin_eval /usr/local/google/home/phawkins/p/cpython/Python/clinic/bltinmodule.c.h:467:20 (python3.13+0x3d8189)
    #6 cfunction_vectorcall_FASTCALL_KEYWORDS /usr/local/google/home/phawkins/p/cpython/Objects/methodobject.c:441:24 (python3.13+0x28a010) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #7 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eafaa) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #8 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eafaa)
    #9 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e24fb) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #10 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #11 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #12 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #13 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x571bb2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #14 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x571bb2)
    #15 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #16 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #17 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #18 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #19 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #20 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #21 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #22 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #23 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
    #24 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #25 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #26 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #27 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #28 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)

I think namedtuple calls eval(), which triggers a race here:

cpython/Python/pythonrun.c

Line 1318 in 0671451

_PyRuntime.signals.unhandled_keyboard_interrupt = 0;

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

gh-128130: Fix unhandled keyboard interrupt data race #129975

Activity

hawkinsp
added
type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
on Dec 20, 2024
picnixz
added
extension-modulesC modules in the Modules dirC modules in the Modules dir

3.13only security fixesonly security fixes

topic-free-threading

3.14bugs and security fixesbugs and security fixes
on Dec 20, 2024
colesbury
changed the title ~~[-]Race when calling collections.namedtuple() concurrently in free-threading mode[/-]~~ [+]Race on `_PyRuntime.signals.unhandled_keyboard_interrupt` when calling `eval()` concurrently in free-threading mode[/+] on Dec 21, 2024
colesbury commented on Dec 21, 2024
colesbury
on Dec 21, 2024
Contributor
More actions
I think there's a race condition here even in the default build where a concurrent eval() may wipe out the unhandled_keyboard_interrupt from the main thread.

colesbury commented on Feb 6, 2025

colesbury

on Feb 6, 2025

Contributor

More actions

Also seen when running ./python -m test test_exceptions --parallel-threads=10 -v under Tsan in main (3.14). It's in a different function, but still on _PyRuntime.signals.unhandled_keyboard_interrupt.

WARNING: ThreadSanitizer: data race (pid=990837)
  Write of size 4 at 0x555555cd2530 by thread T595:
    #0 _PyErr_Display Python/pythonrun.c:1127 (python+0x43bad3) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #1 thread_excepthook_file Modules/_threadmodule.c:2189 (python+0x52f068) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #2 thread_excepthook Modules/_threadmodule.c:2268 (python+0x52f068)
    #3 cfunction_vectorcall_O Objects/methodobject.c:523 (python+0x1fd56e) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #4 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x149161) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #5 PyObject_Vectorcall Objects/call.c:327 (python+0x149161)
    #6 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1367 (python+0x8a1ff) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #7 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x37d7fc) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #8 _PyEval_Vector Python/ceval.c:1761 (python+0x37d7fc)
    #9 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14957b) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #10 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x14f138) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #11 method_vectorcall Objects/classobject.c:72 (python+0x14f138)
    #12 _PyVectorcall_Call Objects/call.c:273 (python+0x14b9b7) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #13 _PyObject_Call Objects/call.c:348 (python+0x14be80) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #14 PyObject_Call Objects/call.c:373 (python+0x14be80)
    #15 thread_run Modules/_threadmodule.c:354 (python+0x52e235) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #16 pythread_wrapper Python/thread_pthread.h:242 (python+0x466079) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)

  Previous write of size 4 at 0x555555cd2530 by thread T599:
    #0 _PyErr_Display Python/pythonrun.c:1127 (python+0x43bad3) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #1 thread_excepthook_file Modules/_threadmodule.c:2189 (python+0x52f068) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #2 thread_excepthook Modules/_threadmodule.c:2268 (python+0x52f068)
    #3 cfunction_vectorcall_O Objects/methodobject.c:523 (python+0x1fd56e) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #4 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x149161) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #5 PyObject_Vectorcall Objects/call.c:327 (python+0x149161)
    #6 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1367 (python+0x8a1ff) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #7 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x37d7fc) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #8 _PyEval_Vector Python/ceval.c:1761 (python+0x37d7fc)
    #9 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14957b) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #10 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x14f138) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #11 method_vectorcall Objects/classobject.c:72 (python+0x14f138)
    #12 _PyVectorcall_Call Objects/call.c:273 (python+0x14b9b7) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #13 _PyObject_Call Objects/call.c:348 (python+0x14be80) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #14 PyObject_Call Objects/call.c:373 (python+0x14be80)
    #15 thread_run Modules/_threadmodule.c:354 (python+0x52e235) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #16 pythread_wrapper Python/thread_pthread.h:242 (python+0x466079) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)

colesbury
added 3 commits that reference this issue on Feb 10, 2025
pythongh-128130: Fix unhandled keyboard interrupt data race
9bf5214
pythongh-128130: Fix unhandled keyboard interrupt data race
a5267ca
gh-128130: Fix unhandled keyboard interrupt data race (gh-129975)
451f291
nascheme commented on Mar 12, 2025
nascheme
on Mar 12, 2025
Member
More actions
It looks like this bug has been fixed by gh-128130 and can be closed.

While testing with TSAN, I found another warning and made a PR to fix it: gh-131126.
hawkinsp commented on Mar 21, 2025
hawkinsp
on Mar 21, 2025
ContributorAuthor
More actions
Closing, on the assumption the PR linked above fixed this!
hawkinsp
closed this as completed on Mar 21, 2025
hawkinsp
reopened this on Mar 24, 2025

hawkinsp commented on Mar 24, 2025

hawkinsp

on Mar 24, 2025

ContributorAuthor

More actions

Not fixed on the 3.13 branch:

WARNING: ThreadSanitizer: data race (pid=113280)
  Write of size 4 at 0x567f47699168 by thread T6 (mutexes: read M0):
    #0 run_eval_code_obj /__w/jax/jax/cpython/Python/pythonrun.c:1362:53 (python3.13+0x4a2ee6) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #1 run_mod /__w/jax/jax/cpython/Python/pythonrun.c:1466:19 (python3.13+0x4a26d5) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #2 _PyRun_StringFlagsWithName /__w/jax/jax/cpython/Python/pythonrun.c:1261:15 (python3.13+0x49fa64) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #3 PyRun_StringFlags /__w/jax/jax/cpython/Python/pythonrun.c:1273:12 (python3.13+0x49fa64)
    #4 builtin_exec_impl /__w/jax/jax/cpython/Python/bltinmodule.c (python3.13+0x3d99c2) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #5 builtin_exec /__w/jax/jax/cpython/Python/clinic/bltinmodule.c.h:556:20 (python3.13+0x3d99c2)
    #6 cfunction_vectorcall_FASTCALL_KEYWORDS /__w/jax/jax/cpython/Objects/methodobject.c:441:24 (python3.13+0x28a5d0) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #7 _PyObject_VectorcallTstate /__w/jax/jax/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eb0ea) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #8 PyObject_Vectorcall /__w/jax/jax/cpython/Objects/call.c:327:12 (python3.13+0x1eb0ea)
    #9 _PyEval_EvalFrameDefault /__w/jax/jax/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e385b) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #10 _PyEval_EvalFrame /__w/jax/jax/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df98a) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #11 _PyEval_Vector /__w/jax/jax/cpython/Python/ceval.c:1814:12 (python3.13+0x3df98a)
    #12 _PyFunction_Vectorcall /__w/jax/jax/cpython/Objects/call.c (python3.13+0x1eb75f) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #13 _PyObject_VectorcallTstate /__w/jax/jax/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef7e0) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #14 method_vectorcall /__w/jax/jax/cpython/Objects/classobject.c:92:18 (python3.13+0x1ef7e0)
    #15 _PyVectorcall_Call /__w/jax/jax/cpython/Objects/call.c:273:16 (python3.13+0x1eb3d3) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #16 _PyObject_Call /__w/jax/jax/cpython/Objects/call.c:348:16 (python3.13+0x1eb3d3)
    #17 PyObject_Call /__w/jax/jax/cpython/Objects/call.c:373:12 (python3.13+0x1eb455) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #18 _PyEval_EvalFrameDefault /__w/jax/jax/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e5a42) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #19 _PyEval_EvalFrame /__w/jax/jax/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df98a) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #20 _PyEval_Vector /__w/jax/jax/cpython/Python/ceval.c:1814:12 (python3.13+0x3df98a)
    #21 _PyFunction_Vectorcall /__w/jax/jax/cpython/Objects/call.c (python3.13+0x1eb75f) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #22 _PyObject_VectorcallDictTstate /__w/jax/jax/cpython/Objects/call.c:135:15 (python3.13+0x1ea2dd) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #23 _PyObject_Call_Prepend /__w/jax/jax/cpython/Objects/call.c:504:24 (python3.13+0x1ebdd7) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #24 slot_tp_call /__w/jax/jax/cpython/Objects/typeobject.c:9556:15 (python3.13+0x2f9718) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #25 _PyObject_MakeTpCall /__w/jax/jax/cpython/Objects/call.c:242:18 (python3.13+0x1ea54c) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #26 _PyObject_VectorcallTstate /__w/jax/jax/cpython/./Include/internal/pycore_call.h:166:16 (python3.13+0x1eb1a8) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #27 PyObject_Vectorcall /__w/jax/jax/cpython/Objects/call.c:327:12 (python3.13+0x1eb1a8)
    #28 _PyEval_EvalFrameDefault /__w/jax/jax/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e385b) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #29 _PyEval_EvalFrame /__w/jax/jax/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df98a) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #30 _PyEval_Vector /__w/jax/jax/cpython/Python/ceval.c:1814:12 (python3.13+0x3df98a)
    #31 _PyFunction_Vectorcall /__w/jax/jax/cpython/Objects/call.c (python3.13+0x1eb75f) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #32 _PyObject_VectorcallTstate /__w/jax/jax/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef72f) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #33 method_vectorcall /__w/jax/jax/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef72f)
    #34 _PyVectorcall_Call /__w/jax/jax/cpython/Objects/call.c:273:16 (python3.13+0x1eb3d3) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #35 _PyObject_Call /__w/jax/jax/cpython/Objects/call.c:348:16 (python3.13+0x1eb3d3)
    #36 PyObject_Call /__w/jax/jax/cpython/Objects/call.c:373:12 (python3.13+0x1eb455) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #37 thread_run /__w/jax/jax/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x568202) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #38 pythread_wrapper /__w/jax/jax/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4c1107) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)

  Previous write of size 4 at 0x567f47699168 by thread T7 (mutexes: read M0):
    #0 run_eval_code_obj /__w/jax/jax/cpython/Python/pythonrun.c:1362:53 (python3.13+0x4a2ee6) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #1 run_mod /__w/jax/jax/cpython/Python/pythonrun.c:1466:19 (python3.13+0x4a26d5) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #2 _PyRun_StringFlagsWithName /__w/jax/jax/cpython/Python/pythonrun.c:1261:15 (python3.13+0x49fa64) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #3 PyRun_StringFlags /__w/jax/jax/cpython/Python/pythonrun.c:1273:12 (python3.13+0x49fa64)
    #4 builtin_exec_impl /__w/jax/jax/cpython/Python/bltinmodule.c (python3.13+0x3d99c2) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #5 builtin_exec /__w/jax/jax/cpython/Python/clinic/bltinmodule.c.h:556:20 (python3.13+0x3d99c2)
    #6 cfunction_vectorcall_FASTCALL_KEYWORDS /__w/jax/jax/cpython/Objects/methodobject.c:441:24 (python3.13+0x28a5d0) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #7 _PyObject_VectorcallTstate /__w/jax/jax/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eb0ea) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #8 PyObject_Vectorcall /__w/jax/jax/cpython/Objects/call.c:327:12 (python3.13+0x1eb0ea)
    #9 _PyEval_EvalFrameDefault /__w/jax/jax/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e385b) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #10 _PyEval_EvalFrame /__w/jax/jax/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df98a) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #11 _PyEval_Vector /__w/jax/jax/cpython/Python/ceval.c:1814:12 (python3.13+0x3df98a)
    #12 _PyFunction_Vectorcall /__w/jax/jax/cpython/Objects/call.c (python3.13+0x1eb75f) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #13 _PyObject_VectorcallTstate /__w/jax/jax/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef7e0) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #14 method_vectorcall /__w/jax/jax/cpython/Objects/classobject.c:92:18 (python3.13+0x1ef7e0)
    #15 _PyVectorcall_Call /__w/jax/jax/cpython/Objects/call.c:273:16 (python3.13+0x1eb3d3) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #16 _PyObject_Call /__w/jax/jax/cpython/Objects/call.c:348:16 (python3.13+0x1eb3d3)
    #17 PyObject_Call /__w/jax/jax/cpython/Objects/call.c:373:12 (python3.13+0x1eb455) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #18 _PyEval_EvalFrameDefault /__w/jax/jax/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e5a42) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #19 _PyEval_EvalFrame /__w/jax/jax/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df98a) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #20 _PyEval_Vector /__w/jax/jax/cpython/Python/ceval.c:1814:12 (python3.13+0x3df98a)
    #21 _PyFunction_Vectorcall /__w/jax/jax/cpython/Objects/call.c (python3.13+0x1eb75f) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #22 _PyObject_VectorcallDictTstate /__w/jax/jax/cpython/Objects/call.c:135:15 (python3.13+0x1ea2dd) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #23 _PyObject_Call_Prepend /__w/jax/jax/cpython/Objects/call.c:504:24 (python3.13+0x1ebdd7) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #24 slot_tp_call /__w/jax/jax/cpython/Objects/typeobject.c:9556:15 (python3.13+0x2f9718) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #25 _PyObject_MakeTpCall /__w/jax/jax/cpython/Objects/call.c:242:18 (python3.13+0x1ea54c) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #26 _PyObject_VectorcallTstate /__w/jax/jax/cpython/./Include/internal/pycore_call.h:166:16 (python3.13+0x1eb1a8) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #27 PyObject_Vectorcall /__w/jax/jax/cpython/Objects/call.c:327:12 (python3.13+0x1eb1a8)
    #28 _PyEval_EvalFrameDefault /__w/jax/jax/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e385b) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #29 _PyEval_EvalFrame /__w/jax/jax/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3df98a) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #30 _PyEval_Vector /__w/jax/jax/cpython/Python/ceval.c:1814:12 (python3.13+0x3df98a)
    #31 _PyFunction_Vectorcall /__w/jax/jax/cpython/Objects/call.c (python3.13+0x1eb75f) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #32 _PyObject_VectorcallTstate /__w/jax/jax/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef72f) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #33 method_vectorcall /__w/jax/jax/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef72f)
    #34 _PyVectorcall_Call /__w/jax/jax/cpython/Objects/call.c:273:16 (python3.13+0x1eb3d3) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #35 _PyObject_Call /__w/jax/jax/cpython/Objects/call.c:348:16 (python3.13+0x1eb3d3)
    #36 PyObject_Call /__w/jax/jax/cpython/Objects/call.c:373:12 (python3.13+0x1eb455) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #37 thread_run /__w/jax/jax/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x568202) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)
    #38 pythread_wrapper /__w/jax/jax/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4c1107) (BuildId: d6d54de3753b7c6777cd2a5c08fbef6663ab66e8)

colesbury commented on Mar 24, 2025
colesbury
on Mar 24, 2025
Contributor
More actions
The PR was merged into main, but I didn't backport it to 3.13. I've been generally hesitant to backport PRs that primarily fix TSAN warnings as opposed to crashes or other user visible misbehavior.

If it makes testing Jax easier, let me know and we can reconsider backporting the fix.
hawkinsp commented on Mar 24, 2025
hawkinsp
on Mar 24, 2025
ContributorAuthor
More actions

The PR was merged into main, but I didn't backport it to 3.13. I've been generally hesitant to backport PRs that primarily fix TSAN warnings as opposed to crashes or other user visible misbehavior.

If it makes testing Jax easier, let me know and we can reconsider backporting the fix.

It's not critical. It's just an entry in a tsan suppression file.
https://github.com/jax-ml/jax/blob/d3836f160cc6270280ba75c0ae5b12cf1b99535f/.github/workflows/tsan-suppressions.txt#L27
I'm just trying to be aggressive in making sure that these issues get fixed upstream since I want 3.14 to be race free for us.

Actually, we're doing pretty well overall. We're down to only 2 unfixed races in our test suite (#129748, #128050), assuming the things tagged as fixed in 3.14 are indeed fixed.
colesbury commented on Mar 24, 2025
colesbury
on Mar 24, 2025
Contributor
More actions
Are you able to test with 3.14 as well? If not, what are the blockers?
hawkinsp commented on Mar 24, 2025
hawkinsp
on Mar 24, 2025
ContributorAuthor
More actions

Are you able to test with 3.14 as well? If not, what are the blockers?

No blockers. We just don't have that test. We've done it manually a few times. Let me add that to our CI.
hawkinsp commented on Apr 7, 2025
hawkinsp
on Apr 7, 2025
ContributorAuthor
More actions
Closing, we haven't seen this race under 3.14.
hawkinsp
closed this as completed on Apr 7, 2025

Metadata

Assignees

No one assigned

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Race on _PyRuntime.signals.unhandled_keyboard_interrupt when calling eval() concurrently in free-threading mode #128130

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

colesbury commented on Dec 21, 2024

colesbury commented on Feb 6, 2025

nascheme commented on Mar 12, 2025

hawkinsp commented on Mar 21, 2025

hawkinsp commented on Mar 24, 2025

colesbury commented on Mar 24, 2025

hawkinsp commented on Mar 24, 2025

colesbury commented on Mar 24, 2025

hawkinsp commented on Mar 24, 2025

hawkinsp commented on Apr 7, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Race on `_PyRuntime.signals.unhandled_keyboard_interrupt` when calling `eval()` concurrently in free-threading mode #128130