Description

mpage

opened on Apr 8, 2024

Issue body actions

Feature or enhancement

We need to make the TSAN tests pass with the GIL disabled before we can disable the GIL by default in free-threaded builds.

This should proceed in two phases:

1. Add suppressions for existing TSAN warnings.
2. Triage, fix, and remove the suppressions for the warnings enumerated in (1).

How to run the TSAN tests

1. Build with TSAN:
   env CC=clang CXX=clang++ ./configure --disable-gil --with-thread-sanitizer --with-pydebug && make -j
2. Run tests:
   env TSAN_OPTIONS=suppressions=<repo_root>/Tools/tsan/suppressions_free_threading.txt ./python -m test --tsan -j4

Working on a race

1. Verify that the TSAN tests are passing using the steps from above.
2. Pick a suppression from the section below and assign it to yourself (edit it and add your username next to it). Some of them may be related. If you find that other suppressions are related to the race you're working on please assign them to yourself or contact their owner if they're already assigned.
3. Delete the suppression from <repo_root>/Tools/tsan/suppressions_free_threading.txt , run the TSAN tests, and verify that the race is reported by TSAN. You may need to comment out unrelated functions (notably, _PyEval_EvalFrameDefault) in order to reproduce the race.
4. Fix the race.

Suppressions

### Tasks
- [ ] PyInterpreterState_ThreadHead
- [ ] PyThreadState_Clear
- [ ] PyThreadState_Next
- [ ] PyUnstable_InterpreterFrame_GetLine
- [ ] _PyFrame_GetCode
- [ ] _PyFrame_Initialize
- [ ] _PyObject_TryGetInstanceAttribute
- [ ] _Py_slot_tp_getattr_hook
- [x] _multiprocessing_SemLock_acquire_impl
- [x] _multiprocessing_SemLock_release_impl
- [ ] add_threadstate
- [ ] assign_version_tag
- [ ] dump_traceback
- [ ] fatal_error
- [ ] free_threadstate
- [ ] list_get_item_ref
- [x] new_reference
- [ ] pthread_create
- [ ] set_default_allocator_unlocked
- [ ] tstate_delete_common
- [ ] tstate_is_freed
- [ ] type_modified_unlocked
- [ ] write_thread_id

Linked PRs

• gh-117657: Add TSAN suppressions for the free-threaded build #117736
• gh-117657: Quiet TSAN warning about a data race between start_the_world() and tstate_try_attach() #117828
• gh-117657: Quiet more TSAN warnings due to incorrect modeling of compare/exchange #117830
• gh-117657: Fix data races in the method cache in free-threaded builds #117954
• gh-117657: Quiet erroneous TSAN reports of data races in _PySeqLock #117955
• gh-117657: Fix small issues with instrumentation and TSAN #118064
• gh-117657: use relaxed loads for checking dict keys immortality #118067
• [gh-117657] _Py_MergeZeroLocalRefcount isn't loading ob_ref_shared with strong enough semantics #118111
• gh-117657: Quiet TSAN warnings about remaining non-atomic accesses of tstate->state #118165
• gh-117657: Fixes a few small TSAN issues in dictobject #118200
• [gh-117657] Fix some issues with TSAN in typeobject #118249
• gh-117657: Add a couple more TSAN suppressions #118256
• gh-117657: Fix race data race in _Py_IsOwnedByCurrentThread() #118258
• gh-117657: Fix TSAN list set failure #118260
• gh-117657: Fix data race in _Py_IsImmortal #118261
• gh-117657: Fix data races when writing / reading ob_gc_bits #118292
• gh-117657: Disable the function/code cache in free-threaded builds #118301
• gh-117657: TSAN fix race on gstate->young.count #118313
• gh-117657: Don't specialize RESUME_CHECK when specialization is disabled #118349
• gh-117657: Fix TSAN race involving import lock #118523
• gh-117657: Use clang 18 in TSAN builds #118533
• gh-117657: Replace TSAN suppresions with more specific rules #118722
• gh-117657: Log TSAN warnings to separate files and archive them #118747
• gh-117657: initialize_new_array #118769
• [3.13] gh-117657: Fix data races when writing / reading ob_gc_bits (GH-118292) #118796
• gh-117657: Acquire a critical section around SemLock.__{enter,exit}__ #118812
• gh-117657: initialize_new_array remove race #118842
• gh-117657: Fix QSBR race condition #118843
• [3.13] gh-117657: Acquire a critical section around SemLock.__{enter,exit}__ (GH-118812) #118856
• gh-117657: Fix data races reported by TSAN on interp->threads.main #118865
• [3.13] gh-117657: Replace TSAN suppresions with more specific rules (GH-118722) #118870
• [3.13] gh-117657: Fix data races reported by TSAN on interp->threads.main (GH-118865) #118904
• [3.13] gh-117657: Fix QSBR race condition (GH-118843) #118905
• [3.13] gh-117657: Log TSAN warnings to separate files and archive them (GH-118747) #118931
• gh-117657: Fix itertools.count thread safety #119267
• gh-117657: Fix itertools.count thread safety #119268
• gh-117657: Fix missing atomic in dict_resize #119312
• gh-117657: TSAN Fix race in PyMember_Get and PyMember_Set, for Py_T_OBJECT_EX type #119368
• [3.13] gh-117657: Fix missing atomic in dict_resize #119416
• [3.13] gh-117657: Fix missing atomic in dict_resize (GH-119312) #119417
• gh-117657: Fix TSAN race in free-threaded GC #119883
• gh-117657: Fix TSAN race in QSBR assertion #119887
• [3.13] gh-117657: Fix TSAN race in free-threaded GC (GH-119883) #119903
• [3.13] gh-117657: Fix TSAN race in QSBR assertion (GH-119887) #119904
• gh-117657: Fix data races report by TSAN unicode-hash #119907
• gh-117657: Add TSAN suppression for set_discard_entry #119908
• [3.13] gh-117657: Add TSAN suppression for set_discard_entry (GH-119908) #119914
• gh-117657: Avoid sem_clockwait in TSAN #119915
• gh-117657: Fix TSAN reported race in _PyEval_IsGILEnabled. #119921
• gh-117657: Fix race involving GC and heap initialization #119923
• gh-117657: Fix race involving immortalizing objects #119927
• [3.13] gh-117657: Fix TSAN reported race in _PyEval_IsGILEnabled. (GH-119921) #119939
• [3.13] gh-117657: Avoid sem_clockwait in TSAN (GH-119915) #119992
• [3.13] gh-117657: Fix race involving immortalizing objects (GH-119927) #120005
• [3.13] gh-117657: Fix itertools.count thread safety (GH-119268) #120007
• [3.13] gh-117657: Fix race involving GC and heap initialization (GH-119923) #120038
• gh-117657: Fix some simple races in instrumentation.c #120118
• gh-117657: Make Py_TYPE and Py_SET_TYPE thread safe #120165
• [3.13] gh-117657: Fix TSAN race involving import lock (GH-118523) #120169
• gh-117657: Fix TSan reported data race on ioctl_works #120175
• gh-117657: Make PyType_HasFeature atomic #120210
• [3.13] gh-117657: Make Py_TYPE and Py_SET_TYPE thread safe (GH-120165) #120403
• [3.13] gh-117657: Make PyType_HasFeature atomic (GH-120210) #120443
• [3.13] gh-117657: Fix some simple races in instrumentation.c (GH-120118) #120444
• gh-117657: Fix data races for set_inheritable in fileutils.c with free-threaded builds #120458
• gh-117657: Make PyType_HasFeature (exported version) atomic #120484
• gh-117657: Using critical section to make _socket.socket.close thread safe #120490
• gh-117657: Add TSAN suppression for set_default_allocator_unlocked #120500
• [3.13] gh-117657: Add TSAN suppression for set_default_allocator_unlocked (GH-120500) #120510
• gh-117657: Make frame clearing thread-safe #120542
• [3.13] gh-117657: Make PyType_HasFeature (exported version) atomic (GH-120484) #120554
• gh-117657: Move static variables in fileutils.c to thread state #120559
• [3.13] gh-117657: Fix TSan reported data race on ioctl_works (GH-120175) #120654
• [3.13] gh-117657: Fix __slots__ thread safety in free-threaded build (GH-119368) #120655
• gh-117657: Enable TSAN check on macOS with free-threads build #120502
• gh-117657: Fix data races reported by TSAN in some set methods #120914
• [3.13] gh-117657: Fix data races reported by TSAN in some set methods (GH-120914) #121240
• gh-117657: Remove TSAN suppressions for _abc.c #121508
• gh-117657: Fix TSAN races in setobject.c #121511
• [3.13] gh-117657: Fix TSAN races in setobject.c (GH-121511) #121541
• gh-117657: Skip test when running under TSan #121549
• gh-117657: Fix TSan race in _PyDict_CheckConsistency #121551
• [3.13] gh-117657: Skip test when running under TSan (GH-121549) #121553
• [3.13] gh-117657: Fix TSan race in _PyDict_CheckConsistency (GH-121551) #121590
• [3.13] gh-117657: Remove TSAN suppressions for _abc.c (GH-121508) #121598
• gh-117657: Skip tests that fork with threads under TSan #121599
• gh-117657: Fix some races in tracebacks #121748
• [3.13] gh-117657: Skip tests that fork with threads under TSan (GH-121599) #121819
• gh-117657: Avoid race in PAUSE_ADAPTIVE_COUNTER in free-threaded build #122190
• gh-117657: Skip tests that use threads after fork #122194
• [3.13] gh-117657: Skip tests that use threads after fork (GH-122194) #122198
• [3.13] gh-117657: Avoid race in PAUSE_ADAPTIVE_COUNTER in free-threaded build (GH-122190) #122475
• gh-117657: TSAN Fix races in PyMember_Get and PyMember_Set, for C extensions #123211
• gh-117657: fix race:sock_recv_impl suppressions for free-thread building #123697
• gh-117657: remove dump_traceback suppression from list #123871
• gh-117657: Use an atomic store to set type flags. #127588
• gh-117657: Fix data race in new_reference for free-threaded build #129665
• gh-117657: Include all of test_free_threading in TSAN tests #129749
• gh-117657: Skip tests that may cause stack overflows under TSan #129751
• gh-117657: Add test_thread_local_bytecode to TSAN tests #129753
• gh-117657: Fix data race in dict_dict_merge #129755
• [3.13] gh-117657: Fix data race in dict_dict_merge (gh-129755) #129808
• gh-117657: Enable test_opcache under TSAN #129831
• [3.13] gh-117657: Enable test_opcache under TSAN (GH-129831) #130597
• gh-117657: Skip some tests when running with TSAN #131555
• gh-117657: Fix TSAN data race in _PyEval_SetTrace assertion #131561
• [3.13] gh-117657: Fix TSAN data race in _PyEval_SetTrace assertion (gh-131561) #131564
• gh-117657: Fix data race in compare_unicode_unicode_threadsafe #131746
• gh-117657: enable test_capi under TSAN #136269
• gh-117657: Remove remaining TSan suppressions #143499

Activity

1. mpage

   added

   type-featureA feature request or enhancementA feature request or enhancement

   topic-free-threading

   on Apr 8, 2024
2. mpage

   self-assigned this

   on Apr 8, 2024
3. colesbury

   added 4 commits that reference this issue on Apr 12, 2024

   Merge branch 'pythongh-117657-add-ft-tsan-suppressions' into nogil-in…

   73c8d7c

   gh-117657: Add TSAN suppressions for the free-threaded build (#117736)

   4783206

   gh-117657: Quiet TSAN warning about a data race between `start_the_wo…

   6e0b327

   gh-117657: Quiet more TSAN warnings due to incorrect modeling of comp…

   0d52383
4. diegorusso

   added 3 commits that reference this issue on Apr 17, 2024

   pythongh-117657: Add TSAN suppressions for the free-threaded build (p…

   c267ff8

   pythongh-117657: Quiet TSAN warning about a data race between `start_…

   2b5a7e4

   pythongh-117657: Quiet more TSAN warnings due to incorrect modeling o…

   f8e2d81
5. DinoV

   added 5 commits that reference this issue on Apr 17, 2024

   gh-117657: Fix data races in the method cache in free-threaded builds (…

   b6c62c7

   Merge branch 'main' into pythongh-117657-seq-lock

   778de8e

   gh-117657: Quiet erroneous TSAN reports of data races in `_PySeqLock` (…

   0d29302

   gh-117657: use relaxed loads for checking dict keys immortality (#118067

   1e4a4c4

   [gh-117657] _Py_MergeZeroLocalRefcount isn't loading ob_ref_shared wi…

   b45af00
6. 253 remaining items

   Load more actions
7. colesbury

   added 3 commits that reference this issue on Mar 21, 2025

   pythongh-117657: Skip some tests when running with TSAN

   39cbaa3

   pythongh-117657: Fix TSAN data race in _PyEval_SetTrace assertion

   d035a90

   gh-117657: Fix TSAN data race in _PyEval_SetTrace assertion (gh-131561)

   0de5e0c
8. miss-islington

   added a commit that references this issue on Mar 21, 2025

   pythongh-117657: Fix TSAN data race in _PyEval_SetTrace assertion (py…

   b38f60d
9. colesbury

   added 2 commits that reference this issue on Mar 21, 2025

   [3.13] gh-117657: Fix TSAN data race in _PyEval_SetTrace assertion (g…

   94fbe97

   gh-117657: Skip some tests when running with TSAN (gh-131555)

   56d0f9a
10. picnixz

    added

    interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

    on Mar 22, 2025
11. colesbury

    added 2 commits that reference this issue on Mar 25, 2025

    pythongh-117657: Fix data race in `compare_unicode_unicode_threadsafe`

    31d8ac1

    gh-117657: Fix data race in `compare_unicode_unicode_threadsafe` (gh-…

    5abff69
12. diegorusso

    added a commit that references this issue on Apr 1, 2025

    pythongh-117657: Fix data race in `compare_unicode_unicode_threadsafe` (

    0accaf5
13. kumaraditya303

    added a commit that references this issue on Jul 8, 2025

    gh-117657: enable `test_capi` under TSAN (#136269)

    5193400
14. AndPuQing

    added a commit that references this issue on Jul 11, 2025

    pythongh-117657: enable `test_capi` under TSAN (python#136269)

    b96cb1d
15. Pranjal095

    added a commit that references this issue on Jul 12, 2025

    pythongh-117657: enable `test_capi` under TSAN (python#136269)

    67f6d60
16. picnixz

    added a commit that references this issue on Jul 13, 2025

    pythongh-117657: enable `test_capi` under TSAN (python#136269)

    536f274
17. nascheme commented on Jan 7, 2026

    

    nascheme

    on Jan 7, 2026

    Member

    More actions

    Great work everyone, this was a big one!

    Reacted by Gregory P. Smith and Daniele Parmeggiani

Issue actions