Description

colesbury

opened on Nov 29, 2023

Issue body actions

Feature or enhancement

GCC and Clang provide ThreadSanitizer, a tool that detects data races. We should add support for thread sanitizer to CPython. Note that Python already supports the memory and address sanitizers.

• Add --with-thread-sanitizer as a configure option
• Define _Py_THREAD_SANITIZER if thread-sanitizer is enabled (see, e.g., _Py_ADDRESS_SANITIZER in pyport.h)
• Move the definition of _Py_NO_SANITIZE_THREAD from obmalloc.c to a place that's more widely available (like pyport.h). We're going to need this in a few places

Eventually, it would be helpful to have a continuous build for the combination of --disable-gil --with-thread-sanitizer. Note that we probably won't want to run all the tests. ThreadSanitizer is slow and also not very useful for single-threaded tests. We should collect a subset of our tests that use threading for the ThreadSanitizer continuous build.

Linked PRs

• gh-112536: Add support for thread sanitizer (TSAN) #112648
• gh-112536: enable tsan in build #113232
• gh-112536: Set up TSAN CI for free-threading #116555
• gh-112536: Define MI_TSAN to 1 for --with-mimalloc and --with-thread-sanitizer #116558
• gh-112536: Add --tsan test for reasonable TSAN execution times. #116601
• gh-112536: Add TSAN builds on Github Actions #116872
• gh-112536: Add more TSAN tests #116896
• gh-112536: Add test_threading to TSAN tests #116898
• GH-112536: Add more TSan tests #116911
• [3.12] gh-112536: Add support for thread sanitizer (TSAN) (gh-112648) #116924
• [3.11] gh-112536: Add support for thread sanitizer (TSAN) (gh-112648) #116926
• [3.12] gh-112536: Add --tsan test for reasonable TSAN execution times. (gh-116601) #116929
• [3.12] gh-112536: Add TSAN build on Github Actions (GH-116872) #116953
• gh-112536: Define _Py_THREAD_SANITIZER on GCC when TSan is enabled #117702
• [3.12] gh-112536: Define _Py_THREAD_SANITIZER on GCC when TSan is enabled (GH-117702) #117713
• [3.10]gh-112536: Add support for thread sanitizer (TSAN)(gh-112648) #123833

Reacted by Erlend E. Aasland, Kirill Podoprigora and Antoine Pitrou

Activity

1. colesbury

   added

   type-featureA feature request or enhancementA feature request or enhancement

   3.13only security fixesonly security fixes

   topic-free-threading

   on Nov 29, 2023
2. samety commented on Dec 3, 2023

   

   samety

   on Dec 3, 2023

   Contributor

   More actions

   Hey,
   I would like to pick this up.
   As a first step tried copying the struct for ASAN to TSAN
   Here is a draft PR: #112648

   Happy to get some initial feedback on it and then add remaining things like _Py_THREAD_SANITIZER etc.
3. colesbury

   assigned

   samety

   on Dec 5, 2023
4. samety commented on Dec 17, 2023

   

   samety

   on Dec 17, 2023

   Contributor

   More actions

   Added --with-thread-sanitizer as a configure option in pr 112648.

   Still looking for a good way to enable TSAN for the continuous build, I will make another PR when I find one.
5. corona10

   added a commit that references this issue on Dec 30, 2023

   gh-112536: Add support for thread sanitizer (TSAN) (gh-112648)

   88cb972
6. kulikjak

   added a commit that references this issue on Jan 22, 2024

   pythongh-112536: Add support for thread sanitizer (TSAN) (pythongh-11…

   0a51506
7. erlend-aasland

   added

   buildThe build process and cross-buildThe build process and cross-build

   on Jan 28, 2024
8. aisk

   added a commit that references this issue on Feb 11, 2024

   pythongh-112536: Add support for thread sanitizer (TSAN) (pythongh-11…

   001cf08
9. corona10

   added 5 commits that reference this issue on Mar 10, 2024

   pythongh-112536: Set up TSAN CI for free-threading

   bd5e85a

   pythongh-112536: Define MI_TSAN to 1 for --with-mimalloc and --with-t…

   a97883f

   Revert "pythongh-112536: Define MI_TSAN to 1 for --with-mimalloc and …

   058edaf

   gh-112536: Define MI_TSAN to 1 for --with-mimalloc and --with-thread-…

   6c4fc20

   pythongh-112536: Add --tsan test for reasonable TSAN execution times.

   60e7b12
10. 41 remaining items
11. adorilson

    added a commit that references this issue on Mar 25, 2024

    pythonGH-112536: Add more TSan tests (python#116911)

    3a416b2
12. colesbury

    added 2 commits that reference this issue on Apr 9, 2024

    pythongh-112536: Define `_Py_THREAD_SANITIZER` on GCC when TSan is en…

    fcb9681

    gh-112536: Define `_Py_THREAD_SANITIZER` on GCC when TSan is enabled (#…

    79eec66
13. miss-islington

    added a commit that references this issue on Apr 10, 2024

    pythongh-112536: Define `_Py_THREAD_SANITIZER` on GCC when TSan is en…

    8206a1e
14. colesbury

    added a commit that references this issue on Apr 10, 2024

    [3.12] gh-112536: Define `_Py_THREAD_SANITIZER` on GCC when TSan is e…

    04d0796
15. diegorusso

    added 7 commits that reference this issue on Apr 17, 2024

    pythongh-112536: Define MI_TSAN to 1 for --with-mimalloc and --with-t…

    ad440fc

    pythongh-112536: Add --tsan test for reasonable TSAN execution times. (…

    648dc16

    pythongh-112536: Add TSAN builds on Github Actions (python#116872)

    5ed7bd3

    pythongh-112536: Add test_threading to TSAN tests (python#116898)

    ba308e6

    pythongh-112536: Add more TSAN tests (python#116896)

    524c393

    pythonGH-112536: Add more TSan tests (python#116911)

    ef7e68b

    pythongh-112536: Define `_Py_THREAD_SANITIZER` on GCC when TSan is en…

    f7aafbb
16. comicfans commented on Aug 27, 2024

    

    comicfans

    on Aug 27, 2024

    More actions

    hello, possible to backport this to 3.10 too ?
17. Glyphack

    added a commit that references this issue on Sep 2, 2024

    pythongh-112536: Add support for thread sanitizer (TSAN) (pythongh-11…

    f54c75e
18. picnixz commented on Sep 8, 2024

    

    picnixz

    on Sep 8, 2024

    Member

    More actions

    Unfortunately, 3.8 to 3.11 are security-only branches. As such, they would not accept such backports. The 3.11 backport was not merged since it was close to the last bugfix deadline.

Issue actions