GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • All gists
  • Back to GitHub
  • Sign in
  • Sign up
  • Code
  • Revisions 1
  • Terms
  • Privacy
  • Security
  • Status
  • Community
  • Docs
  • Contact

This view

  • Original page
  • New address

GeezerVison is showing gist.github.com — cpython free-threading hashlib TSAN race

Instantly share code, notes, and snippets.

@vfdev-5

vfdev-5/check.log Secret

Created January 9, 2025 01:02
Show Gist options
  • Download ZIP
  • Star 0 (0) You must be signed in to star a gist
  • Fork 0 (0) You must be signed in to fork a gist
  • Clone this repository at <script src="https://gist.github.com/vfdev-5/2896eb30fc6ec6888c2f0dbaa31e550e.js"></script>
  • Save vfdev-5/2896eb30fc6ec6888c2f0dbaa31e550e to your computer and use it in GitHub Desktop.
Clone this repository at <script src="https://gist.github.com/vfdev-5/2896eb30fc6ec6888c2f0dbaa31e550e.js"></script>
Save vfdev-5/2896eb30fc6ec6888c2f0dbaa31e550e to your computer and use it in GitHub Desktop.
Download ZIP
cpython free-threading hashlib TSAN race
Raw
check.log
==================
WARNING: ThreadSanitizer: data race (pid=46290)
Write of size 8 at 0x7fffb40402c0 by thread T5:
#0 py_digest_by_name /project/cpython/./Modules/_hashopenssl.c:384:28 (_hashlib.cpython-313t-x86_64-linux-gnu.so+0x7ce6) (BuildId: cc517b229181dc934d8851b36d8993a3b5cedfb1)
#1 py_evp_fromname /project/cpython/./Modules/_hashopenssl.c:914:14 (_hashlib.cpython-313t-x86_64-linux-gnu.so+0x7872) (BuildId: cc517b229181dc934d8851b36d8993a3b5cedfb1)
#2 _hashlib_openssl_sha256_impl /project/cpython/./Modules/_hashopenssl.c:1083:12 (_hashlib.cpython-313t-x86_64-linux-gnu.so+0x6f75) (BuildId: cc517b229181dc934d8851b36d8993a3b5cedfb1)
#3 _hashlib_openssl_sha256 /project/cpython/./Modules/clinic/_hashopenssl.c.h:591:20 (_hashlib.cpython-313t-x86_64-linux-gnu.so+0x6f75)
#4 cfunction_vectorcall_FASTCALL_KEYWORDS /project/cpython/Objects/methodobject.c:441:24 (python3.13t+0x289f20) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#5 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:168:11 (python3.13t+0x1eafea) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#6 PyObject_Vectorcall /project/cpython/Objects/call.c:327:12 (python3.13t+0x1eafea)
#7 _PyEval_EvalFrameDefault /project/cpython/Python/generated_cases.c.h:813:23 (python3.13t+0x3e28eb) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#8 _PyEval_EvalFrame /project/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13t+0x3dea1a) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#9 _PyEval_Vector /project/cpython/Python/ceval.c:1807:12 (python3.13t+0x3dea1a)
#10 _PyFunction_Vectorcall /project/cpython/Objects/call.c (python3.13t+0x1eb65f) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#11 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:168:11 (python3.13t+0x1ef62f) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#12 method_vectorcall /project/cpython/Objects/classobject.c:70:20 (python3.13t+0x1ef62f)
#13 _PyVectorcall_Call /project/cpython/Objects/call.c:273:16 (python3.13t+0x1eb2d3) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#14 _PyObject_Call /project/cpython/Objects/call.c:348:16 (python3.13t+0x1eb2d3)
#15 PyObject_Call /project/cpython/Objects/call.c:373:12 (python3.13t+0x1eb355) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#16 thread_run /project/cpython/./Modules/_threadmodule.c:337:21 (python3.13t+0x564a12) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#17 pythread_wrapper /project/cpython/Python/thread_pthread.h:243:5 (python3.13t+0x4bdd97) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
Previous write of size 8 at 0x7fffb40402c0 by thread T40:
#0 py_digest_by_name /project/cpython/./Modules/_hashopenssl.c:384:28 (_hashlib.cpython-313t-x86_64-linux-gnu.so+0x7ce6) (BuildId: cc517b229181dc934d8851b36d8993a3b5cedfb1)
#1 py_evp_fromname /project/cpython/./Modules/_hashopenssl.c:914:14 (_hashlib.cpython-313t-x86_64-linux-gnu.so+0x7872) (BuildId: cc517b229181dc934d8851b36d8993a3b5cedfb1)
#2 _hashlib_openssl_sha256_impl /project/cpython/./Modules/_hashopenssl.c:1083:12 (_hashlib.cpython-313t-x86_64-linux-gnu.so+0x6f75) (BuildId: cc517b229181dc934d8851b36d8993a3b5cedfb1)
#3 _hashlib_openssl_sha256 /project/cpython/./Modules/clinic/_hashopenssl.c.h:591:20 (_hashlib.cpython-313t-x86_64-linux-gnu.so+0x6f75)
#4 cfunction_vectorcall_FASTCALL_KEYWORDS /project/cpython/Objects/methodobject.c:441:24 (python3.13t+0x289f20) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#5 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:168:11 (python3.13t+0x1eafea) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#6 PyObject_Vectorcall /project/cpython/Objects/call.c:327:12 (python3.13t+0x1eafea)
#7 _PyEval_EvalFrameDefault /project/cpython/Python/generated_cases.c.h:813:23 (python3.13t+0x3e28eb) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#8 _PyEval_EvalFrame /project/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13t+0x3dea1a) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#9 _PyEval_Vector /project/cpython/Python/ceval.c:1807:12 (python3.13t+0x3dea1a)
#10 _PyFunction_Vectorcall /project/cpython/Objects/call.c (python3.13t+0x1eb65f) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#11 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:168:11 (python3.13t+0x1ef62f) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#12 method_vectorcall /project/cpython/Objects/classobject.c:70:20 (python3.13t+0x1ef62f)
#13 _PyVectorcall_Call /project/cpython/Objects/call.c:273:16 (python3.13t+0x1eb2d3) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#14 _PyObject_Call /project/cpython/Objects/call.c:348:16 (python3.13t+0x1eb2d3)
#15 PyObject_Call /project/cpython/Objects/call.c:373:12 (python3.13t+0x1eb355) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#16 thread_run /project/cpython/./Modules/_threadmodule.c:337:21 (python3.13t+0x564a12) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#17 pythread_wrapper /project/cpython/Python/thread_pthread.h:243:5 (python3.13t+0x4bdd97) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
Thread T5 (tid=46296, running) created by main thread at:
#0 pthread_create <null> (python3.13t+0xde1df) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#1 do_start_joinable_thread /project/cpython/Python/thread_pthread.h:290:14 (python3.13t+0x4bcc48) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#2 PyThread_start_joinable_thread /project/cpython/Python/thread_pthread.h:314:9 (python3.13t+0x4bca6a) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#3 ThreadHandle_start /project/cpython/./Modules/_threadmodule.c:422:9 (python3.13t+0x5645a7) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#4 do_start_new_thread /project/cpython/./Modules/_threadmodule.c:1849:9 (python3.13t+0x5645a7)
#5 thread_PyThread_start_joinable_thread /project/cpython/./Modules/_threadmodule.c:1972:14 (python3.13t+0x563641) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#6 cfunction_call /project/cpython/Objects/methodobject.c:540:18 (python3.13t+0x28abb7) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#7 _PyObject_MakeTpCall /project/cpython/Objects/call.c:242:18 (python3.13t+0x1ea44c) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#8 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:166:16 (python3.13t+0x1eb0a8) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#9 PyObject_Vectorcall /project/cpython/Objects/call.c:327:12 (python3.13t+0x1eb0a8)
#10 _PyEval_EvalFrameDefault /project/cpython/Python/generated_cases.c.h:1502:19 (python3.13t+0x3e556d) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#11 _PyEval_EvalFrame /project/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13t+0x3de6f2) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#12 _PyEval_Vector /project/cpython/Python/ceval.c:1807:12 (python3.13t+0x3de6f2)
#13 PyEval_EvalCode /project/cpython/Python/ceval.c:597:21 (python3.13t+0x3de6f2)
#14 run_eval_code_obj /project/cpython/Python/pythonrun.c:1337:9 (python3.13t+0x4a0a5e) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#15 run_mod /project/cpython/Python/pythonrun.c:1422:19 (python3.13t+0x4a0185) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#16 pyrun_file /project/cpython/Python/pythonrun.c:1255:15 (python3.13t+0x49c280) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#17 _PyRun_SimpleFileObject /project/cpython/Python/pythonrun.c:490:13 (python3.13t+0x49c280)
#18 _PyRun_AnyFileObject /project/cpython/Python/pythonrun.c:77:15 (python3.13t+0x49b948) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#19 pymain_run_file_obj /project/cpython/Modules/main.c:410:15 (python3.13t+0x4d7e6f) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#20 pymain_run_file /project/cpython/Modules/main.c:429:15 (python3.13t+0x4d7e6f)
#21 pymain_run_python /project/cpython/Modules/main.c:697:21 (python3.13t+0x4d70bc) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#22 Py_RunMain /project/cpython/Modules/main.c:776:5 (python3.13t+0x4d70bc)
#23 pymain_main /project/cpython/Modules/main.c:806:12 (python3.13t+0x4d74f8) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#24 Py_BytesMain /project/cpython/Modules/main.c:830:12 (python3.13t+0x4d757b) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#25 main /project/cpython/./Programs/python.c:15:12 (python3.13t+0x15c7eb) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
Thread T40 (tid=46331, running) created by main thread at:
#0 pthread_create <null> (python3.13t+0xde1df) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#1 do_start_joinable_thread /project/cpython/Python/thread_pthread.h:290:14 (python3.13t+0x4bcc48) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#2 PyThread_start_joinable_thread /project/cpython/Python/thread_pthread.h:314:9 (python3.13t+0x4bca6a) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#3 ThreadHandle_start /project/cpython/./Modules/_threadmodule.c:422:9 (python3.13t+0x5645a7) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#4 do_start_new_thread /project/cpython/./Modules/_threadmodule.c:1849:9 (python3.13t+0x5645a7)
#5 thread_PyThread_start_joinable_thread /project/cpython/./Modules/_threadmodule.c:1972:14 (python3.13t+0x563641) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#6 cfunction_call /project/cpython/Objects/methodobject.c:540:18 (python3.13t+0x28abb7) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#7 _PyObject_MakeTpCall /project/cpython/Objects/call.c:242:18 (python3.13t+0x1ea44c) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#8 _PyObject_VectorcallTstate /project/cpython/./Include/internal/pycore_call.h:166:16 (python3.13t+0x1eb0a8) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#9 PyObject_Vectorcall /project/cpython/Objects/call.c:327:12 (python3.13t+0x1eb0a8)
#10 _PyEval_EvalFrameDefault /project/cpython/Python/generated_cases.c.h:1502:19 (python3.13t+0x3e556d) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#11 _PyEval_EvalFrame /project/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13t+0x3de6f2) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#12 _PyEval_Vector /project/cpython/Python/ceval.c:1807:12 (python3.13t+0x3de6f2)
#13 PyEval_EvalCode /project/cpython/Python/ceval.c:597:21 (python3.13t+0x3de6f2)
#14 run_eval_code_obj /project/cpython/Python/pythonrun.c:1337:9 (python3.13t+0x4a0a5e) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#15 run_mod /project/cpython/Python/pythonrun.c:1422:19 (python3.13t+0x4a0185) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#16 pyrun_file /project/cpython/Python/pythonrun.c:1255:15 (python3.13t+0x49c280) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#17 _PyRun_SimpleFileObject /project/cpython/Python/pythonrun.c:490:13 (python3.13t+0x49c280)
#18 _PyRun_AnyFileObject /project/cpython/Python/pythonrun.c:77:15 (python3.13t+0x49b948) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#19 pymain_run_file_obj /project/cpython/Modules/main.c:410:15 (python3.13t+0x4d7e6f) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#20 pymain_run_file /project/cpython/Modules/main.c:429:15 (python3.13t+0x4d7e6f)
#21 pymain_run_python /project/cpython/Modules/main.c:697:21 (python3.13t+0x4d70bc) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#22 Py_RunMain /project/cpython/Modules/main.c:776:5 (python3.13t+0x4d70bc)
#23 pymain_main /project/cpython/Modules/main.c:806:12 (python3.13t+0x4d74f8) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#24 Py_BytesMain /project/cpython/Modules/main.c:830:12 (python3.13t+0x4d757b) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
#25 main /project/cpython/./Programs/python.c:15:12 (python3.13t+0x15c7eb) (BuildId: 0d639d83db92b5fe0f1dbe7fdffbc7405ce29a98)
SUMMARY: ThreadSanitizer: data race /project/cpython/./Modules/_hashopenssl.c:384:28 in py_digest_by_name
==================
ThreadSanitizer: reported 1 warnings
Raw
check.py
import hashlib
import concurrent.futures
import threading
def _hash_string(hash_obj, str_var):
hash_obj.update(str_var.encode("utf-8").strip())
jaxlib_version_str = "abcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdef"
compression_algorithm = "zstandard"
def test_hashlib_sha256():
entries = [
(
"jax_lib version",
lambda hash_obj: hash_obj.update(
bytes(jaxlib_version_str.encode("utf-8"))
),
),
(
"compression",
lambda hash_obj: _hash_string(hash_obj, compression_algorithm),
),
("custom_hook", lambda hash_obj: _hash_string(hash_obj, "")),
]
hash_obj = hashlib.sha256()
for _ in range(20):
for name, hashfn in entries:
hashfn(hash_obj)
for _ in range(20):
hash_obj.digest().hex()
if __name__ == "__main__":
num_workers = 40
num_runs = 100
barrier = threading.Barrier(num_workers)
def closure():
barrier.wait()
for _ in range(num_runs):
test_hashlib_sha256()
with concurrent.futures.ThreadPoolExecutor(max_workers=num_workers) as executor:
futures = []
for i in range(num_workers):
futures.append(executor.submit(closure))
assert len(list(f.result() for f in futures)) == num_workers
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You are visitor 01402622 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.