GeezerVison is showing github.com — GH-149142: Fix decimal `mpd_context_t.status` race in free-threading by LindaSummer · Pull Request #150598 · python/cpython
Conversation
Issue
GH-149142
Proposed Changes
Add mutex in free-threading build to guard the PyDecContextObject.ctx.status.
It solves the race in concurrent dec_addstatus and clear_flags.
Comment
For other flags like traps are not guarded in this PR.
I plan to update in later PRs with new test cases.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
chatgpt-codex-connector[bot] reacted with thumbs up emoji
|
During this patch, I find another race in decimal.DefaultContext.
In context_new, we may create a new context from decimal.DefaultContext.
But the decimal.DefaultContext could be updated in another thread.
|
if (state->default_context_template) { |
|
*ctx = *CTX(state->default_context_template); |
|
} |
So I think that we should have a way to get a snapshot of default_context_template just like the contextvar's HMAT in #146482.
Refer to our documentation we could see below:
This context is most useful in multi-threaded environments. Changing one of the fields before threads are started has the effect of setting system-wide defaults. Changing the fields after threads have started is not recommended as it would require thread synchronization to prevent race conditions.
I'm not sure if this case is valid refer to our documentation.
But even it is not suggested, I think that the TSAN may still need to be fixed.
If we plan to fix this one, I am appreciating to create a new issue and try to solve it.
Here is the demo reproduce code and TSAN report.
import decimal
import threading
N_THREADS = 8
ITERATIONS = 100_000
def run_concurrently(workers):
barrier = threading.Barrier(len(workers))
threads = [threading.Thread(target=worker, args=(barrier,))
for worker in workers]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
default_context = decimal.DefaultContext
def creator(barrier):
barrier.wait()
for _ in range(ITERATIONS):
decimal.Context()
def mutator(barrier):
barrier.wait()
for i in range(ITERATIONS):
default_context.prec = 9 + (i % 10)
default_context.Emax = 999999 - (i % 10)
default_context.Emin = -999999 + (i % 10)
default_context.create_decimal("1.23456")
default_context.clear_flags()
run_concurrently([creator] * N_THREADS + [mutator] * N_THREADS)TSAN report.
SUMMARY: ThreadSanitizer: data race Modules/_decimal/_decimal.c:616 in dec_addstatus
==================
==================
WARNING: ThreadSanitizer: data race (pid=1872326)
Write of size 4 at 0x7f21063640bc by thread T16:
#0 dec_addstatus Modules/_decimal/_decimal.c:616 (_decimal.cpython-316t-x86_64-linux-gnu.so+0xf475)
#1 PyDecType_FromCString Modules/_decimal/_decimal.c:2327 (_decimal.cpython-316t-x86_64-linux-gnu.so+0x13106)
#2 PyDecType_FromSequence Modules/_decimal/_decimal.c:2934 (_decimal.cpython-316t-x86_64-linux-gnu.so+0x13106)
#3 PyDec_FromObject Modules/_decimal/_decimal.c:3234 (_decimal.cpython-316t-x86_64-linux-gnu.so+0x13106)
#4 _decimal_Context_create_decimal_impl Modules/_decimal/_decimal.c:3292 (_decimal.cpython-316t-x86_64-linux-gnu.so+0x13106)
#5 _decimal_Context_create_decimal Modules/_decimal/clinic/_decimal.c.h:853 (_decimal.cpython-316t-x86_64-linux-gnu.so+0x13106)
#6 _Py_BuiltinCallFast_StackRef Python/ceval.c:815 (python3.16+0x3bfa1b)
#7 _PyEval_EvalFrameDefault Python/generated_cases.c.h:2420 (python3.16+0x3df9d5)
#8 _PyEval_EvalFrame Include/internal/pycore_ceval.h:122 (python3.16+0x3ec7d7)
#9 _PyEval_Vector Python/ceval.c:2134 (python3.16+0x3ec7d7)
#10 _PyFunction_Vectorcall Objects/call.c:413 (python3.16+0x152110)
#11 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x15370f)
#12 _PyObject_VectorcallPrepend Objects/call.c:855 (python3.16+0x15370f)
#13 method_vectorcall Objects/classobject.c:55 (python3.16+0x1585b6)
#14 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x42729b)
#15 context_run Python/context.c:728 (python3.16+0x42729b)
#16 method_vectorcall_FASTCALL_KEYWORDS Objects/descrobject.c:421 (python3.16+0x1743fe)
#17 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x151cfc)
#18 PyObject_Vectorcall Objects/call.c:327 (python3.16+0x151cfc)
#19 _Py_VectorCallInstrumentation_StackRefSteal Python/ceval.c:766 (python3.16+0x3bf555)
#20 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1846 (python3.16+0x3cf120)
#21 _PyEval_EvalFrame Include/internal/pycore_ceval.h:122 (python3.16+0x3ec7d7)
#22 _PyEval_Vector Python/ceval.c:2134 (python3.16+0x3ec7d7)
#23 _PyFunction_Vectorcall Objects/call.c:413 (python3.16+0x152110)
#24 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x15370f)
#25 _PyObject_VectorcallPrepend Objects/call.c:855 (python3.16+0x15370f)
#26 method_vectorcall Objects/classobject.c:55 (python3.16+0x1585b6)
#27 _PyVectorcall_Call Objects/call.c:273 (python3.16+0x154be8)
#28 _PyObject_Call Objects/call.c:348 (python3.16+0x154fe1)
#29 PyObject_Call Objects/call.c:373 (python3.16+0x154fe1)
#30 thread_run Modules/_threadmodule.c:388 (python3.16+0x608d30)
#31 pythread_wrapper Python/thread_pthread.h:234 (python3.16+0x5072d2)
#32 <null> <null> (libtsan.so.2+0x4cf1a)
Previous read of size 8 at 0x7f21063640b8 by thread T1:
#0 context_new Modules/_decimal/_decimal.c:1468 (_decimal.cpython-316t-x86_64-linux-gnu.so+0xd6af)
#1 type_call Objects/typeobject.c:2467 (python3.16+0x28839b)
#2 _PyObject_MakeTpCall Objects/call.c:242 (python3.16+0x150dae)
#3 _PyObject_VectorcallTstate Include/internal/pycore_call.h:142 (python3.16+0x151d61)
#4 _PyObject_VectorcallTstate Include/internal/pycore_call.h:129 (python3.16+0x151d61)
#5 PyObject_Vectorcall Objects/call.c:327 (python3.16+0x151d61)
#6 _Py_VectorCall_StackRefSteal Python/ceval.c:724 (python3.16+0x3bf125)
#7 _PyEval_EvalFrameDefault Python/generated_cases.c.h:4362 (python3.16+0x3db2fc)
#8 _PyEval_EvalFrame Include/internal/pycore_ceval.h:122 (python3.16+0x3ec7d7)
#9 _PyEval_Vector Python/ceval.c:2134 (python3.16+0x3ec7d7)
#10 _PyFunction_Vectorcall Objects/call.c:413 (python3.16+0x152110)
#11 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x15370f)
#12 _PyObject_VectorcallPrepend Objects/call.c:855 (python3.16+0x15370f)
#13 method_vectorcall Objects/classobject.c:55 (python3.16+0x1585b6)
#14 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x42729b)
#15 context_run Python/context.c:728 (python3.16+0x42729b)
#16 method_vectorcall_FASTCALL_KEYWORDS Objects/descrobject.c:421 (python3.16+0x1743fe)
#17 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x151cfc)
#18 PyObject_Vectorcall Objects/call.c:327 (python3.16+0x151cfc)
#19 _Py_VectorCallInstrumentation_StackRefSteal Python/ceval.c:766 (python3.16+0x3bf555)
#20 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1846 (python3.16+0x3cf120)
#21 _PyEval_EvalFrame Include/internal/pycore_ceval.h:122 (python3.16+0x3ec7d7)
#22 _PyEval_Vector Python/ceval.c:2134 (python3.16+0x3ec7d7)
#23 _PyFunction_Vectorcall Objects/call.c:413 (python3.16+0x152110)
#24 _PyObject_VectorcallTstate Include/internal/pycore_call.h:144 (python3.16+0x15370f)
#25 _PyObject_VectorcallPrepend Objects/call.c:855 (python3.16+0x15370f)
#26 method_vectorcall Objects/classobject.c:55 (python3.16+0x1585b6)
#27 _PyVectorcall_Call Objects/call.c:273 (python3.16+0x154be8)
#28 _PyObject_Call Objects/call.c:348 (python3.16+0x154fe1)
#29 PyObject_Call Objects/call.c:373 (python3.16+0x154fe1)
#30 thread_run Modules/_threadmodule.c:388 (python3.16+0x608d30)
#31 pythread_wrapper Python/thread_pthread.h:234 (python3.16+0x5072d2)
#32 <null> <null> (libtsan.so.2+0x4cf1a) |
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Hi @devdanzin ,
Could you help take a review?
Please correct me if I made any mistake or misunderstanding.
Wish you a good day! ❤
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Refer to our documentation we could see below
You refer to the wrong place in docs. They have a dedicated sections, that says:
To control the defaults so that each thread will use the same values throughout the application, directly modify the DefaultContext object. This should be done before any threads are started so that there won’t be a race condition between threads calling getcontext().
BTW, I'm not sure that referenced issue is valid.
|
LindaSummer reacted with heart emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Refer to our documentation we could see below
You refer to the wrong place in docs. They have a dedicated sections, that says:
To control the defaults so that each thread will use the same values throughout the application, directly modify the DefaultContext object. This should be done before any threads are started so that there won’t be a race condition between threads calling getcontext().
BTW, I'm not sure that referenced issue is valid.
Hi @skirpichev ,
Thanks very much for your time and correction!
Got it! The doc has description of this race condition case, I wouldn't make it a new issue except it breaks the contract.
By the way, could I ask for your your review and give me some suggestions on this patch?
Wish you a good day!🌞
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
could I ask for your your review and give me some suggestions on this patch?
I'm not sure that referenced issue is valid. Perhaps, we should change docs instead.
|
LindaSummer reacted with heart emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
This PR is stale because it has been open for 90 days with no activity.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|