GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-117657: Quiet more TSAN warnings due to incorrect modeling of compare/exchange by mpage · Pull Request #117830 · python/cpython

Conversation

@mpage

@mpage mpage commented Apr 13, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

TSAN reports (example) data races between the non-atomic loads of tstate->state in detach_thread() and tstate_set_detached():

cpython/Python/pystate.c

Lines 2067 to 2080 in eca5362

static void
detach_thread(PyThreadState *tstate, int detached_state)
{
// XXX assert(tstate_is_alive(tstate) && tstate_is_bound(tstate));
assert(tstate->state == _Py_THREAD_ATTACHED);
assert(tstate == current_fast_get());
if (tstate->critical_section != 0) {
_PyCriticalSection_SuspendAll(tstate);
}
#ifdef Py_GIL_DISABLED
_Py_qsbr_detach(((_PyThreadStateImpl *)tstate)->qsbr);
#endif
tstate_deactivate(tstate);
tstate_set_detached(tstate, detached_state);

cpython/Python/pystate.c

Lines 2003 to 2008 in eca5362

static void
tstate_set_detached(PyThreadState *tstate, int detached_state)
{
assert(tstate->state == _Py_THREAD_ATTACHED);
#ifdef Py_GIL_DISABLED
_Py_atomic_store_int(&tstate->state, detached_state);

and the atomic compare/exchange of tstate->state in park_detached_threads():

cpython/Python/pystate.c

Lines 2159 to 2170 in eca5362

static bool
park_detached_threads(struct _stoptheworld_state *stw)
{
int num_parked = 0;
PyInterpreterState *i;
PyThreadState *t;
_Py_FOR_EACH_THREAD(stw, i, t) {
int state = _Py_atomic_load_int_relaxed(&t->state);
if (state == _Py_THREAD_DETACHED) {
// Atomically transition to "suspended" if in "detached" state.
if (_Py_atomic_compare_exchange_int(&t->state,
&state, _Py_THREAD_SUSPENDED)) {

I believe this is due to a bug with how TSAN models compare/exchange. Note that this appears to be a different issue than what we're seeing in #117828; the compare/exhange is succeeding in this case.

According to the standard,

If one evaluation modifies a memory location, and the other reads or modifies the same memory location, and if at least one of the evaluations is not an atomic operation, the behavior of the program is undefined (the program has a data race) unless there exists a happens-before relationship between these two evaluations.

However, according to the standard, A strongly happens-before B if any of the following are true:

  1. A is sequenced-before B.
  2. A synchronizes-with B.
  3. A strongly happens-before X, and X strongly happens-before B.

Using this we can establish a happens-before relationship between each non-atomic load and the compare/exchange:

  1. Each non-atomic load is sequenced-before the seq-cst store to tstate->state that happens in tstate_set_detached(). Therefore, each non-atomic load happens-before the seq-cst store.
  2. The seq-cst store to tstate->state in tstate_set_detached() synchronizes-with the compare/exchange to tstate->state in park_detached_threads(). Thus the seq-cst store happens-before the compare/exchange.
  3. (1) and (2) establish the happens-before relationship between each non-atomic load and the compare/exchange on tstate->state, therefore they should not be flagged as a data races.
  • Issue: Make TSAN tests pass with the GIL disabled in free-threaded builds #117657

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@mpage
Quiet more TSAN warnings due to incorrect modeling of compare/exchange
f9a4eec
TSAN complains about data races between the non-atomic load of `tstate->state` in
`detach_thread()` and `tstate_set_detached()` and the atomic compare/exchange of
`tstate->tstate` in `park_detached_threads()`. I believe this is due to a bug with
how TSAN models compare/exchange (treating it solely as a store, rather than a load/store).

According to the standard,

> If one evaluation modifies a memory location, and the other
> reads or modifies the same memory location, and if at least one
> of the evaluations is not an atomic operation, the behavior of
> the program is undefined (the program has a data race) unless
> there exists a happens-before relationship between these two
> evaluations.

However, according to the standard, A happens-before B if any of the following are
true:

> 1) A is sequenced-before B.
> 2) A synchronizes-with B.
> 3) A strongly happens-before X, and X strongly happens-before B.

Using this we can establish a happens-before relationship between each non-atomic
load and the compare/exchange:

1. Each non-atomic load is sequenced-before the seq-cst store to `tstate->state` that
   happens in `tstate_set_detached()`. Therefore, each non-atomic load happens-before
   the seq-cst store.
2. The seq-cst store to `tstate->state` in `tstate_set_detached()` synchronizes-with
   the compare/exchange to `tstate->state` in `park_detached_threads()`. Thus the seq-cst
   store happens-before the compare/exchange.
3. (1) and (2) establish the happens-before relationship between the non-atomic load and
   the compare/exchange on `tstate->state`, therefore it should not be flagged as a
   data race.
@bedevere-app bedevere-app Bot mentioned this pull request Apr 13, 2024
Make TSAN tests pass with the GIL disabled in free-threaded builds #117657
Closed
@mpage mpage added the skip news label Apr 13, 2024
@mpage
mpage requested a review from colesbury April 13, 2024 01:08
@mpage
mpage marked this pull request as ready for review April 13, 2024 01:08
@mpage
mpage requested a review from ericsnowcurrently as a code owner April 13, 2024 01:08
@bedevere-app bedevere-app Bot added the awaiting review label Apr 13, 2024
colesbury
colesbury approved these changes Apr 15, 2024
View reviewed changes
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Apr 15, 2024
@colesbury
colesbury merged commit 0d52383 into python:main Apr 15, 2024
@bedevere-app bedevere-app Bot removed the awaiting merge label Apr 15, 2024
mpage added a commit to mpage/cpython that referenced this pull request Apr 16, 2024
@mpage
Remove unneeded free-threading suppressions
da4df93
The underlying races were fixed in pythongh-117830 and pythongh-117828
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
@mpage @diegorusso
pythongh-117657: Quiet more TSAN warnings due to incorrect modeling o…
f8e2d81
…f compare/exchange (python#117830)
mpage added a commit to mpage/cpython that referenced this pull request Apr 22, 2024
@mpage
Quiet TSAN warnings about remaining non-atomic accesses of `tstate->s…
c309c66
…tate`

These should be the last instances of TSAN warning of data races when accessing
`tstate->state` non-atomically.

TSAN reports a race between accessing `tstate->state` in `_PyThreadState_Suspend()`
and the compare/exhange in `park_detached_threads`. This is a false positive due
to TSAN modeling failed compare/exchange operations as writes.

TSAN reports a race between accessing `tstate->state` in `_PySemaphore_Wait()`
and the compare/exchange in `park_detached_threads`. This is the same issue that
we saw in pythongh-117830.
@mpage mpage mentioned this pull request Apr 22, 2024
gh-117657: Quiet TSAN warnings about remaining non-atomic accesses of tstate->state #118165
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@colesbury colesbury colesbury approved these changes
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@mpage @colesbury
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.