GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-151218: Fix data race in sys_set_flag for free-threading by bhuvi27 · Pull Request #151220 · python/cpython

Conversation

@bhuvi27

@bhuvi27 bhuvi27 commented Jun 10, 2026 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Fixes #151218

Concurrent calls to sys.set_int_max_str_digits() in free-threaded builds
could double-free the same sys.flags tuple item because sys_set_flag()
updated the slot without synchronization.

Protect sys.flags updates with a mutex in free-threaded builds, and hold
the same lock across flag and interpreter int_max_str_digits state updates
so sys.get_int_max_str_digits() stays consistent with sys.flags. Add a
concurrent stress regression test in test_sys.py.

  • Issue: Data race in sys_set_flag when sys.set_int_max_str_digits() is called concurrently with free-threaded build #151218

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bhuvi27
bhuvi27 requested a review from ericsnowcurrently as a code owner June 10, 2026 05:16
@bedevere-app bedevere-app Bot added the awaiting review label Jun 10, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Jun 10, 2026
Data race in sys_set_flag when sys.set_int_max_str_digits() is called concurrently with free-threaded build #151218
Closed
@bhuvi27
bhuvi27 force-pushed the gh-151218-fix-sys-set-flag-race branch from 2bff4e1 to 099518a Compare June 10, 2026 06:30
@bhuvi27
pythongh-151218: fix data race in sys_set_flag for free-threading
09f17c8
Concurrent calls to sys.set_int_max_str_digits() in free-threaded builds
could double-free the same sys.flags tuple item because sys_set_flag()
updated the slot without synchronization.

Protect sys.flags updates with a mutex in free-threaded builds and hold
the same lock across the flag and interpreter int_max_str_digits state
updates so sys.get_int_max_str_digits() stays consistent with sys.flags.
@bhuvi27
bhuvi27 force-pushed the gh-151218-fix-sys-set-flag-race branch from 099518a to 09f17c8 Compare June 10, 2026 06:58
@picnixz

picnixz commented Jun 10, 2026

Copy link
Copy Markdown
Member

Please do not force push. To contribute cleanly, please read https://devguide.python.org/getting-started/pull-request-lifecycle/#pullrequest.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@da-woods

da-woods commented Jun 10, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

This doesn't guard the read though. And the read through sys.flags is probably difficult to guard because it's an "immutable" object and so shouldn't be changing once it's visible

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bhuvi27

bhuvi27 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

This doesn't guard the read though. And the read through sys.flags is probably difficult to guard because it's an "immutable" object and so shouldn't be changing once it's visible

Good point — you're right that the read path through sys.flags isn't
guarded by the new mutex, so a reader can still observe the old slot
while a writer replaces and decref's it (use-after-free).

A few options I see:

  1. Stop updating sys.flags.int_max_str_digits at runtime entirely —
    treat sys.flags as startup-only (its documented immutability),
    and let sys.get_int_max_str_digits() be the single source of truth
    for the current value. This removes the racy shared object.

  2. Atomic store/load of the slot pointer plus deferred reclamation
    (QSBR) for the old value — keeps the current API but is more
    machinery.

  3. Replace sys.flags with a fresh struct sequence on each update.

Option 1 looks cleanest and matches the "immutable" intent, but it's
a behaviour change. Which direction would you prefer? Happy to redo
the PR around option 1 if that's the right call.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz

picnixz commented Jun 10, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This conversation seems to go against our AI policy. Human interaction is necessary, don't let your agents in automated mode. Please read https://devguide.python.org/getting-started/ai-tools/#guidelines-for-using-ai-tools.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bhuvi27

bhuvi27 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

This conversation seems to go against our AI policy. Human interaction is necessary, don't let your agents in automated mode. Please read https://devguide.python.org/getting-started/ai-tools/#guidelines-for-using-ai-tools.

Sorry — I used an AI assistant to draft PR replies and posted them
without reviewing properly (the first one was off-topic). I'll handle
discussion myself from here.

On the technical side: I understand the mutex doesn't protect reads
through sys.flags. I'm happy to rework the approach once there's
guidance on whether runtime updates to sys.flags should stop entirely.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@skirpichev
skirpichev self-requested a review June 10, 2026 12:58
skirpichev
skirpichev reviewed Jun 11, 2026
View reviewed changes

@skirpichev skirpichev left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that exposing int_max_str_digits in the sys.flags was a mistake. This is an immutable named tuple for command-line flags.

But sys.flags.int_max_str_digits does not fit in this picture: it's for runtime settings, which can be changed by sys.set_int_max_str_digits().

I think this bug should be fixed by changing meaning of the sys.flags.int_max_str_digits. It will keep immutable value, provided as command-line option. We have a different API to access runtime settings, i.e. sys.get_int_max_str_digits(). CC @gpshead

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@gpshead

gpshead commented Jun 11, 2026

Copy link
Copy Markdown
Member

PyConfig_Set() per PEP 741 in 3.14 is a public API that mutates various other sys.flags values at runtime. It is no longer correct to claim that it is a tuple of command line flags that never changes after the interpreter starts. I do not think changing what happens with int_max_str_digits would be right at this point.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gpshead
gpshead requested changes Jun 11, 2026
View reviewed changes

@gpshead gpshead left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall I don't think this PR is the right way to accomplish this. we don't need a static global mutex. use Py_BEGIN_CRITICAL_SECTION on the flags object instead.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot added awaiting changes and removed awaiting review labels Jun 11, 2026
@bedevere-app

bedevere-app Bot commented Jun 11, 2026

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@da-woods

da-woods commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

we don't need a static global mutex. use Py_BEGIN_CRITICAL_SECTION on the flags object instead.

I don't think that works because there's no thread-safety guards on any of the reads from the flags object (because it's a tuple-like object so considered immutable)

overall I don't think this PR is the right way to accomplish this

I do agree with this though

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@skirpichev

skirpichev commented Jun 12, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

PyConfig_Set() per PEP 741 in 3.14 is a public API that mutates various other sys.flags values at runtime. It is no longer correct to claim that it is a tuple of command line flags that never changes after the interpreter starts. I do not think changing what happens with int_max_str_digits would be right at this point.

Indeed. Then struct sequence doesn't look to be the right data structure for this sys attribute anymore. Or, I suspect, we will reinvent the wheel (dict?).

I don't think that works because there's no thread-safety guards on any of the reads from the flags object (because it's a tuple-like object so considered immutable)

Maybe use a dict subclass (to make backward-compatible __getattr__) instead of the named tuple? Mutating of the struct sequence breaks the contract in docs:

void PyStructSequence_SetItem(PyObject *p, Py_ssize_t pos, PyObject *o)
[...]
Sets the field at index pos of the struct sequence p to value o. Like PyTuple_SET_ITEM(), this should only be used to fill in brand new instances.

CC @vstinner (as PEP author)

Edit: So far, option (3) from proposed above seems to be an alternative to this.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gpshead

gpshead commented Jun 12, 2026

Copy link
Copy Markdown
Member

its perfectly fine for it to be a structsequence. we should probably just do as tuples do and replace the instance with a new structsequence instance upon each mutation (a copy with one element changed) instead of trying to treat that as mutable or piling even more odd "some elements can change via getitem/getattr implementations returning new info" shapes on top of it.

the design mistake was ever using tuples for anything in the 1990s followed by recognizing that namedtuples weren't a solution to their ills in the 2000s. IIRC that's how structsequence came into being? an odd duck shaped thing that floats to preserve legacy API expectations while letting us add new .field_names.

the joys of the standard library and specifically old things like sys.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

vstinner
vstinner reviewed Jun 12, 2026
View reviewed changes
Comment thread Python/sysmodule.c
#include "pycore_initconfig.h" // _PyStatus_EXCEPTION()
#include "pycore_interpframe.h" // _PyFrame_GetFirstComplete()
#ifdef Py_GIL_DISABLED
# include "pycore_lock.h" // PyMutex_Lock()

@vstinner vstinner Jun 12, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyMutex_Lock() is part of Python.h. There is no need to include the internal pycore_lock.h header.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Python/sysmodule.c
assert(pos >= 0 && pos < (Py_ssize_t)(Py_ARRAY_LENGTH(flags_fields) - 1));

PyObject *old_value = PyStructSequence_GET_ITEM(flags, pos);
*p_old_value = PyStructSequence_GET_ITEM(flags, pos);

@vstinner vstinner Jun 12, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you call Py_XDECREF(old_value); in the caller? It seems same to keep Py_XDECREF(old_value); in this function, no?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Python/sysmodule.c
};

#ifdef Py_GIL_DISABLED
static PyMutex sys_flags_mutex;

@vstinner vstinner Jun 12, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a comment to explain the purpose of this lock. IMO it should only be used when modifying sys.flags. It's not needed to acquire this lock to get the sys.flags attribute.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Python/sysmodule.c


static int
_PySys_SetFlagInt(Py_ssize_t pos, int value)

@vstinner vstinner Jun 12, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can keep this function, it doesn't hurt.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Python/sysmodule.c
}

#ifdef Py_GIL_DISABLED
PyMutex_Lock(&sys_flags_mutex);

@vstinner vstinner Jun 12, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that you need to acquire the lock to get sys.flags. You only need the lock around the code changing sys.flags members.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner

vstinner commented Jun 12, 2026

Copy link
Copy Markdown
Member

You should also update _PySys_UpdateConfig() to acquire the lock to call set_flags_from_config().

make_flags() doesn't need to be modified, since sys.flags doesn't exist when it's called.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 12, 2026

Copy link
Copy Markdown
Member

I don't think that works because there's no thread-safety guards on any of the reads from the flags object (because it's a tuple-like object so considered immutable)

Would you mind to elaborate why a lock would also be needed to read a sys.flags flag?

sys.flags is a structseq instance which inherits from tuple. Getting a tuple item doesn't use any lock:

static PyObject *
tuple_item(PyObject *op, Py_ssize_t i)
{
    PyTupleObject *a = _PyTuple_CAST(op);
    if (i < 0 || i >= Py_SIZE(a)) {
        PyErr_SetString(PyExc_IndexError, "tuple index out of range");
        return NULL;
    }
    return Py_NewRef(a->ob_item[i]);
}

its perfectly fine for it to be a structsequence. we should probably just do as tuples do and replace the instance with a new structsequence instance upon each mutation

Ah, that sounds like a reasonable solution.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 12, 2026

Copy link
Copy Markdown
Member

I wrote PR gh-151402 to implement the other option: PyConfig_Set() and sys.set_int_max_str_digits() now replace sys.flags, instead of modifying sys.flags in-place.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bhuvi27

bhuvi27 commented Jun 12, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Thanks everyone for the review. I see the consensus is to replace sys.flags on mutation rather than locking in-place updates — and Victor's PR #151402 implements that. I'll close this PR.

vstinner reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bhuvi27 bhuvi27 closed this Jun 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner left review comments
@skirpichev skirpichev skirpichev left review comments
@gpshead gpshead gpshead requested changes
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting changes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Data race in sys_set_flag when sys.set_int_max_str_digits() is called concurrently with free-threaded build

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@bhuvi27 @picnixz @da-woods @gpshead @skirpichev @vstinner
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.