GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-140260: Fix data race in _struct module initialization with subinterpreters by ashm-dev · Pull Request #140909 · python/cpython

Conversation

@ashm-dev

@ashm-dev ashm-dev commented Nov 2, 2025 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor
  • Issue: ThreadSanitizer: data race in _struct module initialization with InterpreterPoolExecutor (free-threading build) #140260

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@ashm-dev
Fix data race in _struct module endian table initialization
243b5f0
@bedevere-app bedevere-app Bot added the awaiting review label Nov 2, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Nov 2, 2025
ThreadSanitizer: data race in _struct module initialization with InterpreterPoolExecutor (free-threading build) #140260
Closed
@ashm-dev
fix doc
4c9892e
@ashm-dev
ashm-dev requested a review from ericsnowcurrently as a code owner November 2, 2025 12:58
@ashm-dev
refactor(c-analyzer): add endian_tables_init_once to globals-to-fix list
95958e8
@ashm-dev
ashm-dev force-pushed the tsan branch from 64f49c0 to 95958e8 Compare November 2, 2025 13:10
ZeroIntensity
ZeroIntensity reviewed Nov 2, 2025
View reviewed changes
Comment thread Modules/_struct.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@ZeroIntensity ZeroIntensity added needs backport to 3.14 bugs and security fixes topic-subinterpreters labels Nov 2, 2025
ZeroIntensity
ZeroIntensity reviewed Nov 2, 2025
View reviewed changes
Comment thread Modules/_struct.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@ashm-dev
Use Py_UNUSED for unused arg in init_endian_tables
3130512
@ashm-dev
ashm-dev force-pushed the tsan branch from 68ffe6d to 3130512 Compare November 2, 2025 16:28
@ashm-dev
ashm-dev requested a review from ZeroIntensity November 2, 2025 16:47
ZeroIntensity
ZeroIntensity reviewed Nov 2, 2025
View reviewed changes

@ZeroIntensity ZeroIntensity left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a test stressing this.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
picnixz
picnixz reviewed Nov 2, 2025
View reviewed changes
Comment thread Modules/_struct.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@ashm-dev
fix: remove duplicate include of pycore_lock.h in _struct.c
d33514b
@ashm-dev
ashm-dev requested a review from ZeroIntensity November 3, 2025 14:31
ZeroIntensity
ZeroIntensity reviewed Nov 3, 2025
View reviewed changes
Comment thread Lib/test/test_struct.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
@ashm-dev
add test
c4f577f
@ashm-dev
ashm-dev force-pushed the tsan branch from 219b7df to c4f577f Compare November 3, 2025 16:13
@ashm-dev
ashm-dev requested a review from ZeroIntensity November 3, 2025 16:22
ZeroIntensity
ZeroIntensity reviewed Nov 3, 2025
View reviewed changes
Comment thread Lib/test/test_struct.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/test/test_struct.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/test/test_struct.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Modules/_struct.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@ashm-dev
Remove return check for endian table init
238c40e
@ashm-dev
ashm-dev force-pushed the tsan branch from e21838a to 238c40e Compare November 3, 2025 16:39
@ashm-dev
Use InterpreterPoolExecutor in struct tests
f4d24ff
@ashm-dev
ashm-dev requested review from ZeroIntensity and picnixz November 3, 2025 16:52
ZeroIntensity
ZeroIntensity reviewed Nov 3, 2025
View reviewed changes
Comment thread Lib/test/test_struct.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/test/test_struct.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Lib/test/test_struct.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
ZeroIntensity
ZeroIntensity reviewed Nov 3, 2025
View reviewed changes
Comment thread Tools/c-analyzer/cpython/globals-to-fix.tsv Resolved

Uh oh!

There was an error while loading. Please reload this page.
@ZeroIntensity

ZeroIntensity commented Nov 5, 2025

Copy link
Copy Markdown
Member

I think the failing job will be solved by a rebase.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
ZeroIntensity
ZeroIntensity approved these changes Nov 5, 2025
View reviewed changes

@ZeroIntensity ZeroIntensity left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
ashm-dev reacted with heart emoji
All reactions
  • ❤️ 1 reaction
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Nov 5, 2025
@ZeroIntensity

ZeroIntensity commented Nov 5, 2025

Copy link
Copy Markdown
Member

Hm, that failure looks unrelated. @encukou, is it possible that #138190 broke something?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@kumaraditya303
Merge branch 'main' into tsan
434dbab
kumaraditya303
kumaraditya303 approved these changes Nov 6, 2025
View reviewed changes
@kumaraditya303
kumaraditya303 enabled auto-merge (squash) November 6, 2025 05:54
@encukou

encukou commented Nov 6, 2025

Copy link
Copy Markdown
Member

It did (now fixed thanks to Victor), but I don't think it broke mobile. Maybe it's sequential tests, or cross-compilation?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@kumaraditya303

kumaraditya303 commented Nov 11, 2025

Copy link
Copy Markdown
Contributor

Seems like Android and iOS do not support subinterpreters so the test should be skipped on those platforms.
ashm-dev reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@ZeroIntensity
ZeroIntensity disabled auto-merge November 11, 2025 20:31
@ZeroIntensity

ZeroIntensity commented Nov 11, 2025

Copy link
Copy Markdown
Member

Good catch, I disabled auto-merge for the time being.
ashm-dev reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@ashm-dev
fix: tests
3844826
@ashm-dev
ashm-dev requested review from ZeroIntensity and kumaraditya303 November 13, 2025 09:29
@kumaraditya303
Merge branch 'main' into tsan
b825107
@kumaraditya303
kumaraditya303 merged commit 63548b3 into python:main Nov 13, 2025
46 checks passed

Uh oh!

There was an error while loading. Please reload this page.
@miss-islington-app

miss-islington-app Bot commented Nov 13, 2025

Copy link
Copy Markdown

Thanks @ashm-dev for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the awaiting merge label Nov 13, 2025
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 13, 2025
@ashm-dev @miss-islington
pythongh-140260: fix data race in _struct module initialization wit…
a154c08 
…h subinterpreters (pythonGH-140909)

(cherry picked from commit 63548b3)

Co-authored-by: Shamil <[email protected]>
@bedevere-app

bedevere-app Bot commented Nov 13, 2025

Copy link
Copy Markdown

GH-141501 is a backport of this pull request to the 3.14 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Nov 13, 2025
@ashm-dev
ashm-dev deleted the tsan branch November 13, 2025 11:03
@bedevere-bot

bedevere-bot commented Nov 13, 2025

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.x (tier-1) has failed when building commit 63548b3.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/345/builds/12669) and take a look at the build logs.
  4. Check if the failure is related to this commit (63548b3) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/345/builds/12669

Failed tests:

  • test.test_os.test_os

Failed subtests:

  • test_timerfd_interval - test.test_os.test_os.TimerfdTests.test_timerfd_interval

Summary of the results of the build (if available):

==

Click to see traceback logs 
Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_os/test_os.py", line 4017, in test_timerfd_interval
    self.assertEqual(self.read_count_signaled(fd), count)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 4 != 3
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
kumaraditya303 pushed a commit that referenced this pull request Nov 13, 2025
@miss-islington @ashm-dev
[3.14] gh-140260: fix data race in _struct module initialization wi…
79195df 
…th subinterpreters (GH-140909) (#141501)

gh-140260: fix data race in `_struct` module initialization with subinterpreters (GH-140909)
(cherry picked from commit 63548b3)

Co-authored-by: Shamil <[email protected]>
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request Nov 13, 2025
@CuriousLearner
Merge branch 'main' into fix-issue21914
b9f473c 
* main: (463 commits)
  pythongh-140601: Add ResourceWarning to iterparse when not closed (pythonGH-140603)
  pythongh-137969: Fix double evaluation of `ForwardRef`s which rely on globals (python#140974)
  pythongh-139109: A new tracing JIT compiler frontend for CPython (pythonGH-140310)
  pythongh-141004: Document `PyErr_RangedSyntaxLocationObject` (python#141521)
  pythongh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (pythonGH-140884)
  pythongh-139653: Add PyUnstable_ThreadState_SetStackProtection() (python#139668)
  pythongh-141004: Document `PyCode_Optimize` (pythonGH-141378)
  pythongh-141004: Document C APIs for dictionary keys, values, and items (pythonGH-141009)
  pythongh-137959: Fix `TIER1_TO_TIER2` macro name in JIT InternalDocs (pythonGH-141496)
  pythongh-139871: Add `bytearray.take_bytes([n])` to efficiently extract `bytes` (pythonGH-140128)
  pythongh-140601: Refactor ElementTree.iterparse() tests (pythonGH-141499)
  pythongh-135801: Add the module parameter to compile() etc (pythonGH-139652)
  pythongh-140260: fix data race in `_struct` module initialization with subinterpreters (python#140909)
  pythongh-137109: refactor warning about threads when forking (python#141438)
  pythongh-141004: Document `PyRun_InteractiveOneObject` (pythonGH-141405)
  pythongh-124111: Fix TCL 9 thread detection (pythonGH-128103)
  pythongh-141442: Add escaping to iOS testbed arguments (python#141443)
  pythongh-140936: Fix JIT assertion crash at finalization if some generator is alive (pythonGH-140969)
  Add details about JIT build infrastructure and updating dependencies to `Tools/jit` (python#141167)
  pythongh-141412: Use reliable target URL for urllib example (pythonGH-141428)
  ...
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request Nov 14, 2025
@CuriousLearner
Merge branch 'main' of github.com:python/cpython into fix-Issue18387
c46ecc5 
* 'main' of github.com:python/cpython: (464 commits)
  pythongh-140601: Add ResourceWarning to iterparse when not closed (pythonGH-140603)
  pythongh-137969: Fix double evaluation of `ForwardRef`s which rely on globals (python#140974)
  pythongh-139109: A new tracing JIT compiler frontend for CPython (pythonGH-140310)
  pythongh-141004: Document `PyErr_RangedSyntaxLocationObject` (python#141521)
  pythongh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (pythonGH-140884)
  pythongh-139653: Add PyUnstable_ThreadState_SetStackProtection() (python#139668)
  pythongh-141004: Document `PyCode_Optimize` (pythonGH-141378)
  pythongh-141004: Document C APIs for dictionary keys, values, and items (pythonGH-141009)
  pythongh-137959: Fix `TIER1_TO_TIER2` macro name in JIT InternalDocs (pythonGH-141496)
  pythongh-139871: Add `bytearray.take_bytes([n])` to efficiently extract `bytes` (pythonGH-140128)
  pythongh-140601: Refactor ElementTree.iterparse() tests (pythonGH-141499)
  pythongh-135801: Add the module parameter to compile() etc (pythonGH-139652)
  pythongh-140260: fix data race in `_struct` module initialization with subinterpreters (python#140909)
  pythongh-137109: refactor warning about threads when forking (python#141438)
  pythongh-141004: Document `PyRun_InteractiveOneObject` (pythonGH-141405)
  pythongh-124111: Fix TCL 9 thread detection (pythonGH-128103)
  pythongh-141442: Add escaping to iOS testbed arguments (python#141443)
  pythongh-140936: Fix JIT assertion crash at finalization if some generator is alive (pythonGH-140969)
  Add details about JIT build infrastructure and updating dependencies to `Tools/jit` (python#141167)
  pythongh-141412: Use reliable target URL for urllib example (pythonGH-141428)
  ...
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request Nov 14, 2025
@CuriousLearner
Merge branch 'main' of github.com:python/cpython into fix-issue24255
c240289 
* 'main' of github.com:python/cpython: (464 commits)
  pythongh-140601: Add ResourceWarning to iterparse when not closed (pythonGH-140603)
  pythongh-137969: Fix double evaluation of `ForwardRef`s which rely on globals (python#140974)
  pythongh-139109: A new tracing JIT compiler frontend for CPython (pythonGH-140310)
  pythongh-141004: Document `PyErr_RangedSyntaxLocationObject` (python#141521)
  pythongh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (pythonGH-140884)
  pythongh-139653: Add PyUnstable_ThreadState_SetStackProtection() (python#139668)
  pythongh-141004: Document `PyCode_Optimize` (pythonGH-141378)
  pythongh-141004: Document C APIs for dictionary keys, values, and items (pythonGH-141009)
  pythongh-137959: Fix `TIER1_TO_TIER2` macro name in JIT InternalDocs (pythonGH-141496)
  pythongh-139871: Add `bytearray.take_bytes([n])` to efficiently extract `bytes` (pythonGH-140128)
  pythongh-140601: Refactor ElementTree.iterparse() tests (pythonGH-141499)
  pythongh-135801: Add the module parameter to compile() etc (pythonGH-139652)
  pythongh-140260: fix data race in `_struct` module initialization with subinterpreters (python#140909)
  pythongh-137109: refactor warning about threads when forking (python#141438)
  pythongh-141004: Document `PyRun_InteractiveOneObject` (pythonGH-141405)
  pythongh-124111: Fix TCL 9 thread detection (pythonGH-128103)
  pythongh-141442: Add escaping to iOS testbed arguments (python#141443)
  pythongh-140936: Fix JIT assertion crash at finalization if some generator is alive (pythonGH-140969)
  Add details about JIT build infrastructure and updating dependencies to `Tools/jit` (python#141167)
  pythongh-141412: Use reliable target URL for urllib example (pythonGH-141428)
  ...
@mhsmith

mhsmith commented Nov 20, 2025

Copy link
Copy Markdown
Member

Seems like Android and iOS do not support subinterpreters so the test should be skipped on those platforms.

Based on discussions on Discord, it looks like this failure would happen on any platform if you run the tests with --single-process. So Android and iOS actually do support subintepreters, it's just that they're the only platforms that use --single-process in GitHub Actions.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025
@ashm-dev @StanFromIreland
pythongh-140260: fix data race in _struct module initialization wit…
1ce571b 
…h subinterpreters (python#140909)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner
@picnixz picnixz Awaiting requested review from picnixz
@ZeroIntensity ZeroIntensity Awaiting requested review from ZeroIntensity
@kumaraditya303 kumaraditya303 Awaiting requested review from kumaraditya303

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

topic-subinterpreters

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@ashm-dev @ZeroIntensity @encukou @kumaraditya303 @bedevere-bot @mhsmith @picnixz
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.