GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-117657: Fix race involving GC and heap initialization by colesbury · Pull Request #119923 · python/cpython

Conversation

@colesbury

@colesbury colesbury commented Jun 1, 2024 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

The _PyThreadState_Bind() function is called before PyEval_AcquireThread() so it's not synchronized with the stop the world GC. We had a race where gc_visit_heaps() might visit a thread's heap while it's being initialized.

Use a simple atomic int to avoid visiting heaps for threads that are not yet fully initialized (i.e., before tstate_mimalloc_bind() is called).

The race was reproducible by running:
python Lib/test/test_importlib/partial/pool_in_threads.py.

  • Issue: Make TSAN tests pass with the GIL disabled in free-threaded builds #117657

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@colesbury
pythongh-117657: Fix race involving GC and heap initialization
9f63a18
The `_PyThreadState_Bind()` function is called before the first
`PyEval_AcquireThread()` so it's not synchronized with the stop the
world GC. We had a race where `gc_visit_heaps()` might visit a thread's
heap while it's being initialized.

Use a simple atomic int to avoid visiting heaps for threads that are not
yet fully initialized (i.e., before `tstate_mimalloc_bind()` is called).

The race was reproducible by running:
`python Lib/test/test_importlib/partial/pool_in_threads.py`.
@colesbury colesbury added skip news topic-free-threading needs backport to 3.13 only security fixes labels Jun 1, 2024
@colesbury
colesbury requested review from DinoV and Fidget-Spinner June 1, 2024 20:42
@colesbury
colesbury requested a review from ericsnowcurrently as a code owner June 1, 2024 20:42
@bedevere-app bedevere-app Bot added the awaiting core review label Jun 1, 2024
@bedevere-app bedevere-app Bot mentioned this pull request Jun 1, 2024
Make TSAN tests pass with the GIL disabled in free-threaded builds #117657
Closed
@DinoV

DinoV commented Jun 3, 2024

Copy link
Copy Markdown
Contributor

I guess we could have also used the presence of current_object_heap and made that an atomic sign but I suppose the extra flag is a little clearer.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

DinoV
DinoV approved these changes Jun 3, 2024
View reviewed changes

@DinoV DinoV left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Jun 3, 2024
@colesbury
colesbury merged commit e69d068 into python:main Jun 4, 2024
@miss-islington-app

miss-islington-app Bot commented Jun 4, 2024

Copy link
Copy Markdown

Thanks @colesbury for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting merge label Jun 4, 2024
@colesbury
colesbury deleted the gh-117657-gc-mimalloc-bind branch June 4, 2024 13:42
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 4, 2024
@colesbury @miss-islington
pythongh-117657: Fix race involving GC and heap initialization (pytho…
30b6d5c
…nGH-119923)

The `_PyThreadState_Bind()` function is called before the first
`PyEval_AcquireThread()` so it's not synchronized with the stop the
world GC. We had a race where `gc_visit_heaps()` might visit a thread's
heap while it's being initialized.

Use a simple atomic int to avoid visiting heaps for threads that are not
yet fully initialized (i.e., before `tstate_mimalloc_bind()` is called).

The race was reproducible by running:
`python Lib/test/test_importlib/partial/pool_in_threads.py`.
(cherry picked from commit e69d068)

Co-authored-by: Sam Gross <[email protected]>
@bedevere-app

bedevere-app Bot commented Jun 4, 2024

Copy link
Copy Markdown

GH-120038 is a backport of this pull request to the 3.13 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 only security fixes label Jun 4, 2024
colesbury added a commit that referenced this pull request Jun 4, 2024
@miss-islington @colesbury
[3.13] gh-117657: Fix race involving GC and heap initialization (GH-1…
ee7d268
…19923) (#120038)

The `_PyThreadState_Bind()` function is called before the first
`PyEval_AcquireThread()` so it's not synchronized with the stop the
world GC. We had a race where `gc_visit_heaps()` might visit a thread's
heap while it's being initialized.

Use a simple atomic int to avoid visiting heaps for threads that are not
yet fully initialized (i.e., before `tstate_mimalloc_bind()` is called).

The race was reproducible by running:
`python Lib/test/test_importlib/partial/pool_in_threads.py`.
(cherry picked from commit e69d068)

Co-authored-by: Sam Gross <[email protected]>
barneygale pushed a commit to barneygale/cpython that referenced this pull request Jun 5, 2024
@colesbury @barneygale
pythongh-117657: Fix race involving GC and heap initialization (pytho…
9073dbb
…n#119923)

The `_PyThreadState_Bind()` function is called before the first
`PyEval_AcquireThread()` so it's not synchronized with the stop the
world GC. We had a race where `gc_visit_heaps()` might visit a thread's
heap while it's being initialized.

Use a simple atomic int to avoid visiting heaps for threads that are not
yet fully initialized (i.e., before `tstate_mimalloc_bind()` is called).

The race was reproducible by running:
`python Lib/test/test_importlib/partial/pool_in_threads.py`.
estyxx pushed a commit to estyxx/cpython that referenced this pull request Jul 17, 2024
@colesbury @estyxx
pythongh-117657: Fix race involving GC and heap initialization (pytho…
02ba316
…n#119923)

The `_PyThreadState_Bind()` function is called before the first
`PyEval_AcquireThread()` so it's not synchronized with the stop the
world GC. We had a race where `gc_visit_heaps()` might visit a thread's
heap while it's being initialized.

Use a simple atomic int to avoid visiting heaps for threads that are not
yet fully initialized (i.e., before `tstate_mimalloc_bind()` is called).

The race was reproducible by running:
`python Lib/test/test_importlib/partial/pool_in_threads.py`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@DinoV DinoV DinoV approved these changes
@Fidget-Spinner Fidget-Spinner Awaiting requested review from Fidget-Spinner
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@colesbury @DinoV
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.