GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-104341: Call _PyEval_ReleaseLock() with NULL When Finalizing the Current Thread by ericsnowcurrently · Pull Request #105109 · python/cpython

Conversation

@ericsnowcurrently

@ericsnowcurrently ericsnowcurrently commented May 30, 2023 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads.

This is a much simpler approach to solving the race than in other PRs I've posted. I'd still like to pursue some of those other ideas but that can be done separately for 3.13+.

(The idea for this approach came out of discussions with @markshannon.)

  • Issue: test__xxsubinterpreters is Occasionally Crashing #104341

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@ericsnowcurrently
Call _PyEval_ReleaseLock() with NULL when finalizing the current thread.
5206f3d
@ericsnowcurrently ericsnowcurrently added skip news needs backport to 3.12 only security fixes labels May 30, 2023
@ericsnowcurrently
ericsnowcurrently requested a review from markshannon May 30, 2023 17:13
@bedevere-bot bedevere-bot mentioned this pull request May 30, 2023
test__xxsubinterpreters is Occasionally Crashing #104341
Closed
ericsnowcurrently
ericsnowcurrently commented May 30, 2023
View reviewed changes
Comment thread Python/ceval_gil.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Python/ceval_gil.c

void
_PyEval_ReleaseLock(PyThreadState *tstate)
_PyEval_ReleaseLock(PyInterpreterState *interp, PyThreadState *tstate)

@ericsnowcurrently ericsnowcurrently May 30, 2023

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other key part of this change is here, where we pass in the interpreter separately from the thread state, which allows tstate to be NULL.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
ericsnowcurrently added 2 commits May 30, 2023 14:06
@ericsnowcurrently
Temporarily adjust tstate->_status.cleared during interpreter_clear().
db6d7ac
@ericsnowcurrently
Disable the broken asserts.
a374f48
@ericsnowcurrently
ericsnowcurrently marked this pull request as ready for review May 30, 2023 21:10
@bedevere-bot bedevere-bot added the awaiting core review label May 30, 2023
@ericsnowcurrently
ericsnowcurrently requested review from Yhg1s, pitrou and tim-one May 30, 2023 22:03
gpshead
gpshead reviewed May 30, 2023
View reviewed changes
Comment thread Python/ceval_gil.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Python/pystate.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
ericsnowcurrently added 2 commits May 30, 2023 18:04
@ericsnowcurrently
Add a note about the race in drop_gil().
432db0c
@ericsnowcurrently
Fix a typo.
1280e0d
@ericsnowcurrently ericsnowcurrently mentioned this pull request May 31, 2023
GH-86275: Implementation of hypothesis stubs for property-based tests, with zoneinfo tests #22863
Merged
pitrou
pitrou reviewed May 31, 2023
View reviewed changes
Comment thread Python/ceval_gil.c Outdated
static void
drop_gil(struct _ceval_state *ceval, PyThreadState *tstate)
{
/* We shouldn't be using a thread state that isn't viable any more. */

@pitrou pitrou May 31, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is cryptic here. It doesn't say why it's here nor in which situation the "non-viable thread state" occurs.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
erlend-aasland reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@ericsnowcurrently ericsnowcurrently May 31, 2023

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've clarified the comment.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@ericsnowcurrently
Clarify a comment.
cc1f2fe
@ericsnowcurrently

ericsnowcurrently commented Jun 1, 2023

Copy link
Copy Markdown
Member Author

@gpshead, @pitrou, do you have any objections to this PR at this point? I'd like to land this ASAP for beta 2. Thanks!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@gpshead
minor comment typo fix.
b64b299
gpshead
gpshead approved these changes Jun 1, 2023
View reviewed changes
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Jun 1, 2023
@gpshead

gpshead commented Jun 1, 2023

Copy link
Copy Markdown
Member

any objections to this PR at this point? I'd like to land this ASAP for beta 2. Thanks!

I think we're good to go: no objections from me. I never assume I can have full confidence about modifications to this code, but CI and buildbots and ultimately beta2 testing should reveal more if anything else lurks...
ericsnowcurrently reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@ericsnowcurrently

ericsnowcurrently commented Jun 1, 2023

Copy link
Copy Markdown
Member Author

Thanks you!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@ericsnowcurrently
ericsnowcurrently merged commit 3698fda into python:main Jun 1, 2023
@miss-islington

miss-islington commented Jun 1, 2023

Copy link
Copy Markdown
Contributor

Thanks @ericsnowcurrently for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the awaiting merge label Jun 1, 2023
@miss-islington

miss-islington commented Jun 1, 2023

Copy link
Copy Markdown
Contributor

Sorry @ericsnowcurrently, I had trouble checking out the 3.12 backport branch.
Please retry by removing and re-adding the "needs backport to 3.12" label.
Alternatively, you can backport using cherry_picker on the command line.
cherry_picker 3698fda06eefb3c01e78c4c07f46fcdd0559e0f6 3.12
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@miss-islington miss-islington assigned ericsnowcurrently Jun 1, 2023
@ericsnowcurrently
ericsnowcurrently deleted the fix-threading-module-shutdown-3.12 branch June 1, 2023 22:24
@ericsnowcurrently ericsnowcurrently added needs backport to 3.12 only security fixes and removed needs backport to 3.12 only security fixes labels Jun 1, 2023
@miss-islington

miss-islington commented Jun 1, 2023

Copy link
Copy Markdown
Contributor

Thanks @ericsnowcurrently for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 1, 2023
@ericsnowcurrently @miss-islington
pythongh-104341: Call _PyEval_ReleaseLock() with NULL When Finalizing…
d7f46a8 
… the Current Thread (pythongh-105109)

This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads.

(The idea for this approach came out of discussions with @markshannon.)
(cherry picked from commit 3698fda)

Co-authored-by: Eric Snow <[email protected]>
@bedevere-bot

bedevere-bot commented Jun 1, 2023

Copy link
Copy Markdown

GH-105209 is a backport of this pull request to the 3.12 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the needs backport to 3.12 only security fixes label Jun 1, 2023
ericsnowcurrently pushed a commit that referenced this pull request Jun 1, 2023
@miss-islington
[3.12] gh-104341: Call _PyEval_ReleaseLock() with NULL When Finalizin…
d2be5c7 
…g the Current Thread (gh-105109) (gh-105209)

This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads.

(The idea for this approach came out of discussions with @markshannon.)
(cherry picked from commit 3698fda)

Co-authored-by: Eric Snow [email protected]
@erlend-aasland erlend-aasland mentioned this pull request Jun 6, 2023
3.12 backport gh 105236 #105358
Closed
@ericsnowcurrently ericsnowcurrently mentioned this pull request Oct 25, 2023
threading Thread.join should call the OS join API #110829
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pitrou pitrou pitrou left review comments
@gpshead gpshead gpshead approved these changes
@markshannon markshannon Awaiting requested review from markshannon
@tim-one tim-one Awaiting requested review from tim-one
@Yhg1s Yhg1s Awaiting requested review from Yhg1s

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@ericsnowcurrently ericsnowcurrently

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@ericsnowcurrently @gpshead @miss-islington @bedevere-bot @pitrou
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.