GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-96387: take_gil() resets drop request before exit by vstinner · Pull Request #96869 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented Sep 16, 2022 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

At Python exit, sometimes a thread holding the GIL can wait forever for a thread (usually a daemon thread) which requested to drop the GIL, whereas the thread already exited. To fix the race condition, the thread which requested the GIL drop now resets its request before exiting.

take_gil() now calls RESET_GIL_DROP_REQUEST() before PyThread_exit_thread() if it called SET_GIL_DROP_REQUEST to fix a race condition with drop_gil().

Issue discovered and analyzed by Mingliang ZHAO.

  • Issue: The main thread hang at drop_gil due to FORCE_SWITCHING after daemon thread exited #96387

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@vstinner vstinner added needs backport to 3.10 needs backport to 3.11 only security fixes labels Sep 16, 2022
@bedevere-bot bedevere-bot added the awaiting core review label Sep 16, 2022
@vstinner
gh-96387: take_gil() resets drop request before exit
c86088c 
At Python exit, sometimes a thread holding the GIL can wait forever
for a thread (usually a daemon thread) which requested to drop the
GIL, whereas the thread already exited. To fix the race condition,
the thread which requested the GIL drop now resets its request before
exiting.

take_gil() now calls RESET_GIL_DROP_REQUEST() before
PyThread_exit_thread() if it called SET_GIL_DROP_REQUEST to fix a
race condition with drop_gil().

Issue discovered and analyzed by Mingliang ZHAO.
@vstinner vstinner mentioned this pull request Sep 16, 2022
The main thread hang at drop_gil due to FORCE_SWITCHING after daemon thread exited #96387
Closed
@vstinner

vstinner commented Sep 16, 2022

Copy link
Copy Markdown
Member Author

@gpshead: Would you mind to review this change?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
vstinner
vstinner commented Sep 16, 2022
View reviewed changes
Comment thread Python/ceval_gil.c
// block forever waiting for the thread which exited. Drop
// requests made by other threads are also reset: these threads
// may have to request again a drop request (iterate one more
// time).

@vstinner vstinner Sep 16, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bugfix can slowdown the Python shutdown by a few "switch interval" seconds (5 ms by default). I expect that the worst case is when all daemon threads need one more iterations, so interval x number of threads: like 500 ms for 100 threads. But with luck, it's way shorter because the thread holding the GIL can get the switch request (and so let other threads exit) before the request is reset ;-)

My expectation is that... nobody will notice, since daemon threads are rare, and this race condition is unlikely ;-)

If tomorrow it becomes a problem, maybe the "GIL drop request" should store the tstate of the thread requesting it, and the request should only be reset if it matchs the current tstate. Not sure if it will speed up the shutdown. Again, I expect that it will not really affect performance at the end ;-)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@vstinner

vstinner commented Sep 16, 2022

Copy link
Copy Markdown
Member Author

@pitrou: If you're around, it would be nice if you could have a look, since you designed the new Pyhon GIL ;-)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
pitrou
pitrou approved these changes Sep 17, 2022
View reviewed changes
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Sep 17, 2022
@python python deleted a comment from Kizzer415 Sep 19, 2022
@python python deleted a comment from Kizzer415 Sep 19, 2022
@vstinner
vstinner merged commit 04f4977 into python:main Sep 19, 2022
@miss-islington

miss-islington commented Sep 19, 2022

Copy link
Copy Markdown
Contributor

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.10, 3.11.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the awaiting merge label Sep 19, 2022
@miss-islington

miss-islington commented Sep 19, 2022

Copy link
Copy Markdown
Contributor

Sorry @vstinner, I had trouble checking out the 3.11 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 04f4977f508583954ad7b9cb09076ee1e57461f8 3.11
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner
vstinner deleted the take_gil branch September 19, 2022 22:14
@miss-islington miss-islington assigned vstinner Sep 19, 2022
@miss-islington

miss-islington commented Sep 19, 2022

Copy link
Copy Markdown
Contributor

Sorry, @vstinner, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 04f4977f508583954ad7b9cb09076ee1e57461f8 3.10
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot

bedevere-bot commented Sep 19, 2022

Copy link
Copy Markdown

GH-96941 is a backport of this pull request to the 3.11 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Sep 19, 2022
@vstinner vstinner added needs backport to 3.11 only security fixes and removed needs backport to 3.10 labels Sep 19, 2022
@miss-islington

miss-islington commented Sep 19, 2022

Copy link
Copy Markdown
Contributor

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@miss-islington

miss-islington commented Sep 19, 2022

Copy link
Copy Markdown
Contributor

Sorry @vstinner, I had trouble checking out the 3.11 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 04f4977f508583954ad7b9cb09076ee1e57461f8 3.11
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner vstinner removed the needs backport to 3.11 only security fixes label Sep 19, 2022
vstinner added a commit that referenced this pull request Sep 20, 2022
@vstinner
gh-96387: take_gil() resets drop request before exit (#96869) (#96941)
6ff5471 
At Python exit, sometimes a thread holding the GIL can wait forever
for a thread (usually a daemon thread) which requested to drop the
GIL, whereas the thread already exited. To fix the race condition,
the thread which requested the GIL drop now resets its request before
exiting.

take_gil() now calls RESET_GIL_DROP_REQUEST() before
PyThread_exit_thread() if it called SET_GIL_DROP_REQUEST to fix a
race condition with drop_gil().

Issue discovered and analyzed by Mingliang ZHAO.

(cherry picked from commit 04f4977)
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Sep 20, 2022
@vstinner @miss-islington
pythongh-96387: take_gil() resets drop request before exit (pythonGH-…
bfb99c8 
…96869) (pythonGH-96941)

At Python exit, sometimes a thread holding the GIL can wait forever
for a thread (usually a daemon thread) which requested to drop the
GIL, whereas the thread already exited. To fix the race condition,
the thread which requested the GIL drop now resets its request before
exiting.

take_gil() now calls RESET_GIL_DROP_REQUEST() before
PyThread_exit_thread() if it called SET_GIL_DROP_REQUEST to fix a
race condition with drop_gil().

Issue discovered and analyzed by Mingliang ZHAO.

(cherry picked from commit 04f4977)
(cherry picked from commit 6ff5471)

Co-authored-by: Victor Stinner <[email protected]>
miss-islington added a commit that referenced this pull request Sep 20, 2022
@miss-islington @vstinner
gh-96387: take_gil() resets drop request before exit (GH-96869) (GH-9…
dcff50a 
…6941)

At Python exit, sometimes a thread holding the GIL can wait forever
for a thread (usually a daemon thread) which requested to drop the
GIL, whereas the thread already exited. To fix the race condition,
the thread which requested the GIL drop now resets its request before
exiting.

take_gil() now calls RESET_GIL_DROP_REQUEST() before
PyThread_exit_thread() if it called SET_GIL_DROP_REQUEST to fix a
race condition with drop_gil().

Issue discovered and analyzed by Mingliang ZHAO.

(cherry picked from commit 04f4977)
(cherry picked from commit 6ff5471)

Co-authored-by: Victor Stinner <[email protected]>
@gpshead gpshead added the needs backport to 3.11 only security fixes label Sep 20, 2022
@miss-islington

miss-islington commented Sep 20, 2022

Copy link
Copy Markdown
Contributor

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@miss-islington

miss-islington commented Sep 20, 2022

Copy link
Copy Markdown
Contributor

Sorry, @vstinner, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 04f4977f508583954ad7b9cb09076ee1e57461f8 3.11
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@gpshead gpshead mentioned this pull request Sep 20, 2022
gh-87135: Hang non-main threads that attempt to acquire the GIL during finalization #28525
Closed
@vstinner

vstinner commented Sep 30, 2022

Copy link
Copy Markdown
Member Author

The 3.11 backport is #96941
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@hauntsaninja hauntsaninja removed the needs backport to 3.11 only security fixes label Dec 24, 2022
@rebx

rebx commented Mar 10, 2024

Copy link
Copy Markdown

Hi, I wanted to check if this needs to be backported to 3.9 as well? Many thanks
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@itamaro

itamaro commented Mar 11, 2024

Copy link
Copy Markdown
Contributor

Hi, I wanted to check if this needs to be backported to 3.9 as well? Many thanks

3.9 is in security maintenance mode, and this doesn't look like a security-related patch, so probably no.
vstinner reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pitrou pitrou pitrou approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@vstinner vstinner

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

8 participants

@vstinner @miss-islington @bedevere-bot @rebx @itamaro @pitrou @gpshead @hauntsaninja
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.