GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-39877: take_gil() now exits the thread if finalizing by vstinner · Pull Request #18854 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented Mar 8, 2020 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

take_gil() is now responsible to exit the thread if Python is
finalizing. Not only exit before trying to acquire the GIL, but exit
also once the GIL is succesfully acquired.

https://bugs.python.org/issue39877

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@vstinner
bpo-39877: take_gil() now exits the thread if finalizing
9ac0b25 
take_gil() is now responsible to exit the thread if Python is
finalizing. Not only exit before trying to acquire the GIL, but exit
also once the GIL is succesfully acquired.
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Mar 8, 2020
@bedevere-bot bedevere-bot added the awaiting core review label Mar 8, 2020
@vstinner

vstinner commented Mar 8, 2020

Copy link
Copy Markdown
Member Author

@pitrou: My previous change introduced a race condition: https://bugs.python.org/issue39877#msg363667 This change should fix it.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner

vstinner commented Mar 8, 2020 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

I tested manually: with this change and PR #18848, asyncio_gc.py of bpo-19466 does no longer crash.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
pitrou
pitrou requested changes Mar 8, 2020
View reviewed changes

@pitrou pitrou left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the fix is somehow incomplete, see below.
Also, as you say, the problem is "is it safe to destroy the GIL if there are still daemon threads trying to take it?".

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Python/ceval_gil.h Outdated

MUTEX_UNLOCK(gil->mutex);

if (thread_must_exit(tstate)) {

@pitrou pitrou Mar 8, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why you're doing this here. It doesn't seem necessary and, furthermore, it's unlikely to trigger.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@vstinner vstinner Mar 8, 2020

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code path is to prevent https://bugs.python.org/issue39877#msg363667 crash. Are you suggesting to move this code at line 249, after COND_TIMED_WAIT()?

thread_must_exit() at entry is to prevent https://bugs.python.org/issue39877#msg363512 crash.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Python/ceval_gil.h
{
int err = errno;

if (thread_must_exit(tstate)) {

@pitrou pitrou Mar 8, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, but shouldn't you do this after COND_TIMED_WAIT below as well?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-bot bedevere-bot added awaiting changes and removed awaiting core review labels Mar 8, 2020
@bedevere-bot

bedevere-bot commented Mar 8, 2020

Copy link
Copy Markdown

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner
take_gil() exit after COND_TIMED_WAIT()
4f379aa
vstinner
vstinner commented Mar 8, 2020
View reviewed changes
Comment thread Python/ceval_gil.h
to run after wait_for_thread_shutdown() and before Py_Finalize()
completes. For example, when _PyImport_Cleanup() executes Python
code. */
MUTEX_UNLOCK(gil->mutex);

@vstinner vstinner Mar 8, 2020

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if anyone else should be done here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@vstinner vstinner Mar 9, 2020

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example, is COND_SIGNAL(gil->cond) needed to wake up the next thread waiting on COND_TIMED_WAIT()?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@vstinner

vstinner commented Mar 9, 2020

Copy link
Copy Markdown
Member Author

I have made the requested changes; please review again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot added awaiting change review and removed awaiting changes labels Mar 9, 2020
@bedevere-bot

bedevere-bot commented Mar 9, 2020

Copy link
Copy Markdown

Thanks for making the requested changes!

@pitrou: please review the changes made to this pull request.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot
bedevere-bot requested a review from pitrou March 9, 2020 07:44
@vstinner vstinner mentioned this pull request Mar 9, 2020
bpo-19466: Py_Finalize() clears daemon threads earlier #18848
Merged
@vstinner

vstinner commented Mar 9, 2020

Copy link
Copy Markdown
Member Author

I merged PR #18890 which checks if the thread must exit at function exit as well. In short, it restores the Python 3.8 behavior.

I close this PR for now.

We can revisit take_gil() later to try to adjust/optimize it later. I prefer to move step by step.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner vstinner closed this Mar 9, 2020
@vstinner
vstinner deleted the take_gil_exit branch March 9, 2020 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pitrou pitrou Awaiting requested review from pitrou

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting change review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@vstinner @bedevere-bot @pitrou @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.