GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-46070: Fix asyncio initialisation guard by erlend-aasland · Pull Request #30423 · python/cpython

Conversation

@erlend-aasland

@erlend-aasland erlend-aasland commented Jan 5, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

If init flag is set, exit module init successfully immediately.
If not, only set the flag after successful module initialisation.

https://bugs.python.org/issue46070

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Erlend E. Aasland
bpo-46070: Fix asyncio initialisation guard
7783785
If init flag is set, exit successfully immediately.
If not, only set the flag after successful initialisation.
@erlend-aasland
erlend-aasland requested review from 1st1 and asvetlov as code owners January 5, 2022 19:29
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Jan 5, 2022
@bedevere-bot bedevere-bot added the awaiting review label Jan 5, 2022
@erlend-aasland
erlend-aasland requested a review from vstinner January 5, 2022 19:29
@erlend-aasland erlend-aasland added needs backport to 3.9 labels Jan 5, 2022
@erlend-aasland

erlend-aasland commented Jan 5, 2022

Copy link
Copy Markdown
Contributor Author

Maybe we should run this through the buildbots.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland

erlend-aasland commented Jan 5, 2022

Copy link
Copy Markdown
Contributor Author

This might qualify for a NEWS entry, as it fixes segfaults on 3.10 and 3.9 (and 3.8, but 3.8 is in security-fix-mode-only).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jan 7, 2022

Copy link
Copy Markdown
Member

module_init() is called in parallel multiple times from different interpreters (in different threads). I understand that module_init() leaks references when calling multiple times in parallel, since it doesn't use Py_SETREF(). For now, I suggest to not worry about that. I suggest to rather invest time on converting _asyncio static types to heap types and then port _asyncio to multi-phase init: move static globals into a module state.

erlend-aasland reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

vstinner
vstinner approved these changes Jan 7, 2022
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

erlend-aasland reacted with hooray emoji
All reactions
  • 🎉 1 reaction
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting review labels Jan 7, 2022
@erlend-aasland

erlend-aasland commented Jan 7, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

module_init() is called in parallel multiple times from different interpreters (in different threads). I understand that module_init() leaks references when calling multiple times in parallel, since it doesn't use Py_SETREF(). For now, I suggest to not worry about that. I suggest to rather invest time on converting _asyncio static types to heap types and then port _asyncio to multi-phase init: move static globals into a module state.

Yes, there are ref leaks, but as you say they are best fixed by converting global state to module state and implement proper multi-phase init. I've got a private branch for this work. This demands a separate bpo issue, and we should involve the asyncio maintaner before continuing with that effort :)

Thanks for reviewing!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jan 7, 2022

Copy link
Copy Markdown
Member

This might qualify for a NEWS entry, as it fixes segfaults on 3.10 and 3.9 (and 3.8, but 3.8 is in security-fix-mode-only).

Oh, I didn't notice the failing "news" job. Yes, please document the fix. If I understand correctly, this change fix a crash an importing the asyncio module from different sub-interpreters in parallel. And the workaround was to import asyncio in the main interpreter before spawning sub-interpreters.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland

erlend-aasland commented Jan 7, 2022

Copy link
Copy Markdown
Contributor Author

[...] Yes, please document the fix. If I understand correctly, this change fix a crash an importing the asyncio module from different sub-interpreters in parallel. And the workaround was to import asyncio in the main interpreter before spawning sub-interpreters.

Sure, I'll try to write something sensible :) Also, we could probably use a more compact version of your reproducer as a regression test.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Erlend E. Aasland
Add NEWS
c47069d
@vstinner

vstinner commented Jan 7, 2022

Copy link
Copy Markdown
Member

Adding a test is optional for me. It's up to me. But I would like to see a NEWS entry please.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland

erlend-aasland commented Jan 7, 2022

Copy link
Copy Markdown
Contributor Author

Adding a test is optional for me. It's up to me. But I would like to see a NEWS entry please.

You got it 🥁

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

vstinner
vstinner approved these changes Jan 7, 2022
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

erlend-aasland reacted with hooray emoji
All reactions
  • 🎉 1 reaction
@vstinner
vstinner merged commit b127e70 into python:main Jan 7, 2022
@miss-islington

miss-islington commented Jan 7, 2022

Copy link
Copy Markdown
Contributor

Thanks @erlend-aasland for the PR, and @vstinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9, 3.10.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting merge label Jan 7, 2022
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jan 7, 2022
Erlend Egeberg Aasland @miss-islington
bpo-46070: Fix asyncio initialisation guard (pythonGH-30423)
7057c07
If init flag is set, exit successfully immediately.
If not, only set the flag after successful initialization.
(cherry picked from commit b127e70)

Co-authored-by: Erlend Egeberg Aasland <[email protected]>
@bedevere-bot bedevere-bot removed the needs backport to 3.10 label Jan 7, 2022
@bedevere-bot

bedevere-bot commented Jan 7, 2022

Copy link
Copy Markdown

GH-30453 is a backport of this pull request to the 3.10 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jan 7, 2022
Erlend Egeberg Aasland @miss-islington
bpo-46070: Fix asyncio initialisation guard (pythonGH-30423)
cacc876
If init flag is set, exit successfully immediately.
If not, only set the flag after successful initialization.
(cherry picked from commit b127e70)

Co-authored-by: Erlend Egeberg Aasland <[email protected]>
@bedevere-bot

bedevere-bot commented Jan 7, 2022

Copy link
Copy Markdown

GH-30454 is a backport of this pull request to the 3.9 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.9 label Jan 7, 2022
miss-islington added a commit that referenced this pull request Jan 7, 2022
@miss-islington Erlend Egeberg Aasland
bpo-46070: Fix asyncio initialisation guard (GH-30423)
9d18045
If init flag is set, exit successfully immediately.
If not, only set the flag after successful initialization.
(cherry picked from commit b127e70)

Co-authored-by: Erlend Egeberg Aasland <[email protected]>
miss-islington added a commit that referenced this pull request Jan 7, 2022
@miss-islington Erlend Egeberg Aasland
bpo-46070: Fix asyncio initialisation guard (GH-30423)
4d2cfd3
If init flag is set, exit successfully immediately.
If not, only set the flag after successful initialization.
(cherry picked from commit b127e70)

Co-authored-by: Erlend Egeberg Aasland <[email protected]>
@erlend-aasland
erlend-aasland deleted the asyncio-init-guard branch January 7, 2022 16:09
@prahal prahal mentioned this pull request Mar 22, 2022
[BUG] Kodi crashing when switching from an add-on to another LibreELEC/LibreELEC.tv#6266
Closed
@graysky graysky mannequin mentioned this pull request Apr 10, 2022
[subinterpreters] crash when importing _sre in subinterpreters in parallel (Python 3.9 regression) #90228
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner approved these changes
@1st1 1st1 Awaiting requested review from 1st1 1st1 is a code owner
@asvetlov asvetlov Awaiting requested review from asvetlov asvetlov is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@erlend-aasland @vstinner @miss-islington @bedevere-bot @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.