GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — [3.12] gh-109860: Use a New Thread State When Switching Interpreters, When Necessary (gh-110245) by ericsnowcurrently · Pull Request #110709 · python/cpython

Conversation

@ericsnowcurrently

@ericsnowcurrently ericsnowcurrently commented Oct 11, 2023 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

In a few places we switch to another interpreter without knowing if it has a thread state associated with the current thread. For the main interpreter there wasn't much of a problem, but for subinterpreters we were mostly okay re-using the tstate created with the interpreter (located via PyInterpreterState_ThreadHead()). There was a good chance that tstate wasn't actually in use by another thread.

However, there are no guarantees of that. Furthermore, re-using an already used tstate is currently fragile. To address this, now we create a new thread state in each of those places and use it.

One consequence of this change is that PyInterpreterState_ThreadHead() may not return NULL (though that won't happen for the main interpreter).

(cherry-picked from commit f5198b0)

  • Issue: PyThreadState Sometimes Used for Different Threads for Subinterpreters  #109860

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@ericsnowcurrently
[3.12] pythongh-105716: Support Background Threads in Subinterpreters…
e19af2c
… Consistently (pythongh-109921)

The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish.

We add PyInterpreterState.threads.main, with some internal C-API functions.

(cherry-picked from commit 1dd9dee)
@bedevere-app bedevere-app Bot mentioned this pull request Oct 11, 2023
gh-109860: Use a New Thread State When Switching Interpreters, When Necessary #110245
Merged
@bedevere-app bedevere-app Bot added the skip news label Oct 11, 2023
@bedevere-app bedevere-app Bot mentioned this pull request Oct 11, 2023
PyThreadState Sometimes Used for Different Threads for Subinterpreters #109860
Closed
ericsnowcurrently added 8 commits October 12, 2023 01:41
@ericsnowcurrently
pythongh-76785: Use Pending Calls When Releasing Cross-Interpreter Da…
b1e4a24
…ta (pythongh-109556)

This fixes some crashes in the _xxinterpchannels module, due to a race between interpreters.
(cherry picked from commit fd7e08a)

Co-authored-by: Eric Snow <[email protected]>
@ericsnowcurrently
Move _PyCrossInterpreterData_ReleaseAndRawFree to internal API.
aeaecee
@ericsnowcurrently
Move _PyCrossInterpreterData_ReleaseAndRawFree to internal API.
1940c78
@ericsnowcurrently
Set Py_BUILD_CORE_MODULE for _xxinterpchannelsmodule.
49f99ca
@ericsnowcurrently
[3.12] pythongh-109853: Fix sys.path[0] For Subinterpreters (pythongh…
f945ed2
…-109994)

This change makes sure sys.path[0] is set properly for subinterpreters.  Before, it wasn't getting set at all.

This change does not address the broader concerns from pythongh-109853.

(cherry-picked from commit a040a32)
@ericsnowcurrently
Do not modify PyConfig. Add sys_path_0 to _PyRuntimeState instead.
ea57f0c
@ericsnowcurrently
[3.12] pythongh-109860: Use a New Thread State When Switching Interpr…
9a3588e
…eters, When Necessary (pythongh-110245)

In a few places we switch to another interpreter without knowing if it has a thread state associated with the current thread.  For the main interpreter there wasn't much of a problem, but for subinterpreters we were *mostly* okay re-using the tstate created with the interpreter (located via PyInterpreterState_ThreadHead()).  There was a good chance that tstate wasn't actually in use by another thread.

However, there are no guarantees of that.  Furthermore, re-using an already used tstate is currently fragile.  To address this, now we create a new thread state in each of those places and use it.

One consequence of this change is that PyInterpreterState_ThreadHead() may not return NULL (though that won't happen for the main interpreter).

(cherry-picked from commit f5198b0)
@ericsnowcurrently
Drop PyThreadState.whence.
4fb7f0f
@ericsnowcurrently
ericsnowcurrently force-pushed the backport-f5198b0-fix-using-wrong-tstate branch from c40f162 to 4fb7f0f Compare October 12, 2023 07:46
@ericsnowcurrently ericsnowcurrently added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 12, 2023
@bedevere-bot

bedevere-bot commented Oct 12, 2023

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @ericsnowcurrently for commit 4fb7f0f 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 12, 2023
@ericsnowcurrently ericsnowcurrently mentioned this pull request Nov 28, 2023
[3.12] gh-105699: Re-enable the Multiple-Interpreters Stress Tests (gh-107572) #112483
Closed
@hugovk

hugovk commented Aug 9, 2024

Copy link
Copy Markdown
Member

Triage: is this backport still needed?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk hugovk added the pending The issue will be closed if no feedback is provided label Feb 17, 2025
@ambv

ambv commented Apr 8, 2025

Copy link
Copy Markdown
Contributor

3.12 is entering security-only fixes mode today.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ambv ambv closed this Apr 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

pending The issue will be closed if no feedback is provided skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@ericsnowcurrently @bedevere-bot @hugovk @ambv
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.