GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — PyThreadState Sometimes Used for Different Threads for Subinterpreters · Issue #109860 · python/cpython

PyThreadState Sometimes Used for Different Threads for Subinterpreters  #109860

New issue
Copy link
New issue
Copy link
Closed
Closed
PyThreadState Sometimes Used for Different Threads for Subinterpreters #109860
Copy link
Assignees
ericsnowcurrently
Labels
3.12only security fixesonly security fixes3.13only security fixesonly security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)topic-subinterpreterstype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Description

@ericsnowcurrently
ericsnowcurrently
opened on Sep 25, 2023
Issue body actions

Bug report

In a few places, we treat PyInterpreterState.threads.head (AKA PyInterpreterState_ThreadHead()) for subinterpreters as an unused thread state that we can use in a one-off manner:

  • _PyInterpreterState_IDDecref() (Python/pystate.c)
  • _call_in_interpreter() (Python/pystate.c) - fixed in 3.13+ (gh-76785: Use Pending Calls When Releasing Cross-Interpreter Data #109556)
  • _run_script_in_interpreter() (Modules/_xxsubinterpreter.c)
  • interp_destroy() (Modules/_xxsubinterpreter.c)

The problem is that each thread state should correspond to a single OS thread, and for each OS thread there should be at most one thread state per interpreter. Using PyInterpreterState.threads.head like that violates those assumptions, which can cause problems. 1

Also, some code assumes PyInterpreterState.threads.head is the current thread state, when it might not be:

  • interpreter_clear() (Python/pystate.c)
  • ...

We should fix this by always using the thread state that corresponds to the current thread (and create one if missing). For efficiency, we can use a thread-local (via PyThread_tss_*) to store each interpreter's thread state for that thread.

Linked PRs

  • gh-109860: Use a New Thread State When Switching Interpreters, When Necessary #110245
  • [3.12] gh-109860: Use a New Thread State When Switching Interpreters, When Necessary (gh-110245) #110709

Footnotes

  1. Currently, this mostly doesn't affect the main interpreter. Ideally, the main interpreter wouldn't actually be treated specially. See (Mostly) Stop Special-casing the Main Interpreter #109857. ↩

Reacted by Michael Droettboom and Erlend E. Aasland

Activity

  1. ericsnowcurrently
    added
    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)
    topic-subinterpreters
    3.12only security fixesonly security fixes
    3.13only security fixesonly security fixes
    on Sep 25, 2023
  2. ericsnowcurrently
    self-assigned this
    on Sep 25, 2023
  3. github-project-automation
    added this to Subinterpreterson Sep 25, 2023
  4. github-project-automation
    moved this to Todo in Subinterpreterson Sep 25, 2023
  5. ericsnowcurrently
    added a commit that references this issue on Oct 3, 2023

    gh-109860: Use a New Thread State When Switching Interpreters, When N…

    f5198b0
  6. ericsnowcurrently
    closed this as completedon Oct 3, 2023
  7. github-project-automation
    moved this from Todo to Done in Subinterpreterson Oct 3, 2023
  8. ericsnowcurrently
    added 2 commits that reference this issue on Oct 11, 2023

    [3.12] pythongh-109860: Use a New Thread State When Switching Interpr…

    0ec39ff

    [3.12] pythongh-109860: Use a New Thread State When Switching Interpr…

    9a3588e
  9. Glyphack
    added a commit that references this issue on Sep 2, 2024

    pythongh-109860: Use a New Thread State When Switching Interpreters, …

    bf6f0e3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

  • @ericsnowcurrently
    ericsnowcurrently

Labels

3.12only security fixesonly security fixes3.13only security fixesonly security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)topic-subinterpreterstype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Projects

  • Subinterpreters
    Status
    Done
    Show more project fields

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.