GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — [3.13] gh-87135: Hang non-main threads that attempt to acquire the GIL during finalization (GH-105805) by gpshead · Pull Request #137827 · python/cpython

Conversation

gpshead commented Aug 15, 2025 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Member

Instead of surprise crashes and memory corruption, we now hang threads that attempt to re-enter the Python interpreter after Python runtime finalization has started. These are typically daemon threads (our long standing mis-feature) but could also be threads spawned by extension modules that then try to call into Python. This backport documents that the PyThread_exit_thread public C API should not be used (deprecated in 3.14) as there is no plausible safe way to accomplish that on any supported platform in the face of things like C++ code with finalizers anywhere on a thread's stack. Doing this was the least bad option.

(cherry picked from commit 8cc5aa4)

Issue: pthread_exit & PyThread_exit_thread from PyEval_RestoreThread etc. are harmful #87135

📚 Documentation preview 📚: https://cpython-previews--137827.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions


          [3.13] pythongh-87135: Hang non-main threads that attempt to acquire …

f3e301d

…the GIL during finalization (pythonGH-105805)

Instead of surprise crashes and memory corruption, we now hang threads that attempt to re-enter the Python interpreter after Python runtime finalization has started. These are typically daemon threads (our long standing mis-feature) but could also be threads spawned by extension modules that then try to call into Python. This marks the `PyThread_exit_thread` public C API as deprecated as there is no plausible safe way to accomplish that on any supported platform in the face of things like C++ code with finalizers anywhere on a thread's stack. Doing this was the least bad option.

(cherry picked from commit 8cc5aa4)

Co-authored-by: Jeremy Maitin-Shepard <[email protected]>
Co-authored-by: Gregory P. Smith <[email protected]>

This was referenced Aug 15, 2025

pthread_exit & PyThread_exit_thread from PyEval_RestoreThread etc. are harmful #87135

Closed

gh-87135: Hang non-main threads that attempt to acquire the GIL during finalization #105805

Merged


          state "3.13.7 and earlier"

d946b3d

gpshead force-pushed the backport-8cc5aa4-3.13 branch from 4f77306 to 977b0d4 Compare

August 15, 2025 16:28

gpshead requested review from Yhg1s and serhiy-storchaka

August 15, 2025 16:30

gpshead marked this pull request as ready for review

August 15, 2025 16:30

gpshead requested a review from ericsnowcurrently as a code owner

August 15, 2025 16:30

bedevere-app Bot added the awaiting core review label

Aug 15, 2025

gpshead added type-bug

An unexpected behavior, bug, or error

interpreter-core

(Objects, Python, Grammar, and Parser dirs)

3.13 only security fixes labels

Aug 15, 2025

gpshead self-assigned this

Aug 15, 2025


          backport: do not add the deprecated marker

f412737

gpshead force-pushed the backport-8cc5aa4-3.13 branch from 977b0d4 to f412737 Compare

August 15, 2025 16:34

gpshead added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label

Aug 15, 2025

bedevere-bot commented Aug 15, 2025

Copy link

Copy Markdown

🤖 New build scheduled with the buildbot fleet by @gpshead for commit f412737 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F137827%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label

Aug 15, 2025

serhiy-storchaka approved these changes

Aug 15, 2025

View reviewed changes

serhiy-storchaka left a comment

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot say it LGTM, but it looks less bad than anything else. ☹️ It seems there are no good solutions here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

bedevere-app Bot added awaiting merge and removed awaiting core review labels

Aug 15, 2025


          fix Py_IsFinalizing doc ref

d24d14e

gpshead merged commit 9face21 into python:3.13

Aug 17, 2025

bedevere-app Bot removed the awaiting merge label

Aug 17, 2025

gpshead commented Aug 29, 2025

Copy link

Copy Markdown

Member Author

I'll leave this PR around for reference, but do not intend to merge it so late in the 3.13 release cycle even though I believe this least bad option is the only viable one. It is a behavior change, even if the existing behavior is an undertiministic sometimes crashing or sometimes working. See #87135 (comment) and #123940 (comment) for rationale from @encukou about invasiveness of such a change as in a bugfix release.

encukou reacted with heart emoji

All reactions

❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

PeaBrane mentioned this pull request

Oct 7, 2026

[BUG]: Interpreter exit aborts with "FATAL: exception not rethrown" when a thread re-takes the GIL in a PyO3 binding (PyO3 0.23, Python < 3.13.8) ai-dynamo/dynamo#15800

Open

shoumikhin mentioned this pull request

Nov 2, 2026

Python runtime: stop a backend's log from deadlocking 2 threads pytorch/executorch#24393

Merged

Reviewers

serhiy-storchaka

serhiy-storchaka approved these changes

Yhg1s

Awaiting requested review from Yhg1s

ericsnowcurrently

Awaiting requested review from ericsnowcurrently

ericsnowcurrently is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

gpshead

Labels

3.13

only security fixes

interpreter-core

(Objects, Python, Grammar, and Parser dirs)

type-bug

An unexpected behavior, bug, or error

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

gpshead commented Aug 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-bot commented Aug 15, 2025

Uh oh!

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Uh oh!

gpshead commented Aug 29, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gpshead commented Aug 15, 2025 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.