GeezerVison is showing github.com — Fail PyGILState_Ensure() If Finalizing? · Issue #124622 · python/cpython
Fail PyGILState_Ensure() If Finalizing? #124622DescriptionFeature or enhancement
Proposal:
Once the runtime or current interpreter starts finalizing, we currently exit or stall the current thread as soon as it tries to acquire the GIL, via PyThread_exit_thread() (or PyThread_hang_thread() after gh-105805). That includes in PyGILState_Ensure(). However, in the case of the latter it might make more sense (i.e. friendlier) for the call to fail instead of exiting/hanging the current thread.
FWIW, this is a hypothetical situation I noticed while looking at pystate.c. Also, the same scenario would be affected by gh-124619.
CC @gpshead
Has this already been discussed elsewhere?
No response given
Links to previous discussion of this feature:
No response
Linked PRs
Reacted by Gregory P. Smith Activitygpshead commented on Sep 26, 2024 A challenge in doing this is that PyGILState_Ensure returns an enum PyGILState_STATE value (defined in Include/pystate.h). That only contains LOCKED and UNLOCKED states today, so callers of the function in existing compiled Python C API code will never expect other values. We'd need to know if they were compiled against a recent enough Python version in order to return a new enum value representing an error because the interpreter is being finalized thus no GIL acquisition could possibly succeed. gpshead commented on Sep 26, 2024 I think a new GIL acquisition and release C API would be needed. The way the existing ones get used in existing C code is not amenible to suddenly bolting an error state onto; none of the existing C code is written that way. After the call they always just assume they have the GIL and can proceed. The API was designed as "it'll block and only return once it has the GIL" without any other option. ZeroIntensity commented on Sep 27, 2024 This is something that seems fatal-error worthy rather than a new PyGILState_STATE value -- there's nothing much a dev can do if they want to call the C API during a finalizing interpreter.
I think a new GIL acquisition and release C API would be needed.
That's been loosely discussed recently (in the context of subinterpreters, rather than the GIL), see #123728 (comment) Reacted by Gregory P. Smith rruuaanng commented on Oct 31, 2024 If you don't mind, I will sort out the requirements under discussion and publish some issue in the near future. gpshead commented on Feb 2, 2025 Reacted by Erlend E. Aasland ZeroIntensity commented on Feb 3, 2025 I don't think it's a good idea to add new things with the term "GIL" in them. I'd prefer a totally new API design (e.g., PyThreadState_Ensure) that takes an explicit interpreter rather than an implicit one. ZeroIntensity commented on Feb 4, 2025 Ok, I've been playing around with implementing this. I think we need to very clearly define two things:
- Where should Python stop allowing thread states to be created? When the interpreter starts finalizing? When the runtime starts finalizing? But, there does seem to be some desire in the discussion to use the C API in finalizers, so maybe somewhere later?
- If we go with the latter, then what should happen to threads that are created during finalization? Currently,
Py_Finalize assumes that there's only one thread left, so there's no mechanism in place for getting rid of new thread states. (I do have an open PR that changes this for subinterpreters, but not the main interpreter yet.)
vstinner commented on Feb 5, 2025 I created #129688 to add PyGILState_EnsureOrFail() function. Reacted by RUANG (James Roy) and Dustin Spicuzza vstinner commented on Feb 14, 2025 See also comments on a previous similar attempt: #123728
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|