GeezerVison is showing github.com — Implement stop-the-world functionality (for `--disable-gil` builds) · Issue #111964 · python/cpython
Implement stop-the-world functionality (for --disable-gil builds) #111964DescriptionFeature or enhancement
The --disable-gil builds occasionally need to pause all but one thread. Some examples include:
- Cyclic garbage collection, where this is often called a "stop the world event"
- Before calling
fork(), to ensure a consistent state for internal data structures
- During interpreter shutdown, to ensure that daemon threads aren't accessing Python objects
In the nogil-3.12 fork, a stop-the-world call paused all threads in all interpreters. In CPython 3.13, we probably want to provide two levels of "stop-the-world": per-interpreter and global. In general, per-interpreter pauses are preferable to global pauses, but there are some cases (like before fork()) where want all threads to pause, so that we don't fork() while a thread is modifying some global runtime structure.
In the default build, stop-the-world calls should generally be no-ops, but global pauses may be useful in a few cases, such as before fork() when using multiple interpreters each with their own "GIL".
In the nogil-3.12 fork, this was implemented together with other GC modifications: colesbury/nogil-3.12@2864b6b36e. I'd like to implement it separately in 3.13 to keep the PRs smaller.
Linked PRs
Reacted by Donghee Na Reacted by Donghee Na ActivityHels15 commented on Nov 21, 2023 colesbury commented on Nov 21, 2023 ContributorAuthor More actions colesbury commented on Jan 3, 2024 ContributorAuthor More actions @markshannon: Carl mentioned you had a question about the stop-the-world implementation and reentrancy. The current implementation in the linked PR does not support reentrancy. Prior implementations in the nogil fork supported reentrancy and behaved like recursive mutexes: only the "outer most" calls stopped or resumed threads; "inner" calls only increment/decrement a counter.
The only place reentrancy was useful was during runtime finalization (i.e., interpreter shutdown). The stop-the-world mechanism is useful for forcing daemon threads 1 to a consistent state right before we delete their thread states, but we also trigger the GC later during shutdown. Making the stop-the-world calls reentrant was a convenient "hack", but I think there are other ways to achieve the same effect.
I don't think reentrant stop-the-world calls are generally useful because you are quite restricted on what's safe to call during a stop-the-world pause or you risk deadlock. Other paused threads may be holding the locks you need, so you can't call arbitrary python code or destructors. The GC is (or will be) careful to only call finalizers and weakref callbacks after resuming other threads.
Footnotes
-
and potentially non-daemon threads depending on when you press ctrl-c ↩
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|