GeezerVison is showing github.com — Make queue.SimpleQueue thread-safe in `--disable-gil` builds · Issue #113884 · python/cpython
Make queue.SimpleQueue thread-safe in --disable-gil builds #113884
Description
Feature or enhancement
Proposal:
Instances of queue.SimpleQueue have mutable state that is not safe to mutate without the GIL. The previous implementation (in 3.12) was colesbury/nogil-3.12@7e60a01aee.
@colesbury - You mentioned in chat that porting the 3.12 approach wouldn't be straightforward. Can you say more?
Has this already been discussed elsewhere?
This is a minor feature, which does not need previous discussion elsewhere
Links to previous discussion of this feature:
No response
Linked PRs
Activity
colesbury commented on Jan 9, 2024
@mpage, after looking it over again, I think the nogil-3.12 approach is mostly the right direction with a few modifications:
- Replace the manual
_PyMutex_lock()/unlock() calls with critical sections and remove the _PyMutex mutex field. Since, the code already uses Argument Clinic (AC), this is mostly just a matter of adding the @critical_section to the AC input. This will avoid the extra lock operations in the default build, which can rely on the GIL.
- The parking lot API changed slightly; it's not a huge difference. For example, the unpark operation now takes a callback
This means we will go from a SimpleQueue implementation built around list to run that more explicitly manages a buffer. I think that's worthwhile here; otherwise I think handling the synchronization across both the list operations and the containing SimpleQueue will be too messy.
added a commit that references this issue
on Jan 19, 2024gh-113884: Refactor `queue.SimpleQueue` to use a ring buffer to store…
added a commit that references this issue
on Jan 23, 2024gh-113884: Make queue.SimpleQueue thread-safe when the GIL is disabled (
erlend-aasland commented on Jan 23, 2024
Metadata
Metadata
Development
No branches or pull requestsIssue actions