GeezerVison is showing github.com — gh-134634: Use a pipe instead of a queue for Pool's change notifier by yonatan-genai · Pull Request #148050 · python/cpython
Conversation
yonatan-genai
commented
Apr 3, 2026
•
edited by bedevere-app
Bot
Loading
Uh oh!
There was an error while loading. Please reload this page.
Summary
Pool.__init__ used a multiprocessing.SimpleQueue for its internal _change_notifier, a mechanism that wakes up the worker handler thread when the pool state changes (close, terminate, cache empty). SimpleQueue creates two Lock objects backed by POSIX named semaphores via sem_open(), which requires /dev/shm to be mounted as tmpfs.
The change notifier never crosses a process boundary. It is only accessed by threads within the parent process (the worker handler, result handler, and main thread). Named semaphores were never needed here.
This replaces the SimpleQueue with a _ChangeNotifier class backed by multiprocessing.connection.Pipe (which wraps os.pipe()). It provides the same interface (put, get, empty, _reader) without requiring sem_open().
Impact
Both multiprocessing.Pool and multiprocessing.pool.ThreadPool fail on platforms without /dev/shm, even though ThreadPool only uses threads and never needs inter-process synchronization. Affected platforms include AWS Lambda (no /dev/shm), Android and iOS (sem_open() unavailable), and Docker containers with restricted shared memory.
Changes
Lib/multiprocessing/pool.py: Add _ChangeNotifier class, replace the one line in Pool.__init__ that created the SimpleQueue
Lib/test/_test_multiprocessing.py: Three new tests — notifier type verification + end-to-end pool.map, notification drain under load, worker process recovery after SIGKILL
Misc/NEWS.d/next/Library/: NEWS entry
Testing
Full test_multiprocessing_spawn suite: 447 tests passed, 0 failures. Reference leak check passed.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Pool.__init__ used a multiprocessing.SimpleQueue for its internal
_change_notifier. SimpleQueue creates Lock objects backed by POSIX
named semaphores (sem_open), which requires /dev/shm. The change
notifier never crosses a process boundary, so named semaphores were
never needed.
Replace it with a _ChangeNotifier class backed by
multiprocessing.connection.Pipe (os.pipe). Same interface, no
sem_open dependency.
<claude>
|
All commit authors signed the Contributor License Agreement.

|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
The new tests need @warnings_helper.ignore_fork_in_thread_deprecation_warnings()
to avoid DeprecationWarning when creating Pool with the fork start method
in a multi-threaded test process (Python 3.14+ strictness).
<claude>
|
The macOS / build and test (macos-26) failure is unrelated to this change — it's test_perf_profiler.TestPerfTrampoline.test_trampoline_works_with_forks failing with a non-zero child exit code. This test has a known flakiness issue related to fork child finalization (see gh-144766). I'll submit a separate fix for it.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
My username has been this way for several years… before Claude Code,
OpenClaw, etc. I signed the CLA.
…
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
I’d appreciate it if you could consider reopening…
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Reviewers
gpshead
Awaiting requested review from gpshead
gpshead is a code owner
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|