GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-83069: Use efficient event-driven `subprocess.Popen.wait()` on Linux / macOS / BSD by giampaolo · Pull Request #144047 · python/cpython

gh-83069: Use efficient event-driven subprocess.Popen.wait() on Linux / macOS / BSD #144047

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vstinner merged 68 commits into python:main from giampaolo:subprocess-fast-wait
Jan 28, 2026

Uh oh!

There was an error while loading. Please reload this page.

Merged

gh-83069: Use efficient event-driven subprocess.Popen.wait() on Linux / macOS / BSD #144047

Changes from 1 commit
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
288e4db
Use pidfd_open() and save the FD as an attribute
giampaolo Jan 18, 2026
6e494ca
Create _busy_wait() and move code in there
giampaolo Jan 18, 2026
aaf193c
Move existing code in new _blocking_wait() method
giampaolo Jan 18, 2026
7cbb0ad
Use _wait_pidfd()
giampaolo Jan 18, 2026
d9760c3
Use pidfd_open() not at class level but at method level
giampaolo Jan 18, 2026
5fd8eec
Add kqueue() implementation for macOS and BSD
giampaolo Jan 18, 2026
100b111
Add docstrings
giampaolo Jan 18, 2026
74ac2f4
Be conservative and check for specific error codes
giampaolo Jan 18, 2026
fc0cfd6
Document possible failures
giampaolo Jan 18, 2026
f282459
Add missing import
giampaolo Jan 18, 2026
6125976
Write test for pidfd_open() failing
giampaolo Jan 18, 2026
41dc6c0
Write test case for pidfd_open() / kqueue failing. Assert fallback is…
giampaolo Jan 18, 2026
73ba380
Write test case for kqueue failing. Assert fallback is used.
giampaolo Jan 18, 2026
3c8e603
Move tests in their own class
giampaolo Jan 18, 2026
932ae58
Add test for terminated PID race
giampaolo Jan 18, 2026
bb5080a
Add test_kqueue_race()
giampaolo Jan 19, 2026
f067073
Add test_kqueue_control_error
giampaolo Jan 19, 2026
4d96c2f
Add docstring
giampaolo Jan 19, 2026
fe05acc
Guard against possible slow test
giampaolo Jan 19, 2026
adb444e
Timeout: use math.ceil to avoid truncation
giampaolo Jan 19, 2026
4ec17c1
Remove unused exception var
giampaolo Jan 19, 2026
e807ba9
Timeout: use math.ceil to avoid truncation
giampaolo Jan 19, 2026
1ddc52b
Replace _can_use_kqueue() -> _CAN_USE_KQUEUE
giampaolo Jan 19, 2026
645ef6c
Shorten code
giampaolo Jan 19, 2026
6ee771b
Use waitpid() + WNOHANG even if process exited to avoid rare race
giampaolo Jan 19, 2026
61c6b99
Revert prev change
giampaolo Jan 19, 2026
527646d
Use ceil(timeout) to avoid truncation
giampaolo Jan 19, 2026
0a8a1b2
Remove check for timeout < 0 + rm test case which didn't make sense.
giampaolo Jan 19, 2026
73b97dc
Don't use _blocking_wait() as it has a while loop
giampaolo Jan 19, 2026
2d3c3f7
Add docstring
giampaolo Jan 19, 2026
4eac42f
Rm _busy_wait()
giampaolo Jan 19, 2026
3c156a9
Add comment
giampaolo Jan 19, 2026
dac7d3b
Add assert
giampaolo Jan 19, 2026
5c7ec2f
Update comments
giampaolo Jan 19, 2026
5c29144
Add test for timeout=0
giampaolo Jan 19, 2026
4359b07
Update comment about PID reuse race
giampaolo Jan 19, 2026
81275c8
Update comment
giampaolo Jan 19, 2026
43b500f
Handle rare case where poll() says we're done, but waitpid() doesn't
giampaolo Jan 19, 2026
b64e42b
Update Doc/library/subprocess.rst
giampaolo Jan 19, 2026
a101406
Add news entry
giampaolo Jan 19, 2026
452f8c4
Add entry in Doc/whatsnew/3.15.rst
giampaolo Jan 19, 2026
b0c9890
Fix typo
giampaolo Jan 19, 2026
27b7c9f
Re-wording
giampaolo Jan 19, 2026
e1da996
Raise on timeout < 0 and re-add test case
giampaolo Jan 19, 2026
5c78acc
Check if can really use can_use_pidfd() in unit tests
giampaolo Jan 19, 2026
df0538a
Check if can really use kqueue() in unit tests
giampaolo Jan 19, 2026
6d8e36c
Pre-emptively check whether to use the fast way methods
giampaolo Jan 19, 2026
6ba7465
Add test_fast_path_avoid_busy_loop
giampaolo Jan 19, 2026
e3c7977
Update comments
giampaolo Jan 19, 2026
97cc3be
Fix missing import on Windows
giampaolo Jan 19, 2026
c4342e3
Merge branch 'main' into subprocess-fast-wait
giampaolo Jan 19, 2026
5d78d24
Try to fix doc build error
giampaolo Jan 19, 2026
a916da3
Try to fix doc build error 2
giampaolo Jan 19, 2026
86200bd
Try to fix doc build error 3
giampaolo Jan 19, 2026
85c38bc
Try to fix doc build error 4
giampaolo Jan 19, 2026
3c92c1d
Try to fix doc build error 5
giampaolo Jan 19, 2026
33c8b1f
Minor rewordings
giampaolo Jan 19, 2026
525c047
Small refact
giampaolo Jan 21, 2026
2b931b8
Address review comments (bare assert, don't use ceil())
giampaolo Jan 21, 2026
d1c6e91
Address review comments (use test class VARIABLES)
giampaolo Jan 21, 2026
3e9d303
Address review comments (add periods in doc)
giampaolo Jan 21, 2026
a44b2d7
Merge branch 'main' into subprocess-fast-wait
giampaolo Jan 21, 2026
b4f6020
Remove weird unicode char from doc added by accident
giampaolo Jan 21, 2026
4e17856
Don't import select on Windows
giampaolo Jan 21, 2026
6f6600d
Make kq.control() invocation clearer by passing kwargs
giampaolo Jan 21, 2026
37288c6
Revert previous commit
giampaolo Jan 21, 2026
300692e
Remove bare assert statements
giampaolo Jan 22, 2026
7bee1e9
Merge branch 'main' into subprocess-fast-wait
giampaolo Jan 22, 2026
File filter
Filter by extension

Clear filters
Conversations
Failed to load comments.
Loading

Uh oh!

There was an error while loading. Please reload this page.

Jump to
Jump to file
Failed to load files.
Loading

Uh oh!

There was an error while loading. Please reload this page.

Diff view
Diff view
Prev Previous commit
Next Next commit
Don't use _blocking_wait() as it has a while loop
  • Loading branch information

Uh oh!

There was an error while loading. Please reload this page.

@giampaolo
giampaolo committed Jan 19, 2026
commit 73b97dc089d56f2769e098d045f1fdc65e3854fe

There are no files selected for viewing

32 changes: 18 additions & 14 deletions 32 Lib/subprocess.py
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2135,18 +2135,6 @@ def _busy_wait(self, timeout):
delay = min(delay * 2, remaining, .05)
time.sleep(delay)

def _blocking_wait(self):
while self.returncode is None:
with self._waitpid_lock:
if self.returncode is not None:
break # Another thread waited.
(pid, sts) = self._try_wait(0)
# Check the pid and loop as waitpid has been known to
# return 0 even without WNOHANG in odd situations.
# http://bugs.python.org/issue14396.
if pid == self.pid:
self._handle_exitstatus(sts)

def _wait(self, timeout):
"""Internal implementation of wait() on POSIX."""
if self.returncode is not None:
Expand All @@ -2155,11 +2143,27 @@ def _wait(self, timeout):
if timeout is not None:
# Try fast wait first (pidfd on Linux, kqueue on BSD/macOS).
if self._wait_pidfd(timeout) or self._wait_kqueue(timeout):
self._blocking_wait()
with self._waitpid_lock:
if self.returncode is not None:
return self.returncode
pid, sts = self._try_wait(os.WNOHANG)
if pid == self.pid:
self._handle_exitstatus(sts)
return self.returncode
else:
self._busy_wait(timeout)
else:
self._blocking_wait()
while self.returncode is None:
with self._waitpid_lock:
if self.returncode is not None:
break # Another thread waited.
(pid, sts) = self._try_wait(0)
# Check the pid and loop as waitpid has been known to
# return 0 even without WNOHANG in odd situations.
# http://bugs.python.org/issue14396.
if pid == self.pid:
self._handle_exitstatus(sts)

return self.returncode


Expand Down
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.