GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — bpo-35537: subprocess can use posix_spawn with pipes by vstinner · Pull Request #11575 · python/cpython Conversation
|
|
Follow-up of PR #11452. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Technically, there is no need to explicitly close parent pipe ends because they have O_CLOEXEC. One small advantage of doing that is to reduce the number of descriptors before the following dup2, which could increase their number (in a corner case when any of descriptors 0, 1, 2 is closed in the parent process) and potentially hit resource limits.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Technically, there is no need, but _posixsubprocess does that. In case of doubt, I prefer to mimick _posixsubprocess behavior.
/* Close parent's pipe ends. */
if (p2cwrite != -1)
POSIX_CALL(close(p2cwrite));
if (c2pread != -1)
POSIX_CALL(close(c2pread));
if (errread != -1)
POSIX_CALL(close(errread));
POSIX_CALL(close(errpipe_read));
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
If we decide to skip the close(), I would prefer to also modify _posixsubprocess. But here I only want to use posix_spawn() in more cases, so I prefer to not modify "unrelated" changes :-)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
* subprocess.Popen can now also use os.posix_spawn() with pipes if pipe file descriptors are greater than 2. * Fix Popen._posix_spawn(): set _child_created to True. * Add Popen._close_pipe_fds() helper function to factorize the code.
|
Since PR #11579, I rebased this PR again :-) |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@gpshead @giampaolo: would you mind to review this change? It allows to use posix_spawn() in subprocess in much more cases. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
If I don't get any feedback before the end of the week, I will just merge my PR. I prefer to merge it early during the 3.8 development cycle, to get more time to fix it if something goes wrong :) |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| os.close(errwrite) | ||
|
|
||
| if devnull_fd is not None: | ||
| os.close(devnull_fd) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
In case of premature failure on X.Close() or os.close(X) the remaining pipes/fds will remain "open". Perhaps it makes sense to use contextlib.ExitStack.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Is it a regression caused by my PR? Or a general remark on existing code? My PR just moves code, no?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Yep, it doesn't look like a regression introduced by you (was already there).
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
At least, I factorized the code so it should help to enhance the code ;-) Maybe open an issue if you want to work on that? I don't see an obvious pattern to ensure that all file descriptors are properly closed.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Yeah, makes sense. I bet there are many other places which can benefit from ExitStack in a similar manner.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
When you're done making the requested changes, leave the comment: |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
@giampaolo: So what do you think of the overall change? :-)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| os.close(errwrite) | ||
|
|
||
| if devnull_fd is not None: | ||
| os.close(devnull_fd) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
At least, I factorized the code so it should help to enhance the code ;-) Maybe open an issue if you want to work on that? I don't see an obvious pattern to ensure that all file descriptors are properly closed.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@giampaolo: "LGTM" Yay! Thanks for the approval! I merged my PR. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@giampaolo: As I wrote, please open a new issue if you would like to enhance the new _close_pipe_fds() helper method. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Close pipes/fds in subprocess by using ExitStack. "In case of premature failure on X.Close() or os.close(X) the remaining pipes/fds will remain "open". Perhaps it makes sense to use contextlib.ExitStack." - Rationale: #11575 (comment)
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.