GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-35537: Document posix_spawn() change in subprocess by vstinner · Pull Request #11668 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented Jan 24, 2019 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Document that subprocess.Popen no longer raise an exception on error
like missing program on very specific platforms when using
os.posix_spawn() is used.

https://bugs.python.org/issue35537

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner
vstinner requested a review from gpshead as a code owner January 24, 2019 16:49
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Jan 24, 2019
@bedevere-bot bedevere-bot added docs Documentation in the Doc dir awaiting merge labels Jan 24, 2019
eamanu
eamanu reviewed Jan 25, 2019
View reviewed changes
Comment thread Doc/library/subprocess.rst Outdated

@eamanu eamanu Jan 25, 2019

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about if we say: Now, Popen can use ... ```` or just remove the now``` .

Just a suggestion

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Doc/library/subprocess.rst Outdated

@eamanu eamanu Jan 25, 2019

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you mention some example of that platforms?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Jan 25, 2019

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to not mention WSL and QEMU User Emulation since it's a bug and these bugs can be fixed whenever. I prefer to not mention them to be more future-proof (not have to update the doc).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@gpshead gpshead Jan 26, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd mention the platforms anyways with a "at the time of this writing, buggy platforms we are aware of include Windows Subsystem for Linux (version XYZ) and QEMU (version XYZ)" ... If there are bugreports filed against them, those can be linked to from the doc as well.

Otherwise it such text sets off alarm bells for anyone reading it, they'll have the same question: am I impacted? do I use any "very specific platforms"? and come seeking more information.

when we become aware of upstream bugs being fixed we can update the text to say "versions prior to ABC" or similar.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner

vstinner commented Jan 25, 2019

Copy link
Copy Markdown
Member Author

@izbyshev: Would you mind to review this change? Are you fine with documentating the limitation rather trying to fixing it?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner vstinner added the skip news label Jan 25, 2019
@izbyshev

izbyshev commented Jan 25, 2019 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

@vstinner I hope that problems with QEMU, WSL and any other Linux environments where vfork is emulated via fork will be avoided if bpo-35823 is accepted. If that happens, there will be no known platform that both (a) has the problem you describe and (b) is eligible for posix_spawn as currently implemented in subprocess. Therefore, this change will effectively be the statement that we're OK with using posix_spawn in the future on platforms that suffer from such problems. Or, to put it another way, that we're willing to take the risk if we can't reliably check whether such issues exist when we decide to use posix_spawn. Is that what you intend to do?

Personally, I'd wait at least until we're done with bpo-35823. If it's merged, I'd prefer not to merge a part about "very specific platforms" and wait until we encounter another such case. If it's rejected, I agree that we should document this issue. Also, we could add a parameter to subprocess.Popen() to allow users to force traditional fork()/exec() path to workaround such issues.

gpshead reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jan 25, 2019 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Personally, I'd wait at least until we're done with bpo-35823.

Ok, let's do that ;-)

gpshead reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner vstinner added the DO-NOT-MERGE label Jan 25, 2019
@vstinner vstinner changed the title bpo-35537: Document posix_spawn() change in subprocess [WIP] bpo-35537: Document posix_spawn() change in subprocess Jan 25, 2019
@vstinner

vstinner commented Jan 25, 2019

Copy link
Copy Markdown
Member Author

@izbyshev: More generally, I like how things are going one. Slowly, we listed more and more bugs and corner cases, and we had to experiment posix_spawn() and dig into the code (well, in practice that part mostly done by you @izbyshev :-D) to be able to get this list. I'm fine with dropping posix_spawn() from subprocess if we can successfully use vfork in _posixsubprocess. I'm open to different options, as soon as subprocess runs faster :-)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
bpo-35537: Document posix_spawn() change in subprocess
1450a33
Document that subprocess.Popen no longer raise an exception on error
like missing program on very specific platforms when using
os.posix_spawn() is used.
@vstinner vstinner changed the title [WIP] bpo-35537: Document posix_spawn() change in subprocess bpo-35537: Document posix_spawn() change in subprocess Apr 16, 2019
@vstinner vstinner removed the DO-NOT-MERGE label Apr 16, 2019
@vstinner

vstinner commented Apr 16, 2019

Copy link
Copy Markdown
Member Author

Four months later, I see no activity on the idea of modiying _posixsubprocess to use vfork(), so I updated this change and I propose to merge it.

@eamanu and @gpshead: Would you mind to review my update PR?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
vstinner merged commit d7befad into python:master Apr 25, 2019
@bedevere-bot bedevere-bot removed the awaiting merge label Apr 25, 2019
@vstinner
vstinner deleted the subprocess_spawn_note branch April 25, 2019 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gpshead gpshead Awaiting requested review from gpshead
1 more reviewer
@eamanu eamanu eamanu left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

docs Documentation in the Doc dir skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@vstinner @izbyshev @gpshead @eamanu @the-knights-who-say-ni @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.