GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-85299: Add note warning about entry point guard for asyncio example by slateny · Pull Request #93457 · python/cpython

Conversation

@slateny

@slateny slateny commented Jun 3, 2022

Copy link
Copy Markdown
Contributor

#85299

https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@slateny
Add note warning about entry point guard for asyncio example
97d022e
@slateny
slateny requested review from 1st1 and asvetlov as code owners June 3, 2022 09:33
@bedevere-bot bedevere-bot added docs Documentation in the Doc dir awaiting review labels Jun 3, 2022
@slateny slateny added the skip news label Jun 3, 2022
kumaraditya303
kumaraditya303 reviewed Oct 13, 2022
View reviewed changes
Comment thread Doc/library/asyncio-eventloop.rst Outdated

asyncio.run(main())

Note that on macOS, an entry point guard may be needed as the example would

@kumaraditya303 kumaraditya303 Oct 13, 2022

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to just add __name__ == '__main__'. This happens on Windows too not just macOS.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@gvanrossum gvanrossum Oct 13, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@slateny slateny Oct 14, 2022

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point, Windows also defaults to spawn so looks like it runs into the same issue as mac. Should this note still remain as an fyi, or just change the example to fix the problem and leave out the note? It's customary to have the guard in general but knowing why it fails without might still be helpful, but I don't feel too strongly either way.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@kumaraditya303
kumaraditya303 requested review from gvanrossum and kumaraditya303 October 13, 2022 13:43
gvanrossum
gvanrossum requested changes Oct 13, 2022
View reviewed changes
Comment thread Doc/library/asyncio-eventloop.rst Outdated

asyncio.run(main())

Note that on macOS, an entry point guard may be needed as the example would

@gvanrossum gvanrossum Oct 13, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-bot bedevere-bot added awaiting changes and removed awaiting review labels Oct 13, 2022
@bedevere-bot

bedevere-bot commented Oct 13, 2022

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@gvanrossum

gvanrossum commented Oct 14, 2022

Copy link
Copy Markdown
Member

Okay, I am beginning to understand the issue. It's due to the process pool example. If I comment out this block of code from the example:

    # 3. Run in a custom process pool:
    with concurrent.futures.ProcessPoolExecutor() as pool:
        result = await loop.run_in_executor(
            pool, cpu_bound)
        print('custom process pool', result)

everything is fine. If I leave that in the run (on macOS) fails spectacularly with a RuntimeError that has many lines of traceback, then a multi-line error message, and then more traceback lines. Adding the __main__ guard makes this go away.

I presume that's one of the many reasons not to use ProcessPoolExecutor. :-(

I propose to rephrase the note to mention the platform and just link to https://docs.python.org/3/library/multiprocessing.html#the-spawn-and-forkserver-start-methods, not to the change note.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@slateny
Add entry guard, clarify note on platform
16cb255
gvanrossum
gvanrossum reviewed Oct 15, 2022
View reviewed changes
Comment thread Doc/library/asyncio-eventloop.rst Outdated
Comment on lines +1212 to +1215
Note that on Windows and macOS, the entry point guard may be necessary as the
example would otherwise cause a :exc:`RuntimeError`. For more details, see the
`Safe importing of main module
<https://docs.python.org/3/library/multiprocessing.html#the-spawn-and-forkserver-start-methods>`_.

@gvanrossum gvanrossum Oct 15, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to simplify the note to the following:

Note that the entry point guard (if __name__ == '__main__') is required for option 3, due to the peculiarities of multiprocessing, which is used by ProcessPoolExecutor. See (link).

There should be three cross-references:

  • multiprocessing (the module)
  • ProcessPoolExecutor (the class)
  • Safe importing of main module

For the latter I recommend to add a label to the target section in multiprocessing.rst rather than linking to the website. (@CAM-Gerlach can provide details on how to do this.)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@slateny slateny Oct 16, 2022

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding though is that option 3 fails when spawn/forkserver is the process start method, and only Windows + mac default to spawn, while unix defaults to fork, so saying 'is required' may be a bit too strong. If 'may be required' instead, then the reader might wonder why 'may', in which case I think a link to https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods somewhere would help explain things.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@gvanrossum gvanrossum Oct 16, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's too subtle for the readers. We're not stating the law, we're just explaining the example.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
slateny reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@CAM-Gerlach CAM-Gerlach Oct 16, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment for a suggestion implementing the above and the details Guido mentioned.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
CAM-Gerlach
CAM-Gerlach reviewed Oct 16, 2022
View reviewed changes
Comment thread Doc/library/asyncio-eventloop.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.
slateny and others added 3 commits October 16, 2022 06:06
@slateny
Add ref target in multiprocessing
0129bdf
@slateny @CAM-Gerlach
Use rewording for multiprocessing
fd68a5e 
Co-authored-by: C.A.M. Gerlach <[email protected]>
@slateny
Add missing grave sign
8ba4b0c
gvanrossum
gvanrossum approved these changes Oct 16, 2022
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting changes labels Oct 16, 2022
@gvanrossum
gvanrossum merged commit 79fd6cc into python:main Oct 16, 2022
@bedevere-bot bedevere-bot removed the awaiting merge label Oct 16, 2022
@gvanrossum gvanrossum added needs backport to 3.10 needs backport to 3.11 only security fixes labels Oct 16, 2022
@miss-islington

miss-islington commented Oct 16, 2022

Copy link
Copy Markdown
Contributor

Thanks @slateny for the PR, and @gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@miss-islington

miss-islington commented Oct 16, 2022

Copy link
Copy Markdown
Contributor

Thanks @slateny for the PR, and @gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 16, 2022
@slateny @miss-islington
pythongh-85299: Add note warning about entry point guard for asyncio …
09fe4f9 
…example (pythonGH-93457)

(cherry picked from commit 79fd6cc)

Co-authored-by: Stanley <[email protected]>
@bedevere-bot

bedevere-bot commented Oct 16, 2022

Copy link
Copy Markdown

GH-98334 is a backport of this pull request to the 3.11 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Oct 16, 2022
@gvanrossum

gvanrossum commented Oct 16, 2022

Copy link
Copy Markdown
Member

Let’s see if we can backport this.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot

bedevere-bot commented Oct 16, 2022

Copy link
Copy Markdown

GH-98335 is a backport of this pull request to the 3.10 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the needs backport to 3.10 label Oct 16, 2022
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 16, 2022
@slateny @miss-islington
pythongh-85299: Add note warning about entry point guard for asyncio …
8a7bff2 
…example (pythonGH-93457)

(cherry picked from commit 79fd6cc)

Co-authored-by: Stanley <[email protected]>
miss-islington added a commit that referenced this pull request Oct 16, 2022
@miss-islington @slateny
gh-85299: Add note warning about entry point guard for asyncio example (
2630738 
GH-93457)

(cherry picked from commit 79fd6cc)

Co-authored-by: Stanley <[email protected]>
miss-islington added a commit that referenced this pull request Oct 16, 2022
@miss-islington @slateny
gh-85299: Add note warning about entry point guard for asyncio example (
e26c526 
GH-93457)

(cherry picked from commit 79fd6cc)

Co-authored-by: Stanley <[email protected]>
@slateny
slateny deleted the s/85299 branch October 16, 2022 20:40
carljm added a commit to carljm/cpython that referenced this pull request Oct 17, 2022
@carljm
Merge branch 'main' into typewatch
cdbe93b 
* main: (31 commits)
  pythongh-95913: Move subinterpreter exper removal to 3.11 WhatsNew (pythonGH-98345)
  pythongh-95914: Add What's New item describing PEP 670 changes (python#98315)
  Remove unused arrange_output_buffer function from zlibmodule.c. (pythonGH-98358)
  pythongh-98174: Handle EPROTOTYPE under macOS in test_sendfile_fallback_close_peer_in_the_middle_of_receiving (python#98316)
  pythonGH-98327: Reduce scope of catch_warnings() in _make_subprocess_transport (python#98333)
  pythongh-93691: Compiler's code-gen passes location around instead of holding it on the global compiler state (pythonGH-98001)
  pythongh-97669: Create Tools/build/ directory (python#97963)
  pythongh-95534: Improve gzip reading speed by 10% (python#97664)
  pythongh-95913: Forward-port int/str security change to 3.11 What's New in main (python#98344)
  pythonGH-91415: Mention alphabetical sort ordering in the Sorting HOWTO (pythonGH-98336)
  pythongh-97930: Merge with importlib_resources 5.9 (pythonGH-97929)
  pythongh-85525: Remove extra row in doc (python#98337)
  pythongh-85299: Add note warning about entry point guard for asyncio example (python#93457)
  pythongh-97527: IDLE - fix buggy macosx patch (python#98313)
  pythongh-98307: Add docstring and documentation for SysLogHandler.createSocket (pythonGH-98319)
  pythongh-94808: Cover `PyFunction_GetCode`, `PyFunction_GetGlobals`, `PyFunction_GetModule` (python#98158)
  pythonGH-94597: Deprecate child watcher getters and setters (python#98215)
  pythongh-98254: Include stdlib module names in error messages for NameErrors (python#98255)
  Improve speed. Reduce auxiliary memory to 16.6% of the main array. (pythonGH-98294)
  [doc] Update logging cookbook with an example of custom handling of levels. (pythonGH-98290)
  ...
pablogsal pushed a commit that referenced this pull request Oct 22, 2022
@miss-islington @slateny @pablogsal
gh-85299: Add note warning about entry point guard for asyncio example (
a2820a0 
GH-93457)

(cherry picked from commit 79fd6cc)

Co-authored-by: Stanley <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CAM-Gerlach CAM-Gerlach CAM-Gerlach left review comments
@gvanrossum gvanrossum gvanrossum approved these changes
@1st1 1st1 Awaiting requested review from 1st1 1st1 is a code owner
@asvetlov asvetlov Awaiting requested review from asvetlov asvetlov is a code owner
@kumaraditya303 kumaraditya303 Awaiting requested review from kumaraditya303 kumaraditya303 is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

docs Documentation in the Doc dir skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@slateny @bedevere-bot @gvanrossum @miss-islington @CAM-Gerlach @kumaraditya303
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.