GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-115957: Close coroutine if the TaskGroup is inactive by Jason-Y-Z · Pull Request #116009 · python/cpython

Conversation

@Jason-Y-Z

@Jason-Y-Z Jason-Y-Z commented Feb 27, 2024 •
edited by terryjreedy
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

This PR addresses the issue where when we call TaskGroup.create_task() with an inactive TaskGroup, we still get a RuntimeWarning for the coroutine. This change makes sure that the coroutine is closed in this case and updated the tests accordingly.


📚 Documentation preview 📚: https://cpython-previews--116009.org.readthedocs.build/

  • Issue: asyncio.TaskGroup does not close unawaited coroutines #115957

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

qin-yu reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@Jason-Y-Z
Jason-Y-Z requested review from 1st1, asvetlov, gvanrossum, kumaraditya303 and willingc as code owners February 27, 2024 21:56
@bedevere-app bedevere-app Bot added the awaiting review label Feb 27, 2024
@bedevere-app bedevere-app Bot mentioned this pull request Feb 27, 2024
asyncio.TaskGroup does not close unawaited coroutines #115957
Closed
@Jason-Y-Z
Jason-Y-Z force-pushed the fix-issue-115957 branch 2 times, most recently from e4ab02f to 6a0265d Compare February 27, 2024 22:17
@arthur-tacca

arthur-tacca commented Feb 28, 2024

Copy link
Copy Markdown

Thanks for doing this! I was dangerously close to doing it myself even though I really don't have time, just for the kudos of being a "Python contributor", so I'm really glad you did 😄

My comments:

  • You have described this as affecting "...if the task group has been closed" but it also affects task groups that have not yet been entered and those that are shutting down (but not yet closed). Perhaps the wording should be something like "when create_task is called on a TaskGroup that is not active (because it has not yet been entered, is shutting down or has closed)"
  • I could be mistaken but I think that test would pass even without your change? It looks like it tests that asyncio.TaskGroup.create_task() will raise RuntimeError if the task group has closed, but it already did that. To test the actual change it would need to check that the coroutine has been closed (by calling coro.send(None) and checking that gives RuntimeError: cannot reuse already awaited coroutine).
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Jason-Y-Z

Jason-Y-Z commented Feb 28, 2024

Copy link
Copy Markdown
Contributor Author

Thanks for doing this! I was dangerously close to doing it myself even though I really don't have time, just for the kudos of being a "Python contributor", so I'm really glad you did 😄

My comments:

  • You have described this as affecting "...if the task group has been closed" but it also affects task groups that have not yet been entered and those that are shutting down (but not yet closed). Perhaps the wording should be something like "when create_task is called on a TaskGroup that is not active (because it has not yet been entered, is shutting down or has closed)"
  • I could be mistaken but I think that test would pass even without your change? It looks like it tests that asyncio.TaskGroup.create_task() will raise RuntimeError if the task group has closed, but it already did that. To test the actual change it would need to check that the coroutine has been closed (by calling coro.send(None) and checking that gives RuntimeError: cannot reuse already awaited coroutine).

Thanks for the comments Arthur! Yeah that makes sense, I will update the wording there.

Regarding the test, it raises a RuntimeWarning without the change for me. Maybe we have slightly different configs? But yeah let me try more specifically sending on the coroutine as well. Thanks again!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@arthur-tacca

arthur-tacca commented Feb 28, 2024

Copy link
Copy Markdown

Sorry, to be honest I haven't actually run the tests (or for that matter cloned the CPython repo at all), I was just reading the test code. If the test suite automatically fails if there's a warning then you can ignore that comment.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Jason-Y-Z
Jason-Y-Z force-pushed the fix-issue-115957 branch from 7f28db0 to 8e9866e Compare February 28, 2024 20:41
@Jason-Y-Z Jason-Y-Z changed the title gh-115957: Close coroutine if the TaskGroup has been closed gh-115957: Close coroutine if the TaskGroup is inactive Feb 28, 2024
@Jason-Y-Z

Jason-Y-Z commented Feb 29, 2024

Copy link
Copy Markdown
Contributor Author

Sorry for tagging but a gentle nudge for reviewing please 🙏 @gvanrossum

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gvanrossum

gvanrossum commented Mar 1, 2024

Copy link
Copy Markdown
Member

Sorry for tagging but a gentle nudge for reviewing please 🙏 @gvanrossum

Yeah, it's in my queue.

Jason-Y-Z reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Jason-Y-Z

Jason-Y-Z commented Mar 1, 2024

Copy link
Copy Markdown
Contributor Author

Sorry for tagging but a gentle nudge for reviewing please 🙏 @gvanrossum

Yeah, it's in my queue.

Got it, thank you 🙏

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Jason-Y-Z
Jason-Y-Z force-pushed the fix-issue-115957 branch 2 times, most recently from 542d7c2 to e7c3a5e Compare March 2, 2024 11:30
gvanrossum
gvanrossum requested changes Mar 5, 2024
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the tests you changed and added still pass even if I comment out the three coro.close() calls added to taskgroups.py. While they print the RuntimeWarning message, they don't fail. Could you try to detect the warnings in at least one of the tests?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Doc/library/asyncio-task.rst
Comment on lines +338 to +344
.. versionchanged:: 3.13

Close the given coroutine if the task group is not active.

@gvanrossum gvanrossum Mar 5, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that the convention is that a description of the new/changed behavior is added to the main text of the documentation (in this case the two lines right above) and then the versionchanged directive is used to clarify when this behavior was added or changed. In this case that would be a little redundant but I still feel it's appropriate.

For example, the main text could include information about when a task group is considered inactive (e.g. not yet entered, already finished, or in the process of shutting down).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@Jason-Y-Z Jason-Y-Z Mar 5, 2024

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation! I've updated the main text

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Doc/whatsnew/3.13.rst Outdated

* When :func:`asyncio.TaskGroup.create_task` is called on an inactive
:class:`asyncio.TaskGroup`, the given coroutine will be closed (which
prevents a :exc:`RuntimeWarning`).

@gvanrossum gvanrossum Mar 5, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd clarify that the RuntimeWarning is about "coroutine 'blank' was never awaited".

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@Jason-Y-Z Jason-Y-Z Mar 5, 2024

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comments! I've updated the wording

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot added awaiting changes and removed awaiting review labels Mar 5, 2024
@bedevere-app

bedevere-app Bot commented Mar 5, 2024

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@terryjreedy terryjreedy added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label Mar 5, 2024
@terryjreedy terryjreedy changed the title gh-115957: Close coroutine if the TaskGroup is inactive gh-115957: Close coroutine if the TaskGroup is inactive Mar 5, 2024
Jason-Y-Z and others added 3 commits March 5, 2024 08:51
@Jason-Y-Z
pythongh-115957: Close coroutine if the TaskGroup is inactive
5bcc3d5
@blurb-it @Jason-Y-Z
📜🤖 Added by blurb_it.
19c8bea
@Jason-Y-Z
Catch warnings in inactive tg test and update docs
5d0d518
@Jason-Y-Z
Jason-Y-Z force-pushed the fix-issue-115957 branch from 22d94f3 to 5d0d518 Compare March 5, 2024 09:21
@Jason-Y-Z

Jason-Y-Z commented Mar 5, 2024

Copy link
Copy Markdown
Contributor Author

It looks like the tests you changed and added still pass even if I comment out the three coro.close() calls added to taskgroups.py. While they print the RuntimeWarning message, they don't fail. Could you try to detect the warnings in at least one of the tests?

Thanks for the detailed explanations @gvanrossum ! I have updated one of the tests so that it now explicitly captures and asserts no warning. I've tried it on my machine (MacOS) and it is now failing without the change. Please let me know if there is any other improvement I can apply. Thanks!

I have made the requested changes; please review again

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot added awaiting change review and removed awaiting changes labels Mar 5, 2024
@bedevere-app

bedevere-app Bot commented Mar 5, 2024

Copy link
Copy Markdown

Thanks for making the requested changes!

@gvanrossum: please review the changes made to this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app
bedevere-app Bot requested a review from gvanrossum March 5, 2024 09:26
gvanrossum
gvanrossum approved these changes Mar 6, 2024
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks good! The tests pass in CI, one of the tests fails if I undo the changes in taskgroups.py only, so I will merge.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting change review labels Mar 6, 2024
@gvanrossum

gvanrossum commented Mar 6, 2024

Copy link
Copy Markdown
Member

PS. Next time please don't force push. We squash changes when we merge into main anyway, and the force push makes it harder to re-review the code.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gvanrossum
gvanrossum merged commit ce0ae1d into python:main Mar 6, 2024
@bedevere-app bedevere-app Bot removed the awaiting merge label Mar 6, 2024
@Jason-Y-Z

Jason-Y-Z commented Mar 7, 2024

Copy link
Copy Markdown
Contributor Author

Thanks @gvanrossum for the review and for the note! I will remember not to force push next time 👌

gvanrossum reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 25, 2024
@Jason-Y-Z @adorilson
pythongh-115957: Close coroutine if TaskGroup.create_task() raises an…
258c243
… error (python#116009)
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
@Jason-Y-Z @diegorusso
pythongh-115957: Close coroutine if TaskGroup.create_task() raises an…
d5be5e3
… error (python#116009)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gvanrossum gvanrossum gvanrossum approved these changes
@1st1 1st1 Awaiting requested review from 1st1 1st1 is a code owner
@asvetlov asvetlov Awaiting requested review from asvetlov asvetlov is a code owner
@kumaraditya303 kumaraditya303 Awaiting requested review from kumaraditya303 kumaraditya303 is a code owner
@willingc willingc Awaiting requested review from willingc willingc is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

interpreter-core (Objects, Python, Grammar, and Parser dirs)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@Jason-Y-Z @arthur-tacca @gvanrossum @terryjreedy
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.