GeezerVison is showing github.com — Introduce task groups to asyncio and change task cancellation semantics · Issue #90908 · python/cpython
Introduce task groups to asyncio and change task cancellation semantics #90908Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/gvanrossum'
closed_at = None
created_at = <Date 2022-02-14.20:44:29.619>
labels = ['3.11', 'expert-asyncio']
title = 'Introduce task groups to asyncio and change task cancellation semantics'
updated_at = <Date 2022-02-25.15:33:21.072>
user = 'https://github.com/gvanrossum' bugs.python.org fields:
activity = <Date 2022-02-25.15:33:21.072>
actor = 'yduprat'
assignee = 'gvanrossum'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2022-02-14.20:44:29.619>
creator = 'gvanrossum'
dependencies = []
files = []
hgrepos = []
issue_num = 46752
keywords = ['patch', 'needs review']
message_count = 10.0
messages = ['413260', '413305', '413306', '413335', '413344', '413346', '413348', '413353', '413467', '413577']
nosy_count = 8.0
nosy_names = ['gvanrossum', 'njs', 'asvetlov', 'dhalbert', 'yselivanov', 'tinchester', 'iritkatriel', 'yduprat']
pr_nums = ['31270', '31398', '31409', '31411', '31513', '31559']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue46752'
versions = ['Python 3.11'] Activitygvanrossum commented on Feb 14, 2022 After some conversations with Yury, and encouraged by the SC's approval of PEP-654, I am proposing to add a new class, asyncio.TaskGroup, which introduces structured concurrency similar to nurseries in Trio.
I started with EdgeDb's TaskGroup implementation (https://github.com/edgedb/edgedb/blob/master/edb/common/taskgroup.py) and tweaked it only slightly. I also changed a few things in asyncio.Task (see below).
The key change I made to EdgeDb's TaskGroup is that subtasks can keep spawning more subtasks while __aexit__ is running; __aexit__ exits once the last subtask is done. I made this change after consulting some Trio folks, who knew of real-world use cases for this behavior, and did not know of real-world code in need of prohibiting task creation as soon as __aexit__ starts running. I added some tests for the new behavior; none of the existing tests needed to be adjusted to accommodate this change.
(For other changes relative to the EdgeDb's TaskGroup, see #75453.)
In order to avoid the need to monkey-patch the parent task, I added two new methods to asyncio.Task, .cancelled() and .uncancel(), that manage a flag corresponding to __cancel_requested__ in EdgeDb's TaskGroup.
**This introduces a change in behavior around task cancellation:**
- A task that catches CancelledError is allowed to run undisturbed (ignoring further .cancel() calls and allowing any number of await calls!) until it either exits or calls .uncancel().
This change in semantics did not cause any asyncio unittests to fail. However, it may be surprising (especially to Trio folks, where the semantics are pretty much the opposite, once a Trio task is cancelled all further await calls in that task fail unless explicitly shielded).
For the TaskGroup tests to pass, we require a flag that is not cleared. However, it is probably not really required to ignore subsequent .cancel() calls until .uncancel() is called. This just seemed more consistent, and it is what @asvetlov proposed above and implemented in #75494 (using a property .__cancel_requested__ as the API). Reacted by Frederik Aalund and Hash gvanrossum commented on Feb 15, 2022 gvanrossum commented on Feb 15, 2022 Remaining TODO list:
- Add a test showing the need for the .uncancel() call in __aexit__()
(currently on line 97). Dropping that line does not cause any tests
to fail.
- Ensure the taskgroup tests are run with the C and Python Task
implementations.
- Rename tests to have meaningful names.
- I have a few ideas for minor cleanups that I will do later.
- Documentation and What's New entry (in a separate PR, probably).
- Update the docs in a few places to de-prioritize asyncio.gather()
and steer people towards TaskGroups.
(We could also add something like Trio's cancel scopes, e.g. based on
Andrew Svetlov's async-timeout, which has a mature API.
But that should be a separate bpo issue.) dhalbert commented on Feb 16, 2022 gvanrossum commented on Feb 16, 2022 @dhalbert, it's probably better to file a new issue if you want changes to gather(). Although I suppose that if we want to deemphasize it, we shouldn't be adding new features to it. My own new feature idea would be to have it wait for all tasks and then if there are any exceptions, raise an ExceptionGroup. That (like any new gather() behaviors) would require a new keyword-only flag to gather(). If we're going to deemphasize it I might not bother though.
There's one thing that gather() does that TaskGroup doesn't: it gives us the return values from the tasks. The question is whether that's useful. If it is maybe we should *not* deepmhasize gather() quite as much and then adding new features would be okay. gvanrossum commented on Feb 16, 2022 I've created a separate issue for cancel scopes: bpo-46771. 1st1 commented on Feb 16, 2022
There's one thing that gather() does that TaskGroup doesn't: it gives us the return values from the tasks.
That's easy to do with task groups too:
async with TaskGroup() as g:
r1 = g.create_task(coro1())
r2 = g.create_task(coro2())
print(r1.result())
# or
print(await r2) # I *think* this should workReacted by Giorgi Reacted by Giorgi gvanrossum commented on Feb 16, 2022 I have a PR up to typeshed to add the new Task methods and a new stub file taskgroups.pyi: python/typeshed#7240 gvanrossum commented on Feb 18, 2022 gvanrossum commented on Jun 30, 2022 Now the docs are up we can close this. added a commit that references this issue on Oct 1, 2022gh-90908: Document asyncio.Task.cancelling() and asyncio.Task.uncance… added 2 commits that reference this issue on Oct 1, 2022pythongh-90908: Document asyncio.Task.cancelling() and asyncio.Task.u… gh-90908: Document asyncio.Task.cancelling() and asyncio.Task.uncance… added a commit that references this issue on Oct 24, 2022gh-90908: Document asyncio.Task.cancelling() and asyncio.Task.uncance…
MetadataMetadataStatusShow more project fields StatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|