GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-128588: gh-128550: remove eager tasks optimization that missed and introduced incorrect cancellations by graingert · Pull Request #129063 · python/cpython

Conversation

@graingert

@graingert graingert commented Jan 20, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

… introduced incorrect cancellations

  • Issue: Cancellation leaks out of asyncio.TaskGroup on 3.12 when using eager tasks #128588
  • Issue: TaskGroup fails to cancel tasks pending creation, if that task starts a failing task #128550

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@graingert
pythongh-128588: pythongh-128550: remove eager tasks optimization tha…
9fce599 
…t missed and introduced incorrect cancellations
@bedevere-app bedevere-app Bot mentioned this pull request Jan 20, 2025
Cancellation leaks out of asyncio.TaskGroup on 3.12 when using eager tasks #128588
Closed
@blurb-it
📜🤖 Added by blurb_it.
73042cf
@graingert graingert mentioned this pull request Jan 20, 2025
TaskGroup's task's exception's participate in a reference cycle, via TaskGroup.create_task -> task -> exception and loop.create_task -> task -> exception #128552
Closed
@graingert
graingert marked this pull request as ready for review January 20, 2025 13:23
@graingert
graingert requested review from 1st1, asvetlov, kumaraditya303 and willingc as code owners January 20, 2025 13:23
@bedevere-app bedevere-app Bot added the awaiting review label Jan 20, 2025
kumaraditya303
kumaraditya303 reviewed Jan 20, 2025
View reviewed changes
Comment thread Lib/test/test_asyncio/test_taskgroups.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
kumaraditya303
kumaraditya303 reviewed Jan 20, 2025
View reviewed changes
Comment thread Lib/test/test_asyncio/test_taskgroups.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
graingert
graingert commented Jan 20, 2025
View reviewed changes
Comment thread Lib/test/test_asyncio/test_taskgroups.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
@graingert
Update Lib/test/test_asyncio/test_taskgroups.py
867bacb
kumaraditya303
kumaraditya303 reviewed Jan 20, 2025
View reviewed changes
Comment thread Misc/NEWS.d/next/Library/2025-01-20-13-12-39.gh-issue-128550.AJ5TOL.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.
graingert and others added 2 commits January 20, 2025 16:27
@graingert
Update Lib/test/test_asyncio/test_taskgroups.py
c460fa9
@graingert @kumaraditya303
Update Misc/NEWS.d/next/Library/2025-01-20-13-12-39.gh-issue-128550.A…
187abc7 
…J5TOL.rst

Co-authored-by: Kumar Aditya <[email protected]>
graingert
graingert commented Jan 20, 2025
View reviewed changes
Comment thread Misc/NEWS.d/next/Library/2025-01-20-13-12-39.gh-issue-128550.AJ5TOL.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.
kumaraditya303
kumaraditya303 reviewed Jan 20, 2025
View reviewed changes
Comment thread Lib/test/test_asyncio/test_taskgroups.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
kumaraditya303 and others added 2 commits January 20, 2025 22:01
@kumaraditya303
Update Lib/test/test_asyncio/test_taskgroups.py
874f758
@kumaraditya303 @graingert
Update Misc/NEWS.d/next/Library/2025-01-20-13-12-39.gh-issue-128550.A…
74ef5cc 
…J5TOL.rst

Co-authored-by: Thomas Grainger <[email protected]>
kumaraditya303
kumaraditya303 approved these changes Jan 20, 2025
View reviewed changes
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Jan 20, 2025
@kumaraditya303
kumaraditya303 enabled auto-merge (squash) January 20, 2025 16:45
@kumaraditya303 kumaraditya303 added type-bug An unexpected behavior, bug, or error topic-asyncio needs backport to 3.12 only security fixes needs backport to 3.13 only security fixes labels Jan 20, 2025
@kumaraditya303
kumaraditya303 merged commit ed6934e into python:main Jan 20, 2025
@bedevere-app bedevere-app Bot removed the awaiting merge label Jan 20, 2025
@miss-islington-app

miss-islington-app Bot commented Jan 20, 2025

Copy link
Copy Markdown

Thanks @graingert for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jan 20, 2025
@graingert @kumaraditya303 @miss-islington
pythongh-128588: pythongh-128550: remove eager tasks optimization tha…
70fb2d0 
…t missed and introduced incorrect cancellations (pythonGH-129063)

(cherry picked from commit ed6934e)

Co-authored-by: Thomas Grainger <[email protected]>
Co-authored-by: Kumar Aditya <[email protected]>
@miss-islington-app

miss-islington-app Bot commented Jan 20, 2025

Copy link
Copy Markdown

Sorry, @graingert and @kumaraditya303, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker ed6934e71e55d398df8263f4697f58e4a3815f69 3.12
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app

bedevere-app Bot commented Jan 20, 2025

Copy link
Copy Markdown

GH-129089 is a backport of this pull request to the 3.13 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@miss-islington-app miss-islington-app Bot assigned kumaraditya303 Jan 20, 2025
@bedevere-app bedevere-app Bot removed the needs backport to 3.13 only security fixes label Jan 20, 2025
@graingert
graingert deleted the fix-eager-tasks-early-callback branch January 20, 2025 17:36
kumaraditya303 added a commit that referenced this pull request Jan 20, 2025
@miss-islington @graingert @kumaraditya303
[3.13] gh-128588: gh-128550: remove eager tasks optimization that mis…
a1c48a7 
…sed and introduced incorrect cancellations (GH-129063) (#129089)

gh-128588: gh-128550: remove eager tasks optimization that missed and introduced incorrect cancellations (GH-129063)
(cherry picked from commit ed6934e)

Co-authored-by: Thomas Grainger <[email protected]>
Co-authored-by: Kumar Aditya <[email protected]>
graingert added a commit to graingert/cpython that referenced this pull request Jan 20, 2025
@graingert @kumaraditya303
pythongh-128588: pythongh-128550: remove eager tasks optimization tha…
59aff22 
…t missed and introduced incorrect cancellations (python#129063)

Co-authored-by: Kumar Aditya <[email protected]>
@itamaro

itamaro commented Jan 20, 2025

Copy link
Copy Markdown
Contributor

What's the impact of removing this optimization on the async_tree pyperformance benchmarks?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@kumaraditya303

kumaraditya303 commented Jan 21, 2025

Copy link
Copy Markdown
Contributor

What's the impact of removing this optimization on the async_tree pyperformance benchmarks?What's the impact of removing this optimization on the async_tree pyperformance benchmarks?

I haven't benchmarked yet, I'll do it but I expect an insignificant impact.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
kumaraditya303 added a commit that referenced this pull request Jan 21, 2025
@graingert @kumaraditya303
[3.12] gh-128588: fix refcycles in eager task creation and remove eag…
bc21454 
…er tasks optimization that missed and introduced incorrect cancellations (#129063) (#128586)

Co-authored-by: Kumar Aditya <[email protected]>
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this pull request Jan 21, 2025
@graingert @kumaraditya303 @srinivasreddy
pythongh-128588: pythongh-128550: remove eager tasks optimization tha…
c765847 
…t missed and introduced incorrect cancellations (python#129063)

Co-authored-by: Kumar Aditya <[email protected]>
@itamaro

itamaro commented Feb 2, 2025

Copy link
Copy Markdown
Contributor

I haven't benchmarked yet, I'll do it but I expect an insignificant impact.

any update on benchmarking results @kumaraditya303 ?

benchmarking of the original PR showed up to 4x speedup on some of the async benchmarks. anything changed that you expect removing the optimization would have insignificant impact?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@kumaraditya303

kumaraditya303 commented Feb 3, 2025

Copy link
Copy Markdown
Contributor

any update on benchmarking results kumaraditya303 ?
benchmarking of #104140 showed up to 4x speedup on some of the async benchmarks. anything changed that you expect removing the optimization would have insignificant impact?

I had asked Michael to benchmark this last week but I haven't heard back from them, I'll ping them again but meanwhile I compared the rough number by comparing benchmarks with and without this patch and I don't see any large slowdown on the benchmarks:

With this PR:

image

Without this PR:
image
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@kumaraditya303

kumaraditya303 commented Feb 4, 2025

Copy link
Copy Markdown
Contributor

The results are now at https://github.com/faster-cpython/benchmarking-public/blob/main/results/bm-20250120-3.14.0a4%2B-ed6934e/bm-20250120-linux-x86_64-python-ed6934e71e55d398df82-3.14.0a4%2B-ed6934e-vs-base.md#benchmarks-with-tag-asyncio and there's no slowdown.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@itamaro

itamaro commented Feb 9, 2025

Copy link
Copy Markdown
Contributor

Hmm, I think I see what's going on here.. Looking more closely at the original PR, it appears the benchmarking used a patched version of pyperformance to add an "eager" flavor to the async tree benchmarks. Since these flavors don't exist in the released version of pyperformance, we don't see them in the benchmark runs...
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@hugovk hugovk removed the needs backport to 3.12 only security fixes label Feb 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kumaraditya303 kumaraditya303 kumaraditya303 approved these changes
@1st1 1st1 Awaiting requested review from 1st1 1st1 is a code owner
@asvetlov asvetlov Awaiting requested review from asvetlov asvetlov is a code owner
@willingc willingc Awaiting requested review from willingc willingc is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@kumaraditya303 kumaraditya303

Labels

topic-asyncio type-bug An unexpected behavior, bug, or error

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@graingert @itamaro @kumaraditya303 @hugovk
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.