GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-105836: Fix asyncio.run_coroutine_threadsafe leaving underlying cancelled asyncio task running by iaalm · Pull Request #141696 · python/cpython

Conversation

@iaalm

@iaalm iaalm commented Nov 18, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor
  • This issue caused by _chain_future which supports concurrent.futures.Future and asycio.Future. While asyncio.run_coroutine_threadsafe gives destination a concurrent.futures.Future, the dest_loop is None. Then dest_loop is source_loop is not good enough for the check. We should always the current running loop for the future operating.
  • In case of concurrent.futures.Future created by asyncio.run_coroutine_threadsafe, we need to wait for the cancellation before return. That's what I done in _cancel_future_in_loop.
  • Issue: asyncio.run_coroutine_threadsafe leaves underlying cancelled asyncio task running #105836

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@iaalm
Fix pythongh-105836
f69f706
@iaalm
iaalm requested review from 1st1, asvetlov, kumaraditya303 and willingc as code owners November 18, 2025 08:27
@python-cla-bot

python-cla-bot Bot commented Nov 18, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot added the awaiting review label Nov 18, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Nov 18, 2025
asyncio.run_coroutine_threadsafe leaves underlying cancelled asyncio task running #105836
Closed
@gvanrossum
gvanrossum self-requested a review November 25, 2025 00:55
gvanrossum
gvanrossum reviewed Nov 25, 2025
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

I tried this with the example from gh-105836. This indeed fixes it, but still requires the await asyncio.sleep(0) after the shutdown() call. (Also note that the example passes if there are two sleep(0) calls.)

Is there a way to do better?

Also, probably that example should be added as a unit test (preferably without long sleeps).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@iaalm
Add unit test
8fa2cac
@iaalm

iaalm commented Nov 25, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Hi @gvanrossum,

Added one (no very cool) test case. The new test failed on windows, checking..

Regarding the asyncio.sleep(0), the example in gh-105836 uses Thread.run() which actually run the function in main thread (maybe by typo?). In such cases, because future.cancel() is a sync function running in loop. There's no good way in my opinion to wait for the cancellation in future.cancel(). So one asyncio.sleep(0) is always needed to execute the cancellation work. I think that is also the reason all test in test_asyncio.test_tasks.RunCoroutineThreadsafeTests needs a asyncio.sleep(0) after cancel.
My code change is mainly avoiding another sleep to run the "future.cancel()" itself.

# emulates some worker thread (e.g. APScheduler)
    runner = threading.Thread(target=myapp.start_async_task, args=[thread_event])
    runner.run()

Does that make sense?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

iaalm added 2 commits November 25, 2025 16:56
@iaalm
Fix code style
42f3885
@iaalm
Revert test
5398219
@gvanrossum
gvanrossum marked this pull request as draft November 25, 2025 17:41
@bedevere-app bedevere-app Bot removed the awaiting review label Nov 25, 2025
@gvanrossum gvanrossum added needs backport to 3.13 only security fixes needs backport to 3.14 bugs and security fixes labels Nov 25, 2025
@gvanrossum gvanrossum added this to asyncio Nov 25, 2025
@github-project-automation github-project-automation Bot moved this to Todo in asyncio Nov 25, 2025
@gvanrossum gvanrossum moved this from Todo to In Progress in asyncio Nov 25, 2025
@iaalm
Improve test case
d7675d3
gvanrossum
gvanrossum requested changes Nov 26, 2025
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nearly there! Thanks for persevering.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/test/test_asyncio/test_tasks.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_asyncio/test_tasks.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot added the awaiting changes label Nov 26, 2025
@bedevere-app

bedevere-app Bot commented Nov 26, 2025

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

iaalm added 2 commits November 27, 2025 21:33
@iaalm
Run test in another thread
f430392
@iaalm
Use lock to make sure target started
670e875
@iaalm

iaalm commented Nov 27, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

will investigate why the case fail on windows tomorrow (need to find a windows device😝)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gvanrossum
gvanrossum requested changes Nov 27, 2025
View reviewed changes
Comment thread Lib/test/test_asyncio/test_tasks.py Outdated
import random
import re
import sys
from threading import Event

@gvanrossum gvanrossum Nov 27, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd move this import into the test function -- otherwise it could be confused with asyncio.Event.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/test/test_asyncio/test_tasks.py Outdated
# self.loop.run_in_executor(None, _in_another_thread)
thread_future = asyncio.run_coroutine_threadsafe(self.add(1, 2), self.loop)
await asyncio.sleep(0)
target_started = Event()

@gvanrossum gvanrossum Nov 27, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd use import threading followed by target_started = threading.Event() to be super clear about what kind of event this is.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/test/test_asyncio/test_tasks.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_asyncio/test_tasks.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@iaalm
Revert "Run test in another thread"
f061b1d
@iaalm
iaalm marked this pull request as ready for review December 5, 2025 16:23
@bedevere-app bedevere-app Bot added the awaiting review label Dec 5, 2025
gvanrossum
gvanrossum approved these changes Dec 6, 2025
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now the tests pass, I'm approving this.

I just want @kumaraditya303 to have a look as well.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

iaalm reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Dec 6, 2025
@kumaraditya303
improve test
c97e497
@kumaraditya303

kumaraditya303 commented Dec 6, 2025

Copy link
Copy Markdown
Contributor

I have improved the test to not rely on _ready and removed the windows hack, the test passes on this branch while fails on current main:

Running Debug|x64 interpreter...
== CPython 3.15.0a1+ (heads/cancel_in_loop:787c9fc5de0, Dec 6 2025, 20:45:00) [MSC v.1944 64 bit (AMD64)]
== Windows-11-10.0.26200-SP0 little-endian
== Python build: debug
== cwd: D:\cpython\build\test_python_worker_24356æ
== CPU count: 12
== encodings: locale=cp1252 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 1952231759
0:00:00 Run 1 test sequentially in a single process
0:00:00 [1/1] test_asyncio.test_tasks
test_run_coroutine_threadsafe_and_cancel (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_and_cancel) ... FAIL

======================================================================
FAIL: test_run_coroutine_threadsafe_and_cancel (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_and_cancel)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cpython\Lib\test\test_asyncio\test_tasks.py", line 3704, in test_run_coroutine_threadsafe_and_cancel
    self.assertTrue(task.cancelled())
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: False is not true

----------------------------------------------------------------------
Ran 1 test in 0.014s

FAILED (failures=1)
Task was destroyed but it is pending!
task: <Task cancelling name='Task-2' coro=<sleep() done, defined at D:\cpython\Lib\asyncio\tasks.py:687> wait_for=<Future cancelled> cb=[_chain_future.<locals>._call_set_state() at D:\cpython\Lib\asyncio\futures.py:397]>
test test_asyncio.test_tasks failed
0:00:00 [1/1/1] test_asyncio.test_tasks failed (1 failure)

== Tests result: FAILURE ==

1 test failed:
    test_asyncio.test_tasks

Total duration: 542 ms
Total tests: run=1 (filtered) failures=1
Total test files: run=1/1 (filtered) failed=1
Result: FAILURE
iaalm reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gvanrossum
gvanrossum approved these changes Dec 6, 2025
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kumaraditya303, that looks much better!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@kumaraditya303 kumaraditya303 added the topic-asyncio label Dec 6, 2025
kumaraditya303
kumaraditya303 reviewed Dec 6, 2025
View reviewed changes
Comment thread Misc/NEWS.d/next/Library/2025-11-18-15-48-13.gh-issue-105836.sbUw24.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303
Apply suggestion from @kumaraditya303
e3a4011
kumaraditya303
kumaraditya303 approved these changes Dec 6, 2025
View reviewed changes
@kumaraditya303
kumaraditya303 enabled auto-merge (squash) December 6, 2025 19:09
@kumaraditya303
kumaraditya303 merged commit 14715e3 into python:main Dec 6, 2025
46 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@github-project-automation github-project-automation Bot moved this from In Progress to Done in asyncio Dec 6, 2025
@miss-islington-app

miss-islington-app Bot commented Dec 6, 2025

Copy link
Copy Markdown

Thanks @iaalm for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting merge label Dec 6, 2025
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Dec 6, 2025
@iaalm @kumaraditya303 @miss-islington
pythongh-105836: Fix asyncio.run_coroutine_threadsafe leaving under…
1863bd2
…lying cancelled asyncio task running (pythonGH-141696)

(cherry picked from commit 14715e3)

Co-authored-by: Kaisheng Xu <[email protected]>
Co-authored-by: Kumar Aditya <[email protected]>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Dec 6, 2025
@iaalm @kumaraditya303 @miss-islington
pythongh-105836: Fix asyncio.run_coroutine_threadsafe leaving under…
8bb3a35
…lying cancelled asyncio task running (pythonGH-141696)

(cherry picked from commit 14715e3)

Co-authored-by: Kaisheng Xu <[email protected]>
Co-authored-by: Kumar Aditya <[email protected]>
@bedevere-app

bedevere-app Bot commented Dec 6, 2025

Copy link
Copy Markdown

GH-142358 is a backport of this pull request to the 3.14 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Dec 6, 2025
@bedevere-app

bedevere-app Bot commented Dec 6, 2025

Copy link
Copy Markdown

GH-142359 is a backport of this pull request to the 3.13 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 only security fixes label Dec 6, 2025
kumaraditya303 added a commit that referenced this pull request Dec 9, 2025
@miss-islington @iaalm @kumaraditya303
[3.13] gh-105836: Fix asyncio.run_coroutine_threadsafe leaving unde…
b2fff0e
…rlying cancelled asyncio task running (GH-141696) (#142359)

gh-105836: Fix `asyncio.run_coroutine_threadsafe` leaving underlying cancelled asyncio task running (GH-141696)
(cherry picked from commit 14715e3)

Co-authored-by: Kaisheng Xu <[email protected]>
Co-authored-by: Kumar Aditya <[email protected]>
kumaraditya303 added a commit that referenced this pull request Dec 9, 2025
@miss-islington @iaalm @kumaraditya303
[3.14] gh-105836: Fix asyncio.run_coroutine_threadsafe leaving unde…
5d5d92b
…rlying cancelled asyncio task running (GH-141696) (#142358)

gh-105836: Fix `asyncio.run_coroutine_threadsafe` leaving underlying cancelled asyncio task running (GH-141696)
(cherry picked from commit 14715e3)

Co-authored-by: Kaisheng Xu <[email protected]>
Co-authored-by: Kumar Aditya <[email protected]>
@sairon sairon mentioned this pull request Feb 5, 2026
call_soon_threadsafe callbacks from executor no longer run before await run_in_executor() returns in 3.14.3 #144506
Closed
@pentschev pentschev mentioned this pull request Feb 13, 2026
Fixes hangs when Python future notifier is enabled rapidsai/ucxx#590
Merged
@samypr100 samypr100 mentioned this pull request Apr 4, 2026
fix(tests): explicit yield in test_create_ssl_server_manual_connection_lost MagicStack/uvloop#743
Merged
@coderabbitai coderabbitai Bot mentioned this pull request Jul 14, 2026
feat(navigation): add operator-safe movement modes omarespejel/dimos#4
Closed
1 task done
@jordangarside jordangarside mentioned this pull request Sep 22, 2026
asyncio run_in_executor / to_thread results not delivered until next timer under monkey.patch_all() on CPython ≥ 3.13.12 / 3.14.3 gevent/gevent#2213
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gvanrossum gvanrossum gvanrossum approved these changes
@kumaraditya303 kumaraditya303 kumaraditya303 approved these changes
@1st1 1st1 Awaiting requested review from 1st1 1st1 is a code owner
@asvetlov asvetlov Awaiting requested review from asvetlov asvetlov is a code owner
@willingc willingc Awaiting requested review from willingc willingc is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

topic-asyncio

Projects

asyncio
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@iaalm @gvanrossum @kumaraditya303
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.