GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-128002: use `_PyObject_SetMaybeWeakref` when creating tasks in asyncio by kumaraditya303 · Pull Request #128885 · python/cpython

Conversation

@kumaraditya303

@kumaraditya303 kumaraditya303 commented Jan 15, 2025 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor
  • Issue: Audit asyncio thread safety  #128002

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@kumaraditya303
use _PyObject_SetMaybeWeakref
8a1c13d
@kumaraditya303 kumaraditya303 added topic-asyncio skip news topic-free-threading labels Jan 15, 2025
@kumaraditya303
kumaraditya303 requested review from 1st1, asvetlov and willingc as code owners January 15, 2025 15:32
@bedevere-app bedevere-app Bot mentioned this pull request Jan 15, 2025
Audit asyncio thread safety #128002
Closed
9 tasks done
@bedevere-app bedevere-app Bot added the awaiting core review label Jan 15, 2025
@kumaraditya303
kumaraditya303 requested a review from colesbury January 15, 2025 15:33
@kumaraditya303
sort headers
51f092d
@colesbury

colesbury commented Jan 15, 2025

Copy link
Copy Markdown
Contributor

Can you add a test for this? I think something like the following would catch the bug:

  1. Create some tasks in the main thread
  2. Call all_tasks() in a different thread
  3. Check that you get all the tasks
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303

kumaraditya303 commented Jan 23, 2025

Copy link
Copy Markdown
Contributor Author

I think the following code appears to catch that bug:

import asyncio
import threading


def func():
    loop = asyncio.EventLoop()

    async def coro():
        await asyncio.sleep(0.01)

    lock = threading.Lock()
    count = 0
    async def main():
        nonlocal count
        while True:
            with lock:
                asyncio.create_task(coro())
                count = len(asyncio.all_tasks(loop))

    th = threading.Thread(target=loop.run_until_complete, args=(main(),))
    th.start()

    def check():
        with lock:
            assert count == len(asyncio.all_tasks(loop))

    threads = [threading.Thread(target=check) for _ in range(10)]
    for t in threads:
        t.start()

    for t in threads:
        t.join()


func()

Without the fix, I see many assertion errors and with it there isn't any.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303
add test
3add9e4
@kumaraditya303

kumaraditya303 commented Jan 23, 2025

Copy link
Copy Markdown
Contributor Author

@colesbury Added test based on the above code.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303
Merge branch 'main' into maybe-weakref
1d3a37d
colesbury
colesbury approved these changes Jan 23, 2025
View reviewed changes
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Jan 23, 2025
@kumaraditya303
use a less strict test
ede0c49
@kumaraditya303

kumaraditya303 commented Jan 24, 2025

Copy link
Copy Markdown
Contributor Author

I have tweaked the test to be a little less strict when comparing, it still tests the bug.

Without fix:

❯ ./python -m test test_asyncio.test_free_threading -F
Using random seed: 2944425156
0:00:00 load avg: 0.61 Run tests sequentially in a single process
0:00:00 load avg: 0.61 [  1] test_asyncio.test_free_threading
Warning -- Uncaught thread exception: AssertionError
Exception in thread Thread-5 (check):
Warning -- Uncaught thread exception: AssertionError
Exception in thread Thread-2 (check):
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/realkumaraditya/cpython/Lib/threading.py", line 1054, in _bootstrap_inner
    self.run()
    ~~~~~~~~^^
  File "/home/realkumaraditya/cpython/Lib/threading.py", line 996, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/realkumaraditya/cpython/Lib/test/test_asyncio/test_free_threading.py", line 86, in check
    self.assertSetEqual(tasks & self.all_tasks(loop), tasks)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/realkumaraditya/cpython/Lib/unittest/case.py", line 1176, in assertSetEqual
    self.fail(self._formatMessage(msg, standardMsg))
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/realkumaraditya/cpython/Lib/unittest/case.py", line 741, in fail
    raise self.failureException(msg)
AssertionError: Items in the second set but not the first:
<Task pending name='Task-1' coro=<TestFreeThreading.test_all_tasks_different_thread.<locals>.main() running at /home/realkumaraditya/cpython/Lib/test/test_asyncio/test_free_threading.py:77> cb=[_run_until_complete_cb() at /home/realkumaraditya/cpython/Lib/asyncio/base_events.py:181]>
colesbury reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kumaraditya303
kumaraditya303 merged commit 8e0b360 into python:main Jan 24, 2025
@bedevere-app bedevere-app Bot removed the awaiting merge label Jan 24, 2025
@kumaraditya303
kumaraditya303 deleted the maybe-weakref branch January 24, 2025 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@colesbury colesbury colesbury approved these changes
@1st1 1st1 Awaiting requested review from 1st1 1st1 is a code owner
@asvetlov asvetlov Awaiting requested review from asvetlov asvetlov is a code owner
@willingc willingc Awaiting requested review from willingc willingc is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news topic-asyncio topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@kumaraditya303 @colesbury
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.