GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-128002: fix many thread safety issues in asyncio by kumaraditya303 · Pull Request #128147 · python/cpython

Conversation

@kumaraditya303

@kumaraditya303 kumaraditya303 commented Dec 21, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Changes:

  • Makes _asyncio.Task and _asyncio.Future thread-safe by adding critical sections
  • Add assertions to check for thread safety checking locking of object by critical sections in internal functions
  • Make _asyncio.all_tasks thread safe when eager tasks are used
  • Change asyncio state lock to a mutex rather than critical section to fix re-entrancy crash, not really sure about this one
  • Add a thread safety test
  • Issue: Audit asyncio thread safety  #128002

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
kumaraditya303 added 3 commits December 21, 2024 08:29
@kumaraditya303
fix thread safety
f3642ae
@kumaraditya303
keep working
1f63b9e
@kumaraditya303
keep working
26e9932
@kumaraditya303
kumaraditya303 requested review from 1st1, asvetlov and willingc as code owners December 21, 2024 09:41
@bedevere-app bedevere-app Bot mentioned this pull request Dec 21, 2024
Audit asyncio thread safety #128002
Closed
9 tasks done
@bedevere-app bedevere-app Bot added the awaiting core review label Dec 21, 2024
@kumaraditya303 kumaraditya303 added topic-asyncio skip news labels Dec 21, 2024
kumaraditya303 added 4 commits December 21, 2024 09:44
@kumaraditya303
fix refcounting
4381ae2
@kumaraditya303
fix refcounting
b972121
@kumaraditya303
add asserts
eeb0273
@kumaraditya303
more asserts
fd1b9cd
@kumaraditya303
kumaraditya303 requested a review from colesbury December 23, 2024 08:16
kumaraditya303 added 8 commits December 23, 2024 08:28
@kumaraditya303
more asserts
b628463
@kumaraditya303
Merge branch 'main' of https://github.com/python/cpython into asyncio…
501b6bb 
…-tsafe
@kumaraditya303
fix test
86448af
@kumaraditya303
Merge branch 'main' of https://github.com/python/cpython into asyncio…
303bd9a 
…-tsafe
@kumaraditya303
fix crash by using mutex instead of critical_section
0e4a775
@kumaraditya303
fix all_tasks and use critical section as before
1cb3a6f
@kumaraditya303
add supressions
195c87d
@kumaraditya303
Merge branch 'main' of https://github.com/python/cpython into asyncio…
49f3393 
…-tsafe
@kumaraditya303

kumaraditya303 commented Dec 31, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

@colesbury While running the tests locally on forever mode I see frequent tsan warnings and rarely some crashes in exceptions.c arising from things like setting traceback on a exception, should I add those to the suppressions list?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
kumaraditya303 added 2 commits January 1, 2025 18:08
@kumaraditya303
remove suppressions for sockets, it is fixed now
847da18
@kumaraditya303
merge fixes for socket
41a86a6
colesbury
colesbury reviewed Jan 2, 2025
View reviewed changes

@colesbury colesbury left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still reading through the other locking changes, but a comment on all_tasks below:

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Modules/_asynciomodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Modules/_asynciomodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@kumaraditya303 kumaraditya303 mentioned this pull request Jan 2, 2025
data races in frame inspection and tracebacks #128421
Closed
@kumaraditya303
add link to issue in suppressions
2ff2af4
@kumaraditya303

kumaraditya303 commented Jan 2, 2025

Copy link
Copy Markdown
Contributor Author

Filed #128421 for existing data races in traceback and frame inspection.
colesbury reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
colesbury
colesbury reviewed Jan 2, 2025
View reviewed changes
Comment thread Tools/tsan/suppressions_free_threading.txt Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Modules/_asynciomodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Modules/_asynciomodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
kumaraditya303 added 2 commits January 2, 2025 16:59
@kumaraditya303
use list as temp storage for tasks before filtering
5411a44
@kumaraditya303
fix FutureIter_am_send_lock_held
1afac5f
kumaraditya303
kumaraditya303 commented Jan 2, 2025
View reviewed changes
Comment thread Modules/_asynciomodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
colesbury
colesbury reviewed Jan 2, 2025
View reviewed changes
Comment thread Modules/_asynciomodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
kumaraditya303 added 3 commits January 2, 2025 19:05
@kumaraditya303
make it->future immutable
606ef5c
@kumaraditya303
Merge branch 'main' of https://github.com/python/cpython into asyncio…
f2ea4c6 
…-tsafe
@kumaraditya303
remove unnecessary incref/decref of task
8bd0da7
@kumaraditya303
kumaraditya303 requested a review from colesbury January 3, 2025 15:28
colesbury
colesbury approved these changes Jan 3, 2025
View reviewed changes

@colesbury colesbury left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with two comments below

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Modules/_asynciomodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Tools/tsan/suppressions_free_threading.txt Resolved

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Jan 3, 2025
kumaraditya303 added 2 commits January 4, 2025 07:19
@kumaraditya303
remove suppressions
5030cc6
@kumaraditya303
fix comment
2676957
kumaraditya303
kumaraditya303 commented Jan 4, 2025
View reviewed changes
Comment thread Tools/tsan/suppressions_free_threading.txt Resolved

Uh oh!

There was an error while loading. Please reload this page.
@kumaraditya303
Update Tools/tsan/suppressions_free_threading.txt
6e60222
@kumaraditya303
kumaraditya303 merged commit 513a4ef into python:main Jan 4, 2025
@bedevere-app bedevere-app Bot removed the awaiting merge label Jan 4, 2025
@kumaraditya303
kumaraditya303 deleted the asyncio-tsafe branch January 4, 2025 08:48
WolframAlph pushed a commit to WolframAlph/cpython that referenced this pull request Jan 4, 2025
@kumaraditya303 @WolframAlph
pythongh-128002: fix many thread safety issues in asyncio (python#128147
bdfe2ca 
)

* Makes `_asyncio.Task` and `_asyncio.Future` thread-safe by adding critical sections
* Add assertions to check for thread safety checking locking of object by critical sections in internal functions
* Make `_asyncio.all_tasks` thread safe when eager tasks are used
* Add a thread safety test
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this pull request Jan 8, 2025
@kumaraditya303 @srinivasreddy
pythongh-128002: fix many thread safety issues in asyncio (python#128147
4a1dabe 
)

* Makes `_asyncio.Task` and `_asyncio.Future` thread-safe by adding critical sections
* Add assertions to check for thread safety checking locking of object by critical sections in internal functions
* Make `_asyncio.all_tasks` thread safe when eager tasks are used
* Add a thread safety test
@kumaraditya303 kumaraditya303 added the topic-free-threading label Jun 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@colesbury colesbury colesbury approved these changes
@1st1 1st1 Awaiting requested review from 1st1 1st1 is a code owner
@asvetlov asvetlov Awaiting requested review from asvetlov asvetlov is a code owner
@willingc willingc Awaiting requested review from willingc willingc is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news topic-asyncio topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@kumaraditya303 @colesbury
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.