Description

colesbury

opened on Jun 24, 2024

Issue body actions

Bug report

1. fi_freelist isn't thread-safe (move to pycore_freelist.h and follow that pattern)
2. enter_task, leave_task, and swap_current_task aren't thread-safe due to shared state->current_tasks and borrowed references.
3. register_task and unregister_task aren't thread-safe due to shared state->asyncio_tasks linked list
4. _asyncio_all_tasks_impl isn't thread-safe due the the asyncio_tasks linked list.

For 2, 3, and 4, we can consider using critical sections to protect the accesses to state->current_tasks and state->asyncio_tasks.

Longer term, moving data to per-loop will probably help with multi-threaded scaling.

Linked PRs

• gh-120974: Use common freelist code in asyncio #122132
• gh-120974: Make _asyncio._enter_task atomic in the free-threaded build #122138
• gh-120974: Make _asyncio._leave_task atomic in the free-threaded build #122139
• [3.13] gh-120974: Make _asyncio._enter_task atomic in the free-threaded build (GH-122138) #122152
• [3.13] gh-120974: Make _asyncio._leave_task atomic in the free-threaded build (GH-122139) #122186
• gh-120974: Make asyncio swap_current_task safe in free-threaded build #122317
• [3.13] gh-120974: Make asyncio swap_current_task safe in free-threaded build (GH-122317) #122612
• GH-120974: make _asyncio_all_tasks_impl thread safe #122801

Activity

1. colesbury

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   topic-asyncio

   3.13only security fixesonly security fixes

   topic-free-threading

   3.14bugs and security fixesbugs and security fixes

   on Jun 24, 2024
2. github-project-automation

   added this to asyncioon Jun 24, 2024
3. github-project-automation

   moved this to Todo in asyncioon Jun 24, 2024
4. kumaraditya303 commented on Jun 25, 2024

   

   kumaraditya303

   on Jun 25, 2024

   Contributor

   More actions

   I think that the pure Python fallback isn't thread safe either, FWIW I think there are more thread safety issues than these, asyncio code was written with assumption of GIL.
5. corona10 commented on Jun 25, 2024

   

   corona10

   on Jun 25, 2024

   Member

   More actions

   asyncio code was written with assumption of GIL.

   Yeah, but for landing PEP 703, we are trying to change modules, not depending on the GIL, even if we need to re-write some parts of the implementation with a conditional flag. (If the new written part requires some performance overhead)
6. colesbury

   added 3 commits that reference this issue on Jul 22, 2024

   pythongh-120974: Use common freelist code in asyncio

   21cdcd2

   pythongh-120974: Make _asyncio._enter_task atomic in the free-threade…

   2f638e9

   pythongh-120974: Make _asyncio._leave_task atomic in the free-threade…

   8cc6498
7. kumaraditya303

   added a commit that references this issue on Jul 23, 2024

   gh-120974: Make _asyncio._enter_task atomic in the free-threaded build (

   47847aa
8. miss-islington

   added a commit that references this issue on Jul 23, 2024

   pythongh-120974: Make _asyncio._enter_task atomic in the free-threade…

   8faab2e
9. kumaraditya303

   added a commit that references this issue on Jul 23, 2024

   [3.13] gh-120974: Make _asyncio._enter_task atomic in the free-thread…

   48154e7
10. 6 remaining items
11. nohlson

    added 4 commits that reference this issue on Jul 24, 2024

    pythongh-120974: Make _asyncio._leave_task atomic in the free-threade…

    3361407

    pythongh-120974: Make _asyncio._enter_task atomic in the free-threade…

    6b74fbe

    pythongh-120974: Use common freelist code in asyncio (python#122132)

    72ead6a

    pythongh-120974: Make _asyncio._leave_task atomic in the free-threade…

    7e66952
12. colesbury

    added 2 commits that reference this issue on Jul 26, 2024

    pythongh-120974: Make asyncio `swap_current_task` safe in free-thread…

    317a7e2

    pythongh-120974: Make asyncio `swap_current_task` safe in free-thread…

    846f6f8
13. kumaraditya303

    added a commit that references this issue on Aug 2, 2024

    gh-120974: Make asyncio `swap_current_task` safe in free-threaded bui…

    b5e6fb3
14. miss-islington

    added a commit that references this issue on Aug 2, 2024

    pythongh-120974: Make asyncio `swap_current_task` safe in free-thread…

    6a87edc
15. kumaraditya303

    added a commit that references this issue on Aug 2, 2024

    [3.13] gh-120974: Make asyncio `swap_current_task` safe in free-threa…

    c1efeb3
16. brandtbucher

    added a commit that references this issue on Aug 7, 2024

    pythongh-120974: Make asyncio `swap_current_task` safe in free-thread…

    475aeda
17. kumaraditya303

    added a commit that references this issue on Aug 11, 2024

    GH-120974: make `_asyncio.all_tasks` thread safe (#122801)

    3aaed08
18. kumaraditya303

    closed this as completedon Aug 11, 2024
19. github-project-automation

    moved this from Todo to Done in asyncioon Aug 11, 2024
20. blhsing

    added 2 commits that reference this issue on Aug 22, 2024

    pythongh-120974: Make asyncio `swap_current_task` safe in free-thread…

    d3909a4

    pythonGH-120974: make `_asyncio.all_tasks` thread safe (python#122801)

    b1eeb6a

Issue actions