GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Fix `call_soon_threadsafe()` racing with `close()` on free-threaded Python by maartenbreddels · Pull Request #789 · MagicStack/uvloop

Conversation

@maartenbreddels

@maartenbreddels maartenbreddels commented Nov 8, 2026

Copy link
Copy Markdown

Fixes #788.

On a free-threaded build, call_soon_threadsafe() can run while another thread closes the loop.
The poster can pass the _closed check, and then _close() sets handler_async to None.
The poster then calls send() on None and the process segfaults.
The same race also makes close() fail with deque mutated during iteration or new callbacks were queued during loop closing.
With the GIL, the method runs as one step, so this did not happen.
This is a different race than #720.

The fix

The loop gets a cython.pymutex (_close_lock).
call_soon_threadsafe() holds it while it reads _closed, appends to _ready, and calls handler_async.send().
_close() holds it only while it checks and sets _closed.
After that, no poster touches _ready or handler_async again, so the rest of _close() runs without the lock.

The lock is not reentrant, so no code that can allocate Python objects runs under it.
An allocation can start a GC, and a finalizer can call call_soon_threadsafe() on the same loop.
For that reason the poster raises RuntimeError('Event loop is closed') only after it releases the lock.
For the same reason, _close() cancels the handles in _ready after it releases the lock.

I first tried with cython.critical_section(self).
It fixed the segfault, but close() still failed 1 to 3 times in 20 seconds.
The reason: the deque append can block on the deque's own lock, and a blocked thread suspends its critical sections.
The closing thread can then run while the poster is between the check and the append.
A PyMutex stays held while the thread blocks, so it does not have this gap.

call_soon_threadsafe() is the only way other threads post callbacks to the loop.
Other threads may also call is_closed(), which only reads a flag.
Signal handling runs on the loop thread.
The async generator finalizer hook can run on any thread, but it also posts through call_soon_threadsafe().
shutdown_default_executor() and set_debug() post through call_soon_threadsafe(), so they get the fix too.

Tests

New test: test_call_soon_threadsafe_while_closing (UV and AIO).
Two threads create, run, and close loops while four threads post to them, for 1 second.
The only error a poster may get is RuntimeError('Event loop is closed'); any other error fails the test.
The threads are daemon threads and the join has a timeout, so a deadlock fails the test instead of hanging the suite.
On master it fails 3 of 3 runs on 3.14t (close() errors).
With the fix it passes.

I ran these tests on 3.14t, 3.14, and 3.9, all pass:
test_call_soon_threadsafe_while_closing, test_call_soon_threadsafe_safety, test_freethreading, test_close, for TestBaseUV and TestBaseAIO.
test_freethreading skips on the GIL builds, as expected.

The standalone repro from the issue, on 3.14t: three 20-second runs and one 60-second run, 0 crashes and 0 close() failures.
The 60-second run made 118 million posts.
Before the fix it crashes in under a second.

A GC re-entry check (a __del__ on a cycle that posts to a closed loop, with a low GC threshold) does not deadlock on 3.9, 3.14, and 3.14t.

Cost, on an M-series Mac:

  • One thread: about 141 to 145 ns per call_soon_threadsafe(), with and without the fix, on 3.14 and 3.14t.
  • 16 threads posting to one running loop on 3.14t: 11.8 to 12.9 million posts in 10 seconds, with and without the fix.

Disclosure: the analysis, the fix, and the tests were written with Claude Code, and reviewed by three AI reviewers. The crash comes from our own production-like load tests.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@maartenbreddels @claude
Fix call_soon_threadsafe() racing with close() on free-threaded Python
886b562
call_soon_threadsafe() checks _closed, appends to _ready and then calls
handler_async.send().  On a free-threaded build another thread can run
close() in between: it closes handler_async and sets it to None, so the
poster segfaults in send().  The same race also makes close() fail with
"deque mutated during iteration" or "new callbacks were queued during
loop closing".  With the GIL the method runs as one step, so this did
not happen.

A critical section on the loop is not enough: the deque append can block
on the deque's own lock, and that suspends the outer critical section.
A PyMutex stays held while the thread blocks.

The mutex is not reentrant, so nothing that can start a GC runs under
it: a finalizer could post to the same loop and deadlock.  That is why
the closed-loop error is raised after the lock is released.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@maartenbreddels
maartenbreddels marked this pull request as ready for review November 8, 2026 12:04
Vizonex
Vizonex approved these changes Nov 8, 2026
View reviewed changes

@Vizonex Vizonex left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me as to why this would be in place. I'll have to think about implementing this on winloop's end if needed. There seems to be a workflow broken but it's not the library's fault so I approve of this PR.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@maartenbreddels maartenbreddels mentioned this pull request Nov 9, 2026
CI: run the Python 3.9 job on ubuntu-24.04 #790
Open
@maartenbreddels

maartenbreddels commented Nov 9, 2026

Copy link
Copy Markdown
Author

Thanks for the review!

The red job is test (3.9, ubuntu-latest). GitHub moved ubuntu-latest to Ubuntu 26.04, which has no Python 3.9 in setup-python. I opened #790 to run that job on ubuntu-24.04.

We also tested the fix on Linux, in our app's 3.14t image:

  • The repro from Segfault in call_soon_threadsafe() when another thread closes the loop (free-threaded Python) #788, 60 s per run: stock 0.23.0 crashed in 296 of 296 runs, the patched build in 0 of 3.
  • Our app under load: page load time and throughput are the same with and without the fix, on 3.14t and on 3.14.
Vizonex reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Vizonex

Vizonex commented Nov 10, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

@maartenbreddels In the meantime I've added this bugfix on winloop's side Vizonex/Winloop#189

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Vizonex Vizonex mentioned this pull request Nov 10, 2026
Fix call_soon_threadsafe() racing with close() on free-threaded Python Vizonex/Winloop#189
Merged
3 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@Vizonex Vizonex Vizonex approved these changes
Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Segfault in call_soon_threadsafe() when another thread closes the loop (free-threaded Python)

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@maartenbreddels @Vizonex
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.