GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-81554: Add add_reader support to ProactorEventLoop by minrk · Pull Request #141834 · python/cpython

Conversation

@minrk

@minrk minrk commented Nov 21, 2025 •
edited by github-actions Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Runs select.select in a background thread when add_reader/writer are called. Thread is terminated when the event loop is destroyed.

Solves a major problem with the default event loop lacking these methods (see #81554 for use cases).

Imports SelectorThread implementation to asyncio._selector_thread from Tornado 6.5.2 and uses it in ProactorEventLoop, fixing a longstanding compatibility problem with the default Windows event loop. Feel free to review _selector_thread, but note that it currently has almost no modifications to the original file from tornado (only removing some type hints not supported by the standard library).

Since this is only run on demand, no thread will be spawned for existing code that works on ProactorEventLoop, only new code that previously only worked on SelectorEventLoop or tornado's AddThreadEventLoop will now work with the default event loop.

It is possible to do this without a thread (trio does it), but it seems this is the simplest, most maintainable solution, as indicated in the discussion in #81554.

  • Issue: Configuration of windows event loop for libraries #81554

📚 Documentation preview 📚: https://cpython-previews--141834.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
minrk added 4 commits November 21, 2025 13:46
@minrk
import tornado selector thread
7cfacc4
v6.5.1
License: Apache-2.0
Copyright (c) 2025 The Tornado Authors

unmodified from original, so we can track changes
@minrk
remove some unused typing from selector thread
dabc739
@minrk
Add ProactorEventLoop.add_reader and friends
cc0e55a
via background SelectorThread, imported from tornado
@minrk
update spdx comments for tornado 6.5.2
dacf0bb
@minrk
minrk requested review from 1st1, asvetlov, kumaraditya303 and willingc as code owners November 21, 2025 23:11
@bedevere-app bedevere-app Bot added the awaiting review label Nov 21, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Nov 21, 2025
Configuration of windows event loop for libraries #81554
Open
StanFromIreland
StanFromIreland reviewed Nov 22, 2025
View reviewed changes
Comment thread Doc/library/asyncio-eventloop.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/library/asyncio-platforms.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/library/asyncio-platforms.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/_selector_thread.py
Comment on lines +2 to +3
# SPDX-License-Identifier: PSF-2.0 AND Apache-2.0
# SPDX-FileCopyrightText: Copyright (c) 2025 The Tornado Authors

@StanFromIreland StanFromIreland Nov 22, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this not be in https://github.com/python/cpython/blob/main/Misc/sbom.spdx.json?

See the devguide.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@minrk minrk Nov 24, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not perfectly as-is, since it is adapted and an excerpt, like Lib/asyncio/events.py and a number of others. That doesn't seem to fit the SBOM requirements, in my understanding? I don't see any of the other similarly adapted files in sbom.spdx.json.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@minrk minrk Nov 24, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After addressing review, it has diverged more substantially. This comment structure seems to be how adapted/partial content is handled. Is there anything in the dev guide for that? I couldn't find it. The SBOM page doesn't seem to apply or mention this sort of case.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@minrk @StanFromIreland
Apply suggestions from code review
a720ba0
Co-authored-by: Stan Ulbrych <[email protected]>
vstinner
vstinner reviewed Nov 24, 2025
View reviewed changes
Comment thread Lib/asyncio/_selector_thread.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/_selector_thread.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/_selector_thread.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/_selector_thread.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/_selector_thread.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/_selector_thread.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/_selector_thread.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/_selector_thread.py Outdated
# introduction of a new hook: https://bugs.python.org/issue41962)
self._thread = threading.Thread(
name="Tornado selector",
daemon=True,

@vstinner vstinner Nov 24, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to avoid daemon thread?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@minrk minrk Nov 24, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#86128 discusses why this is a daemon thread. I believe we need to call

with loop._select_cond:
            loop._closing_selector = True
            loop._select_cond.notify()
        try:
            loop._waker_w.send(b"a")
        except BlockingIOError:
            pass

prior to thread.join() for it to wake and exit properly. It looks like I can make it a non-daemon thread if I register the atexit hook with the private threading._register_atexit.

I believe this is only needed to handle process teardown for unclosed event loops. If we can strictly guarantee that EventLoop.close() is called before non-daemon threads are joined, then I don't think this is an issue. But I don't personally know a reliable way other than threading._register_atexit.

atexit.register only runs after threads are joined, not non-daemon threads (hence #86128), meaning that switching the thread to non-daemon will hang process exit until select returns, because no wake is called. A public hook that does exactly what threading._register_atexit does would be great. But since this is now in the stdlib, I could use that here, I suppose (#86128 remains a problem for non-stdlib code, of course).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/asyncio/_selector_thread.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/proactor_events.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

minrk added 5 commits November 24, 2025 08:54
@minrk
check and split fd-like to int FD in proactor add_reader
5135dab
matches tornado behavior

tornado handles this in IOLoop, so we need to include it
@minrk
Address revieww
a5fa125
- remove some older Python compatibility, since this is in the stdlib now
- resolve race between selector thread call soon and EventLoop.close
@minrk
asyncio._selector_thread: avoid daemon thread
8c76be9
rely on private threading._register_atexit to run cleanup prior to thread join
@minrk
add some docstrings to SelectorThread methods
7996d64
based on confusion in feedback
@minrk
use versionchanged:: next
c12d733
according to devguide
vstinner
vstinner reviewed Nov 25, 2025
View reviewed changes
Comment thread Doc/library/asyncio-platforms.rst
When called, :func:`select.select` is run in an additional thread.

The resolution of the monotonic clock on Windows is usually around 15.6
milliseconds. The best resolution is 0.5 milliseconds. The resolution depends on the

@vstinner vstinner Nov 25, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"The resolution of the monotonic clock on Windows is usually around 15.6 milliseconds."

Note unrelated to this PR: that's no longer true in Python 3.13:

On Windows, monotonic() now uses the QueryPerformanceCounter() clock for a resolution of 1 microsecond, instead of the GetTickCount64() clock which has a resolution of 15.6 milliseconds.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

minrk reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread Lib/asyncio/_selector_thread.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/_selector_thread.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

minrk added 2 commits November 28, 2025 17:26
@minrk
use Handles to ensure selector thread don't raise and halt handling
502bfaf
@minrk
more robust selector thread test
eb8739e
for thread asynciness
vstinner
vstinner reviewed Dec 1, 2025
View reviewed changes
Comment thread Lib/test/test_asyncio/test_windows_events.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@minrk
use assert methods
0b017d0
vstinner
vstinner reviewed Dec 2, 2025
View reviewed changes
Comment thread Lib/test/test_asyncio/test_selector_thread.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_asyncio/test_selector_thread.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_asyncio/test_selector_thread.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_asyncio/test_windows_events.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/_selector_thread.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

minrk and others added 4 commits December 5, 2025 09:32
@minrk @vstinner
Apply suggestions from code review
77c304f
Co-authored-by: Victor Stinner <[email protected]>
@minrk
from test import support
f75f7c5
@minrk @vstinner
apply suggestions from review
75c328b
Co-authored-by: Victor Stinner <[email protected]>
@minrk
copy event loop policy teardown from other asyncio tests
567f751
vstinner
vstinner approved these changes Dec 5, 2025
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, the change looks good to me. I didn't test it. I just trust tests and the code that I read :-) I would prefer to have a second review from another core dev.

@kumaraditya303: Would you be interested to review this change?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/test/test_asyncio/test_selector_thread.py
with mock.patch("select.select", wraps=select.select) as mock_select:
self.selector_thread.add_reader(b, mock_recv)
# ready event, but main event loop is blocked for some time
time.sleep(0.1)

@vstinner vstinner Dec 5, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this sleep really needed? If yes, can it be replaced with a synchronization primitive instead of a sleep?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@minrk minrk Dec 6, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would love help writing tests to verify that things don't happen in multithreaded code.

This was added to demonstrate an answer to the question in review that the select loop doesn't keep firing away if the main loop is slow to process one event, and was the best I could come up with to demonstrate that it really is waiting and only calls select once.

The sequence of events that we want to wait for is that the select thread is waiting in select_cond.wait() after select is called the first time. Maybe there's a way to instrument that.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot removed the awaiting review label Dec 5, 2025
@bedevere-app bedevere-app Bot added the awaiting merge label Dec 5, 2025
@kumaraditya303 kumaraditya303 self-assigned this Dec 16, 2025
@fleming79 fleming79 mentioned this pull request Apr 1, 2026
BUG: Advice around avoiding proactor event loop warning raises deprecation warning with Python 3.14+ zeromq/pyzmq#2137
Open
1 task done
@github-actions

github-actions Bot commented May 2, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label May 2, 2026
@stonebig

stonebig commented Aug 10, 2026

Copy link
Copy Markdown

will it be a problem if it's not in python-3.15 final ?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@minrk
Merge branch 'main' into add_reader
9f0434e
@read-the-docs-community

read-the-docs-community Bot commented Aug 13, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Documentation build overview

📚 cpython-previews | 🛠️ Build #34056476 | 📁 Comparing df1db09 against main (b0ba60e)

  🔍 Preview build  

3 files changed
± library/asyncio-eventloop.html
± library/asyncio-platforms.html
± whatsnew/changelog.html
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

vstinner
vstinner reviewed Aug 13, 2026
View reviewed changes
Comment thread Lib/test/test_asyncio/test_selector_thread.py Outdated


def tearDownModule():
asyncio.events._set_event_loop_policy(None)

@vstinner vstinner Aug 13, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like asyncio.events._set_event_loop_policy() has been removed.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

minrk reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@minrk minrk Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, updated and fixed lint

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
minrk added 2 commits August 13, 2026 12:56
@minrk
remove cleanup of unused set_event_loop_policy
1936d6b
@minrk
fix lint
df1db09
@minrk

minrk commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

will it be a problem if it's not in python-3.15 final ?

Yes, it will continue to be a problem, but not a particularly new one (at least since 3.8 made proactor default). Though the typical workaround of using set_event_loop_policy has been removed, making the problem a bit worse.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot removed the stale Stale PR or inactive for long period of time. label Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner approved these changes
@StanFromIreland StanFromIreland StanFromIreland left review comments
@1st1 1st1 Awaiting requested review from 1st1 1st1 is a code owner
@asvetlov asvetlov Awaiting requested review from asvetlov asvetlov is a code owner
@kumaraditya303 kumaraditya303 Awaiting requested review from kumaraditya303 kumaraditya303 is a code owner
@willingc willingc Awaiting requested review from willingc willingc is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@kumaraditya303 kumaraditya303

Labels

awaiting merge

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@minrk @stonebig @vstinner @StanFromIreland @kumaraditya303
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.