GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-128384: Add thread-safe context manager to "warnings" module by nascheme · Pull Request #128300 · python/cpython

Conversation

@nascheme

@nascheme nascheme commented Dec 27, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This PR adds a new thread-local (and async friendly) context manager that uses contextvars. The recommended pattern for new code that doesn't care about backwards compatibility would be:

with warnings.local_context() as ctx:
    ctx.simplefilter(...)
    ...

For code that wants to be compatible with older versions of Python, the suggested code is:

import warnings
try:
    warn_ctx = warnings.get_context
except AttributeError:
   def warn_ctx():
       return warnings

with warn_ctx().catch_warnings():
    warn_ctx().simplefilter(...)
   ...

with warn_ctx().catch_warnings(record=True) as log:
    ...

This change retains warnings.filters and warnings.catch_warnings() as mechanisms that use process global state. Conceptually, there will be two sets of warning filters: the legacy process global warnings.filters list and the thread local get_context()._filters list. The context object returned by get_context() has an API that overlaps with the warnings module function API.

Perhaps we could eventually warn when the process global versions are used but I think that would be many years in the future and not worth considering now.

It would be intuitive if the thread local filtering was inherited when a new thread is created. I've created a separate PR that adds this feature to threading.Thread: gh-128209. That's a potentially controversial change but I think it's the correct thing to do. It will make contextvars behave similarly between asyncio tasks (which already inherit context) and threads. I think people will expect the warnings context manager to have lexical scope. If you see the code:

with warnings.local_context() as ctx:
    ctx.simplefilter(MyWarning)
    inner_function()

Then you would assume that inner_function() is going to have MyWarning filtered, even if it internally spawns a thread.

Regarding backwards compatibility, I did a code search and there are quite a few examples for code that manipulates warnings.filters directly, either mutating it or assigning a different list to the module global. There is also code that does this:

warnings.simplefilter(...)
...
warnings.filters.pop(0)

📚 Documentation preview 📚: https://cpython-previews--128300.org.readthedocs.build/

  • Issue: warnings.catch_warnings is not thread safe or async safe #128384

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@nascheme
nascheme force-pushed the warnings_local_context branch from 1f8b441 to 3952bb0 Compare December 28, 2024 21:07
@nascheme nascheme changed the title Add thread-safe context manager to "warnings" module gh-128384: Add thread-safe context manager to "warnings" module Dec 31, 2024
@bedevere-app bedevere-app Bot mentioned this pull request Dec 31, 2024
warnings.catch_warnings is not thread safe or async safe #128384
Closed
nascheme added 4 commits December 31, 2024 15:24
@nascheme
Add locking to warnings.py.
deb659d
Expose the mutex from _warnings.c and hold it when mutating the filters
list.
@nascheme
Add 'context' parameter to Thread.
902df7c
By default, inherit the context from the thread calling
`Thread.start()`.
@nascheme
Make _contextvars a builtin module.
e61b40e
@nascheme
wip: warnings local_context
3bac8d1
@nascheme nascheme mentioned this pull request Dec 31, 2024
Switch to a new thread-safe utility for catching warnings. jax-ml/jax#25626
Merged
@nascheme
nascheme force-pushed the warnings_local_context branch from 3952bb0 to 3bac8d1 Compare January 1, 2025 05:08
nascheme added 5 commits January 1, 2025 11:47
@nascheme
Rename function, add locking to _filters_mutated().
c7287ad
@nascheme
Use re-entrant lock for warnings module.
1521276
@nascheme
Make _warnings use _warnings_context.
e5c128a
@nascheme
Add unit tests for warnings.Context
948a9e9
@nascheme
Improve Context.resetwarnings().
fc20826
When reset is called, the warnings.filters list should be ignored since
the list of active filters should be empty at that point.
@hawkinsp hawkinsp mentioned this pull request Jan 9, 2025
BUG: move reduction initialization to ufunc initialization numpy/numpy#28123
Merged
@nascheme

nascheme commented Feb 11, 2025

Copy link
Copy Markdown
Member Author

Closing this since gh-130010 seems a better approach.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nascheme nascheme closed this Feb 11, 2025
gst
gst reviewed Apr 7, 2025
View reviewed changes
Comment thread Lib/test/test_warnings/__init__.py


class PyContextTests(ContextTests, unittest.TestCase):
module = c_warnings

@gst gst Apr 7, 2025

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't you want the other one ?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@nascheme nascheme Apr 7, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, you're right.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@1st1 1st1 Awaiting requested review from 1st1 1st1 will be requested when the pull request is marked ready for review 1st1 is a code owner
@erlend-aasland erlend-aasland Awaiting requested review from erlend-aasland erlend-aasland will be requested when the pull request is marked ready for review erlend-aasland is a code owner
@AA-Turner AA-Turner Awaiting requested review from AA-Turner AA-Turner will be requested when the pull request is marked ready for review AA-Turner is a code owner
@emmatyping emmatyping Awaiting requested review from emmatyping emmatyping will be requested when the pull request is marked ready for review emmatyping is a code owner
@itamaro itamaro Awaiting requested review from itamaro itamaro will be requested when the pull request is marked ready for review itamaro is a code owner
@corona10 corona10 Awaiting requested review from corona10 corona10 will be requested when the pull request is marked ready for review corona10 is a code owner
1 more reviewer
@gst gst gst left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@nascheme @gst
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.