GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-120868: Fix breaking change in `logging.config` when using `QueueHandler` by provinzkraut · Pull Request #120872 · python/cpython

Conversation

@provinzkraut

@provinzkraut provinzkraut commented Jun 22, 2024 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

#120868 was caused by an eager construction of multiprocessing.Manager used to check if the queue passed in to the logging config was an instance of multiprocessing.Manager.Queue / multiprocessing.Manager.JoinableQueue. This could cause unexpected exceptions in environments where a multiprocessing.Manager couldn't be created.

The proposed fix moves the check for these classes, and creation of multiprocessing.Manager which is necessary for this, to a point where we've ruled out all other valid options for queue, and are certain that multiprocessing.Manager has been used to produce the object passed via queue.

  • Issue: 3.12.4 breakslogging.config.DictConfig with logging.handlers.QueueHandler on read-only file systems #120868

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
provinzkraut added 2 commits June 22, 2024 12:22
@provinzkraut
pythongh-120868: Fix regression in logging.config where multiprocessi…
0723f49
…ng.Manager was created eagerly
@provinzkraut
Add News fragment
afcc79e
@provinzkraut
provinzkraut requested a review from vsajip as a code owner June 22, 2024 10:30
@ghost

ghost commented Jun 22, 2024 •
edited by ghost
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.
CLA signed

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot mentioned this pull request Jun 22, 2024
3.12.4 breakslogging.config.DictConfig with logging.handlers.QueueHandler on read-only file systems #120868
Closed
@bedevere-app bedevere-app Bot added the awaiting review label Jun 22, 2024
picnixz
picnixz reviewed Jun 22, 2024
View reviewed changes
Comment thread Lib/logging/config.py Outdated
# if it's not an instance of BaseProxy, it also can't be
# an instance of Manager.Queue / Manager.JoinableQueue
if isinstance(qspec, BaseProxy):
proxy_queue = MM().Queue()

@picnixz picnixz Jun 22, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there is an error with the creation, you could maybe wrap that one in an try-except block? (probably catching an OSError but I'm not sure).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@provinzkraut provinzkraut Jun 22, 2024

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would we want to behave different in this case? I'm not sure what the expected behaviour would be here. The only thing I could think of that would make sense is to raise the usual TypeError from the error the instantiation raised, but that would still result in roughly the same traceback as when calling .configure, since it already re-raises exceptions.

Or are you suggesting we take a different path here when we encounter an error?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@picnixz picnixz Jun 22, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I didn't know about the exceptino being wrapped by configure so it may be fine. The thing is, I don't know whether we could be more precise in the error being raised so that you know that's is because multiprocessing queues are not supported.

For instance:

  • you use a bad BaseProxy object which is not a queue and MM().Queue() is fine -> you want the TypeError
  • let's say you have a queue-like object for your readonly system that inherits from MM().Queue() with some tweaks. In this case, you will be able to create your special queue object and expect it to work but MM().Queue() would fail. So instead of having some generic error due to that instantiation, the message could be something else. But I'm not sure whether this corner case is too convoluted or not.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@picnixz picnixz Jun 22, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, I'll just leave the decision to Vinay because they're the maintainer and creator of the logging module.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

provinzkraut reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@provinzkraut provinzkraut Jun 22, 2024

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I'm not sure whether this corner case is too convoluted or not.

Yeah, I was worrying about that. I didn't want to put too much stuff in here just to handle this case. IMO the proper proper fix for this would be to add some level of introspection capabilities in BaseProxy, that allows to perform such checks without creating an instance of the type you want to check against first.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@provinzkraut provinzkraut Jun 22, 2024

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it would make sense to add something in the docs for this, as there's quite a few special cases now that might not be immediately intuitive

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

picnixz reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread Lib/test/test_logging.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

provinzkraut and others added 2 commits June 22, 2024 13:17
@provinzkraut
Update Misc/ACKS to make patchcheck happy
08bc4d6
@provinzkraut
Merge branch 'python:main' into fix-pythongh-120868
a828c0d
@vsajip vsajip added the skip news label Jun 22, 2024
vsajip
vsajip requested changes Jun 22, 2024
View reviewed changes
Comment thread Misc/NEWS.d/next/Library/2024-06-22-11-59-13.gh-issue-120868.XHJ12L.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/logging/config.py
q = self.configure_custom(dict(qspec))
else:

if isinstance(qspec, str):

@vsajip vsajip Jun 22, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Flat is better than nested", so I agree that it's reasonable to rearrange the various tests on qspec in this way.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

provinzkraut reacted with heart emoji
All reactions
  • ❤️ 1 reaction
Comment thread Lib/logging/config.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting review label Jun 22, 2024
@bedevere-app

bedevere-app Bot commented Jun 22, 2024

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot added the awaiting changes label Jun 22, 2024
provinzkraut added 2 commits June 23, 2024 10:01
@provinzkraut
Revert "Add News fragment"
ccc0060
This reverts commit afcc79e.
@provinzkraut
Restructure as per review comment
6daf85e
@provinzkraut

provinzkraut commented Jun 23, 2024

Copy link
Copy Markdown
Contributor Author

Thanks for the review @vsajip!

I have made the requested changes; please review again

:)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot added awaiting change review and removed awaiting changes labels Jun 23, 2024
@bedevere-app

bedevere-app Bot commented Jun 23, 2024

Copy link
Copy Markdown

Thanks for making the requested changes!

@vsajip: please review the changes made to this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app
bedevere-app Bot requested a review from vsajip June 23, 2024 08:11
@vsajip vsajip added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jun 25, 2024
@bedevere-bot

bedevere-bot commented Jun 25, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @vsajip for commit 6daf85e 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jun 25, 2024
vsajip
vsajip approved these changes Jun 27, 2024
View reviewed changes

@vsajip vsajip left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some buildbot failures, but they appear unrelated to this change.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting change review labels Jun 27, 2024
@vsajip
vsajip merged commit 7d9c685 into python:main Jun 27, 2024
@bedevere-app bedevere-app Bot removed the awaiting merge label Jun 27, 2024
@vsajip vsajip added the needs backport to 3.12 only security fixes label Jun 27, 2024
@vsajip vsajip added the needs backport to 3.13 only security fixes label Jun 27, 2024
@miss-islington-app

miss-islington-app Bot commented Jun 27, 2024

Copy link
Copy Markdown

Thanks @provinzkraut for the PR, and @vsajip for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington-app

miss-islington-app Bot commented Jun 27, 2024

Copy link
Copy Markdown

Thanks @provinzkraut for the PR, and @vsajip for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 27, 2024
@provinzkraut @miss-islington
pythongh-120868: Fix breaking change in logging.config when using `…
a28b510
…QueueHandler` (pythonGH-120872)

(cherry picked from commit 7d9c685)

Co-authored-by: Janek Nouvertné <[email protected]>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 27, 2024
@provinzkraut @miss-islington
pythongh-120868: Fix breaking change in logging.config when using `…
d883b8b
…QueueHandler` (pythonGH-120872)

(cherry picked from commit 7d9c685)

Co-authored-by: Janek Nouvertné <[email protected]>
@bedevere-app

bedevere-app Bot commented Jun 27, 2024

Copy link
Copy Markdown

GH-121077 is a backport of this pull request to the 3.12 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.12 only security fixes label Jun 27, 2024
@bedevere-app

bedevere-app Bot commented Jun 27, 2024

Copy link
Copy Markdown

GH-121078 is a backport of this pull request to the 3.13 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 only security fixes label Jun 27, 2024
@provinzkraut
provinzkraut deleted the fix-gh-120868 branch June 27, 2024 07:35
vsajip pushed a commit that referenced this pull request Jun 27, 2024
@miss-islington
[3.13] gh-120868: Fix breaking change in logging.config when using …
9df7392
…`QueueHandler` (GH-120872) (GH-121078)

(cherry picked from commit 7d9c685)
vsajip pushed a commit that referenced this pull request Jun 28, 2024
@miss-islington
[3.12] gh-120868: Fix breaking change in logging.config when using …
b31f7e2
…`QueueHandler` (GH-120872) (GH-121077)

(cherry picked from commit 7d9c685)
mrahtz pushed a commit to mrahtz/cpython that referenced this pull request Jun 30, 2024
@provinzkraut @mrahtz
pythongh-120868: Fix breaking change in logging.config when using `…
38c568b
…QueueHandler` (pythonGH-120872)
estyxx pushed a commit to estyxx/cpython that referenced this pull request Jul 17, 2024
@provinzkraut @estyxx
pythongh-120868: Fix breaking change in logging.config when using `…
e9dd992
…QueueHandler` (pythonGH-120872)
@provinzkraut provinzkraut mentioned this pull request Dec 11, 2024
Bug: Litestar crashes on initialization with Python 3.12.4 and a read-only root filesystem litestar-org/litestar#3591
Closed
1 of 4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@picnixz picnixz picnixz left review comments
@vsajip vsajip vsajip approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@provinzkraut @bedevere-bot @vsajip @picnixz
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.