GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-117182: Allow lazily loaded modules to modify their own __class__ by effigies · Pull Request #117185 · python/cpython

Conversation

@effigies

@effigies effigies commented Mar 23, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

This PR resolves #117182. The meat of the PR is to check whether the module modified its own __class__ during load, and only set the default class if not.

I also realized that the LazyLoader saves the original __class__ in the loader_state dict. I haven't found a case where that __class__ isn't ModuleType, but if it ever does occur, it does make sense to restore that type. And if that type for some reason overrides __getattribute__, it would make sense to use that instead of object.__getattribute__.

  • Issue: Lazy loading conflicts with modules overriding ModuleType #117182

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@effigies
effigies requested review from brettcannon, ericsnowcurrently, ncoghlan and warsaw as code owners March 23, 2024 16:30
@bedevere-app bedevere-app Bot added the awaiting review label Mar 23, 2024
@bedevere-app bedevere-app Bot mentioned this pull request Mar 23, 2024
Lazy loading conflicts with modules overriding ModuleType #117182
Closed
effigies added 3 commits March 28, 2024 21:36
@effigies
pythongh-117182: Test interaction between lazy modules and self-modif…
933c646
…ying modules
@effigies
pythongh-117182: Allow modules to modify their own __class__ when laz…
a5e9670
…ily loaded
@effigies
Add news entry
393bb7c
@effigies
effigies force-pushed the fix-issue-117182 branch from b272bf5 to 393bb7c Compare March 29, 2024 01:37
FFY00
FFY00 approved these changes Apr 9, 2024
View reviewed changes

@FFY00 FFY00 left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@effigies thank you!

I agree with the changes, and the implementation looks good to me.

Regarding backporting, I think it'd be better to just merge this into main as an improvement, without backporting it to older versions. I don't feel strongly about this though, so if another dev thinks it makes sense to backport, that'd be fine by me.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

effigies reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Apr 9, 2024
@FFY00 FFY00 changed the title gh-117182: Allow modules to modify their own __class__ when lazily loaded gh-117182: Allow lazily loaded modules to modify their own __class__ Apr 9, 2024
@FFY00
FFY00 merged commit 19a2202 into python:main Apr 9, 2024
@bedevere-app bedevere-app Bot removed the awaiting merge label Apr 9, 2024
@FFY00

FFY00 commented Apr 9, 2024

Copy link
Copy Markdown
Member

Btw, I was forced to rename as the commit message was too big.

effigies reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@effigies
effigies deleted the fix-issue-117182 branch April 9, 2024 10:14
@lunaris

lunaris commented Apr 17, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Hey, I think this issue is affecting our codebase (https://github.com/pulumi/pulumi, specifically its Python SDK), which makes use of lazy loading modules that modify their own __class__. Responding to the comments above, would it be possible to backport this to 3.11/3.12/any affected versions? Presently Pulumi's Python SDK won't work for our users if they are on 3.11.9 or 3.12 due to this.

If not, is there a workaround at all? I've posted what I think is a minimal reproduction for our use case which I've tested with 3.8.19 ✔️, 3.9,19 ✔️, 3.10.14 ✔️, 3.11.8 ✔️, 3.11.9 🔴, and 3.12.3 🔴 at https://github.com/lunaris/python-lazy-loading-repro if that's useful.

Thanks for any support and thoughts!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@lunaris lunaris mentioned this pull request Apr 17, 2024
Pulumi providers fail on Python 3.11.9 pulumi/pulumi#15882
Closed
@effigies

effigies commented Apr 17, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Ah, that reproduction demonstrates that it's a regression introduced by #114781, which makes sense. We deferred resetting the module class until after load, to prevent races, but that would override subclassed module types where we did not previously.

@FFY00 I think it would be worth back-porting this.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@effigies

effigies commented Apr 17, 2024

Copy link
Copy Markdown
Contributor Author

@lunaris If it doesn't get backported, you can simply copy the LazyLoader and _LazyModule classes and use those instead of importlib.util.LazyLoader. There's nothing magic about the stdlib ones.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@lunaris

lunaris commented Apr 26, 2024

Copy link
Copy Markdown

@effigies Thanks for this -- really useful to know!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@FFY00 FFY00 FFY00 approved these changes
@brettcannon brettcannon Awaiting requested review from brettcannon brettcannon is a code owner
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner
@ncoghlan ncoghlan Awaiting requested review from ncoghlan ncoghlan is a code owner
@warsaw warsaw Awaiting requested review from warsaw warsaw is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Lazy loading conflicts with modules overriding ModuleType

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@effigies @FFY00 @lunaris
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.