GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-155695: Remove resolved names from sys.lazy_modules more consistently by brittanyrey · Pull Request #157714 · python/cpython

Conversation

@brittanyrey

@brittanyrey brittanyrey commented Sep 17, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

summary

Improve sys.lazy_modules to address the following issues:

  • A lazy import of a module that has already been loaded does not get added to sys.lazy_modules .
  • The "pkg.attr" entry for lazy from pkg import attr is cleaned up after reification.

perf

End-to-end (hyperfine, --warmup 50, 1000 runs; 400 for reify-only)

scenario base patched control patched/base control/base
bare startup (~0 lazy stmts) 12.1 ± 0.6 ms 12.0 ± 1.0 ms 12.1 ± 0.7 ms 0.99× 1.00×
synthetic: 243k lazy stmts 43.9 ± 3.0 ms 46.4 ± 2.9 ms 43.7 ± 2.8 ms 1.06× 1.00×
synthetic: 162k reifications 223.7 ± 11.8 ms 228.1 ± 10.7 ms 225.1 ± 11.9 ms 1.02× 1.01×
realistic -X lazy_imports=all app 68.0 ± 3.2 ms 68.3 ± 3.1 ms 67.4 ± 2.4 ms 1.00× 0.99×
  • Issue: Names that have been resolved are not consistently removed from sys.lazy_modules #155695

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@brittanyrey
brittanyrey requested review from DinoV, FFY00, Yhg1s, brettcannon, ericsnowcurrently, kumaraditya303, ncoghlan, pablogsal and warsaw as code owners September 17, 2026 22:22
@bedevere-app bedevere-app Bot mentioned this pull request Sep 17, 2026
Names that have been resolved are not consistently removed from sys.lazy_modules #155695
Closed
@bedevere-app bedevere-app Bot added the awaiting review label Sep 17, 2026
pablogsal
pablogsal requested changes Sep 18, 2026
View reviewed changes

@pablogsal pablogsal left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found one tracking regression when a module fails during initialization, see the inline comment.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Python/import.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot added awaiting changes and removed awaiting review labels Sep 18, 2026
@bedevere-app

bedevere-app Bot commented Sep 18, 2026

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@brittanyrey
brittanyrey requested a review from pablogsal September 18, 2026 17:42
@brittanyrey

brittanyrey commented Sep 21, 2026

Copy link
Copy Markdown
Contributor Author

@pablogsal Addressed comments.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@brettcannon
brettcannon removed their request for review September 21, 2026 21:03
@brittanyrey

brittanyrey commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

cc @encukou
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
pablogsal
pablogsal reviewed Sep 25, 2026
View reviewed changes
Comment thread Python/import.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@pablogsal

pablogsal commented Sep 25, 2026

Copy link
Copy Markdown
Member

Thanks! The initializing-module case is covered now. I left one follow-up comment about the new __spec__ lookup.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
brittanyrey added 4 commits September 26, 2026 22:14
@brittanyrey @pablogsal
Remove resolved names from sys.lazy_modules consistently
aab59a0 
Names only left sys.lazy_modules through _imp._set_lazy_attributes(), which
the import machinery calls from _find_and_load_unlocked(). Two cases never
reached it, so their names were recorded and then kept forever:

- A lazy import of a module already in sys.modules. _find_and_load() returns
  early, so nothing ever discards the name. Do not record it in the first
  place.

- The "pkg.attr" entry for `lazy from pkg import attr`. The import machinery
  only discards module names, and attr is often not a module. Discard it when
  the lazy object is reified, where the name is already known and has been
  resolved either way.

Submodules that are not yet loaded are still tracked: loading a package does
not load its submodules, so those imports can still fire. Names whose
reification failed also stay tracked, since the import can still happen.
@brittanyrey @pablogsal
Keep tracking modules that are still initializing
8dedf69 
A module is in sys.modules while its body runs, so lazy_modules_add()
counted it as loaded and skipped recording the name. If the body then
raises, the module is removed from sys.modules again and the lazy
import is left pending under no name at all.

Check __spec__._initializing so such a module does not count as loaded.
A name added while a module initializes is still discarded once the
import completes, since _set_lazy_attributes() runs after the body.
@brittanyrey @pablogsal
Empty commit to re-run CI
8ef4ae0
@brittanyrey @pablogsal
Read __spec__ from the module dict when checking for initialization
0456068
@pablogsal pablogsal added the needs backport to 3.15 bugs and security fixes label Sep 26, 2026
@pablogsal
pablogsal force-pushed the b-lazy-modules-stale-entries branch from 374d85f to 44ebbaa Compare September 26, 2026 21:39
pablogsal
pablogsal approved these changes Sep 26, 2026
View reviewed changes
@pablogsal
pablogsal enabled auto-merge (squash) September 26, 2026 21:39
@bedevere-app bedevere-app Bot removed the awaiting changes label Sep 26, 2026
@github-project-automation github-project-automation Bot added this to Release and Deferred blockers 🚫 Sep 29, 2026
@github-project-automation github-project-automation Bot moved this to Todo in Release and Deferred blockers 🚫 Sep 29, 2026
@encukou encukou mentioned this pull request Sep 30, 2026
gh-142349: Simplify lazy import resolution #158282
Merged
@hugovk hugovk added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Sep 30, 2026
@bedevere-bot

bedevere-bot commented Sep 30, 2026

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @hugovk for commit ad36b56 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F157714%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Sep 30, 2026
@pablogsal pablogsal added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 1, 2026
@bedevere-bot

bedevere-bot commented Oct 1, 2026

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @pablogsal for commit ad36b56 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F157714%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 1, 2026
@brittanyrey brittanyrey changed the title gh-155695: Remove resolved names from sys.lazy_modules consistently gh-155695: Remove resolved names from sys.lazy_modules more consistently Oct 1, 2026
@brittanyrey
Rebase.
95198ab
@brittanyrey

brittanyrey commented Oct 1, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Added a rebase @encukou
I think the PR can be slimmed down.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@encukou

encukou commented Oct 1, 2026

Copy link
Copy Markdown
Member

Thank you! You got roughly to the same place I did, but faster as I keep double-checking.
I'll look for places to slimming down.
One thing that bothers me is using the human-friendly display name (with a PyUnicode_FromFormat("%U...", m->lz_from); case) when discarding, but addressing that won't slim things down.

Enjoy your PTO! Should I push into this PR, or do you want to continue working on it?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
encukou
encukou approved these changes Oct 2, 2026
View reviewed changes

@encukou encukou left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I left some suggestions at brittanyrey#1; feel free to merge or cherry-pick if those look good.

The PR can be slimmed down by removing the _PyImport_DiscardLazyModule indirection, but keeping all uses of LAZY_MODULES in import.c has its own advantages.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@hugovk hugovk added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 2, 2026
@bedevere-bot

bedevere-bot commented Oct 2, 2026

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @hugovk for commit 95198ab 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F157714%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Oct 2, 2026
encukou
encukou reviewed Oct 2, 2026
View reviewed changes
Comment thread Objects/lazyimportobject.c
}
assert(obj == NULL || !PyLazyImport_CheckExact(obj));
if (obj != NULL) {
PyObject *name = lazy_import_name(lz);

@encukou encukou Oct 2, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If #158521 is merged first, this will change to lazy_import_path since only the root will be resolved here:

Suggested change
PyObject *name = lazy_import_name(lz);
PyObject *name = lazy_import_path(lz);

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
encukou added 2 commits October 2, 2026 13:46
@encukou
Reuse return value from PySet_Discard
84a8e95
@encukou
Share already-imported module between lazy_modules_add & all register…
45d23cd 
…_from_lazy_on_parent
@encukou

encukou commented Oct 2, 2026

Copy link
Copy Markdown
Member

Buildbot failures are pre-existing/unrelated.

I've pushed my suggestions here, except human-friendly display name which will be obsolete if #158521 is merged.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@brittanyrey

brittanyrey commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

Thank you @encukou!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@hugovk
hugovk merged commit be87a85 into python:main Oct 2, 2026
54 checks passed

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the awaiting merge label Oct 2, 2026
@miss-islington-app

miss-islington-app Bot commented Oct 2, 2026

Copy link
Copy Markdown

Thanks @brittanyrey for the PR, and @hugovk for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app

bedevere-app Bot commented Oct 2, 2026

Copy link
Copy Markdown

GH-158607 is a backport of this pull request to the 3.15 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Oct 2, 2026
@encukou encukou mentioned this pull request Oct 2, 2026
gh-158140: Resolve lazily imported sibling submodules independently #158521
Merged
Yhg1s pushed a commit that referenced this pull request Oct 2, 2026
@miss-islington @brittanyrey
@pablogsal @encukou
[3.15] gh-155695: Remove resolved names from sys.lazy_modules more co…
c2eb1a8 
…nsistently (GH-157714) (#158607)

gh-155695: Remove resolved names from sys.lazy_modules more consistently (GH-157714)
(cherry picked from commit be87a85)

Co-authored-by: Brittany Reynoso <[email protected]>
Co-authored-by: Pablo Galindo Salgado <[email protected]>
Co-authored-by: Petr Viktorin <[email protected]>
Kronuz pushed a commit to Kronuz/cpython that referenced this pull request Oct 5, 2026
@miss-islington @brittanyrey
@pablogsal @encukou @Kronuz
[3.15] pythongh-155695: Remove resolved names from sys.lazy_modules m…
c305e7e 
…ore consistently (pythonGH-157714) (python#158607)

pythongh-155695: Remove resolved names from sys.lazy_modules more consistently (pythonGH-157714)
(cherry picked from commit be87a85)

Co-authored-by: Brittany Reynoso <[email protected]>
Co-authored-by: Pablo Galindo Salgado <[email protected]>
Co-authored-by: Petr Viktorin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@encukou encukou encukou approved these changes
@pablogsal pablogsal pablogsal approved these changes
@Yhg1s Yhg1s Awaiting requested review from Yhg1s Yhg1s is a code owner
@DinoV DinoV Awaiting requested review from DinoV DinoV is a code owner
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner
@ncoghlan ncoghlan Awaiting requested review from ncoghlan ncoghlan is a code owner
@warsaw warsaw Awaiting requested review from warsaw warsaw is a code owner
@FFY00 FFY00 Awaiting requested review from FFY00 FFY00 is a code owner
@kumaraditya303 kumaraditya303 Awaiting requested review from kumaraditya303 kumaraditya303 is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

release-blocker

Projects

Release and Deferred blockers 🚫
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@brittanyrey @pablogsal @encukou @bedevere-bot @hugovk
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.