GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — gh-142349: Simplify lazy import resolution by pablogsal · Pull Request #158282 · python/cpython

Conversation

pablogsal commented Sep 27, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Member

The current implementation has some problems. The eval loop and module attribute lookup repeat the code that resolves a lazy import and replaces its placeholder. These copies behave differently. We also hold the global import lock during resolution, which can deadlock with another thread importing a package.

This moves the placeholder structure and resolution code into lazyimportobject.c. One helper now resolves the import and updates the namespace where we found the name. It only replaces the value if it is still the same placeholder or the child module that the normal importer just put there. Assignments and deletions made during the import are preserved. For ordinary dicts, the check and replacement are atomic.

We also remove the global import lock around resolution. Importlib still locks modules while loading them. Each thread keeps a set of placeholders being resolved to detect cycles. Custom import hooks can now run concurrently.

This removes duplicate lookup and cleanup code. The eval loop and module attribute lookup use the same helper, so they no longer need separate implementations of these rules.

With this design, a placeholder records the module to import and where the import was declared. A name taken from that import holds a reference to the original placeholder and records the attribute to look up. When the name is first used, the resolver follows those references, imports the module and performs the attribute lookups in order. The shared helper then replaces the placeholder in the namespace where it was found, provided that binding still holds the placeholder or that child module.

Issue: Implement PEP 810 – Explicit lazy imports #142349

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

pablogsal added 7 commits

September 27, 2026 15:19


          Move lazy resolution next to the placeholder

67ed535

Keep the placeholder structure private to lazyimportobject.c. Move the
resolution and attribute lookup code there so it can use the structure
without exposing its fields to the eval loop or import code.


          Track lazy resolution per thread

ac89dd0

Let importlib handle module locks instead of holding the global import
lock while resolving a placeholder. Keep active placeholders in a set on
the thread and remove them on every exit, including allocation failures.

Use the same cycle and recursion checks when a hook returns a placeholder.
Keep the source references alive until all attribute lookups finish.


          Keep the original lazy import error chain

846f934

Keep an existing cause or context when resolution fails. Add the import
location as a note in that case, without adding the same note twice.
Errors without an existing chain still get the declaration as their cause.


          Avoid waiting in the lazy from shortcut

dd01552

Only reuse a concrete attribute from a module that has finished loading.
Leave lazy attributes for resolution so module hooks still get a chance.
Check the module again after reading its spec, and keep interrupts visible.


          Simplify lazy import declarations

b78ebc9

Normalize the fromlist before calling the filter and check its items
before registering imports. Use one cleanup path and the existing filter
accessor. Pass the same five arguments to custom lazy hooks as to __import__.


          Handle lazy module hooks and children together

917df44

Resolve lazy __getattr__ and __dir__ hooks before calling them. Treat a
hook already being resolved as unavailable so it can import a sibling.

Bind a loaded child before removing its pending entry. Recheck the module
dict when another thread may have completed the load. Reuse modules already
in sys.modules during package cycles, and simplify child registration.


          Share lazy import rebinding

4dc64c8

Remember the namespace where lookup found the placeholder and use one
helper to resolve and replace it. Only replace a binding that still holds
the same placeholder, preserving assignments and deletions during import.

Check and replace ordinary dict entries atomically. Keep the mapping
protocol for custom namespaces and allow reads from readonly namespaces.
Reuse the global lookup helper in the eval loop and remove the duplicate.

pablogsal requested review from DinoV, FFY00, Yhg1s, ZeroIntensity, brettcannon, ericsnowcurrently, kumaraditya303, markshannon, methane, ncoghlan and warsaw as code owners

September 27, 2026 14:21

bedevere-app Bot mentioned this pull request

Sep 27, 2026

Implement PEP 810 – Explicit lazy imports #142349

Closed

bedevere-app Bot added the awaiting core review label

Sep 27, 2026

pablogsal commented Sep 27, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Member Author

@brittanyrey this cleans up a bit of the lazy import code by sharing the resolve and replace logic and keeping the placeholder details in one file. It also removes the global import lock around resolution. Could you take a look?

brittanyrey reacted with heart emoji

All reactions

❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

pablogsal added the skip news label

Sep 27, 2026


          Keep imported attributes on their parent package

ad34fae

Loading a child can replace the parent binding with the module before we fetch the imported attribute. Keep the module returned by the normal importer so the shared helper can replace it in its actual parent namespace. Other values and deletions still prevent replacement, and custom hooks keep control of their assignments.

pablogsal mentioned this pull request

Sep 29, 2026

gh-155695: Remove resolved names from sys.lazy_modules more consistently #157714

Merged

encukou commented Sep 30, 2026

Copy link

Copy Markdown

Member

Marking a release blocker for @hugovk's consideration, per #157714 (comment) . I'll start the review.

But, I must say that I feel the process is broken somewhere. From my side it, subjectively, looks like the issues that led here were ignored (1, 2) and argued as unnecessary by the PEP authors for months, so +884/-946 PR this late in the RC period is quite a surprise. I'm not sure what to do so we don't get this situation again. But, that's a discussion for after the release.