GeezerVison

Sorry fucked up the squash give me sec

Is this ready now @pablogsal?

Here's a somewhat surprising change in behaviour: given these two modules:

A.py:

lazy import xml.missing # CORRECT ONE
import B

xml.missing

B.py:

lazy import xml.missing # WRONG ONE

Before this PR the use of xml.missing in A.py will print a traceback showing the lazy import in A.py as the cause:

% base/python A.py
Traceback (most recent call last):
  File "A.py", line 1, in <module>
    lazy import xml.missing # CORRECT ONE
ImportError: lazy import of 'xml.missing' raised an exception during resolution

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "A.py", line 4, in <module>
    xml.missing
    ^^^
ModuleNotFoundError: No module named 'xml.missing'

With this PR, it reports the lazy import in B.py:

% pr/python A.py
Traceback (most recent call last):
  File "B.py", line 1, in <module>
    lazy import xml.missing # WRONG ONE
ImportError: lazy import of 'xml.missing' raised an exception during resolution

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "A.py", line 4, in <module>
    xml.missing
ModuleNotFoundError: No module named 'xml.missing'

Swapping the order of the lazy imports used to not have an effect on the traceback, but now it does: the last lazy import wins. I'm not sure if that's fixable, but I think if we need a single location the first lazy import would be a better winner.

Swapping the order of the lazy imports used to not have an effect on the traceback, but now it does: the last lazy import wins. I'm not sure if that's fixable, but I think if we need a single location the first lazy import would be a better winner.

Okay I think I see why we can't do that, either. I think this is fixable later (perhaps by merging lazy objects, storing all the locations and selecting an appropriate one when printing the traceback). Still, it's a little annoying that this used to work as users would expect.

I think there's a similar issue with modules defining __import__, although I don't have a simple reproducer: if you have two different modules lazily importing the same pkg.module, with their own __import__ methods shadowing the builtin __import__, module A may end up calling module B's __import__ when it does its reification. I can't see how to avoid that one without giving A and B its own sys.modules entirely.

🤖 New build scheduled with the buildbot fleet by @hugovk for commit 4e83bfa 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F158521%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

I think we can fix the A/B case by keeping the declarations keyed by the importing module’s __name__. When accessing the child, we use the current frame’s globals to pick the right one. Custom-builtins imports can stay out of that registry since they already resolve the full path through their own hook.

I tried this locally. It’s about 70 lines of C. There’s still ambiguity if two namespaces share the same name, or the same filename for anonymous exec, but it fixes this case without keeping globals alive.

What do you think, @Yhg1s?

I think we can fix the A/B case by keeping the declarations keyed by the importing module’s __name__. When accessing the child, we use the current frame’s globals to pick the right one. Custom-builtins imports can stay out of that registry since they already resolve the full path through their own hook.

I tried this locally. It’s about 70 lines of C. There’s still ambiguity if two namespaces share the same name, or the same filename for anonymous exec, but it fixes this case without keeping globals alive.

What do you think, @Yhg1s?

Yes, I was thinking something similar, but I don't think we should try to fix this for 3.15.0. We can pre-emptively file a bug and fix it later.

I merged in the main branch with #157714.

Thanks @pablogsal for the PR, and @Yhg1s for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

GH-158613 is a backport of this pull request to the 3.15 branch.