GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||
GeezerVison is showing github.com — gh-141276: Avoid compiling zipimport source in get_filename by harjothkhara · Pull Request #153799 · python/cpython Conversation
|
|
This fixes the duplicated SyntaxWarning from #141276, could someone take a look? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
the fix generally looks good to me, thanks for the PR!
see inline comments regarding skipping wasted source load.
a few additional requests:
the semantic change - previously, py files with syntax errors in zip would raise from find_spec() and get_filename(). after this change, find_spec() and get_filename() would succeed, and only exec_module() would raise.
while this sounds like a significant behavior change, I think it's acceptable and even desirable, because it brings zipimport in line with the standard filesystem based finder/loader - but it does warrant more explicit coverage in NEWS.
this also makes it somewhat risky to backport, so I'd like @serhiy-storchaka's opinion here as well.
a test that with a py with SyntaxError. assert that find_spec() and get_filename() succeed and return expected spec / filename, and exec_module() raises SyntaxError.
perhaps also have a test that calls get_filename() directly and asserts no warnings.
I see testBadMagic/testBadMTime already cover stale/bad-magic .pyc falling back to .py. could you add:
.pyc + .py with a syntax error → spec picks the .pyc and import succeeds__init__.py with a syntax error + a valid sibling module of the same name → origin is __init__.py and exec raises SyntaxError (no fallthrough to the module)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Move the compile_source=False early return above the archive read, so get_filename() no longer decompresses source it immediately discards. Also update the NEWS entry with itamaro's suggested wording and add tests covering the deferred-SyntaxError behavior: find_spec()/ get_filename() succeed and get_filename() emits no warnings for a module with a syntax error, a valid .pyc still wins over a bad sibling .py, and a package's __init__.py error doesn't fall through to a same-named sibling module.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
thanks for making the requested changes!
please don't forget to follow the bot instructions about commenting on the PR once you've made the requested changes (it's needed so the bot updates PR labels and makes it show up in review queues again, otherwise it's more likely to go unnoticed).
we also need to update the docs (Doc/library/zipimport.rst) to match the modified get_filename behavior (raises when "module couldn't be found", not when "module couldn't be imported".
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
|
I have made the requested changes; please review again. Pushed 8082336 with the NEWS wording, the docstring, the comment, the redundant |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Thanks for making the requested changes! @itamaro: please review the changes made to this pull request. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
thanks, it looks good to me! I'll merge main since it's been a while, and merge if CI passes. cc @serhiy-storchaka @hugovk what do you think about backporting this? (see previous comment) |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Documentation build overview
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
…thon#153799) `zipimporter.get_filename()` compiled source modules just to work out the module's file name. `find_spec()` calls it through `spec_from_loader()`, so importing a source module from a ZIP archive compiled it twice and emitted every `SyntaxWarning` twice. `get_filename()` now returns before reading or compiling the source. As a result, a `SyntaxError` in a zipped source module is now raised by `exec_module()` rather than by `find_spec()` or `get_filename()`, and the source isn't compiled twice during import, which matches the filesystem-based loaders.
|
Thanks @harjothkhara for the PR, and @itamaro for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Thanks @harjothkhara for the PR, and @itamaro for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Sorry, @harjothkhara and @itamaro, I could not cleanly backport this to Please backport manually with cherry_picker, see the devguide for more information. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
GH-159729 is a backport of this pull request to the 3.15 branch. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
hey @harjothkhara, would you like to look into the backport to 3.14? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…H-153799) (#159729) gh-141276: Avoid compiling zipimport source in get_filename (GH-153799) `zipimporter.get_filename()` compiled source modules just to work out the module's file name. `find_spec()` calls it through `spec_from_loader()`, so importing a source module from a ZIP archive compiled it twice and emitted every `SyntaxWarning` twice. `get_filename()` now returns before reading or compiling the source. As a result, a `SyntaxError` in a zipped source module is now raised by `exec_module()` rather than by `find_spec()` or `get_filename()`, and the source isn't compiled twice during import, which matches the filesystem-based loaders. (cherry picked from commit 230917f) Co-authored-by: Harjoth Khara <[email protected]>
pythonGH-153799 landed without a few of the review suggestions: - a versionchanged note on get_filename() in the docs; - a What's New entry describing both the SyntaxWarning fix and the behavior change; - tests for a stale .pyc next to a .py with a syntax error (falls through to the .py, and the SyntaxError is deferred to exec_module()), and for get_filename()/find_spec() not reading the source at all. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01Kq5LwfysHx5inm3J1TQCdS
|
GH-161504 is a backport of this pull request to the 3.14 branch. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…H-153799) (#161504) `zipimporter.get_filename()` compiled source modules just to work out the module's file name. `find_spec()` calls it through `spec_from_loader()`, so importing a source module from a ZIP archive compiled it twice and emitted every `SyntaxWarning` twice. `get_filename()` now returns before reading or compiling the source. As a result, a `SyntaxError` in a zipped source module is now raised by `exec_module()` rather than by `find_spec()` or `get_filename()`, and the source isn't compiled twice during import, which matches the filesystem-based loaders. (cherry picked from commit 230917f) Co-authored-by: Harjoth Khara <[email protected]>
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.