GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — gh-161644: Avoid unmarshalling zipped .pyc files twice on import by itamaro · Pull Request #161700 · python/cpython Conversation
|
| compression | main | this PR | |
|---|---|---|---|
| stored | 96 ms | 79 ms | −18% |
| deflate | 131 ms | 99 ms | −25% |
| zstd | 141 ms | 125 ms | −12% |
Per module, find_spec() + get_code() for a single .pyc (in-process medians):
| pyc size | stored | deflate | zstd |
|---|---|---|---|
| 10 KB | −6% | −1% | −10% |
| 50 KB | −32% | −30% | −34% |
| 500 KB | −49% | −46% | −44% |
| 5 MB | −44% | −41% | −34% |
find_spec() itself no longer scales with pyc size: ~0.1 ms regardless of size, versus 43 ms for a 5 MB pyc on main.
zstd gains less because ZstdDecompressor construction dominates small reads and zstd decodes a whole block before producing output.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
`get_filename()` (and so `find_spec()`) read, decompressed and unmarshalled the whole `.pyc` just to learn the file name, and `exec_module()` then did it all again. Only the 16-byte header matters for picking the file: a bad or stale header falls back to the sibling `.py`, while errors in the marshalled body never do. Split the header checks out of `_unmarshal_code()` and have `get_filename()` run only those, reading (and decompressing) just enough of the member to cover the header. A `.pyc` with a valid header and a corrupt body now fails in `exec_module()` instead of `find_spec()`, with the same exception. Co-Authored-By: Claude Fable 5.1 <[email protected]>
Documentation build overview
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.