GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — Use lazy imports on Python 3.15 to improve startup speed by hugovk · Pull Request #9733 · python-pillow/Pillow Conversation
|
| Before: 29 ms | After: 23 ms |
|---|---|
|
|
Python 3.15 introduces lazy imports:
If you put the new lazy keyword before an import, it won't be actually imported until first use. But lazy is only for 3.15+, it's a syntax error in 3.10-3.14.
For code that cannot use the
lazykeyword directly (for example, when supporting Python versions older than 3.15 while still using lazy imports on 3.15+), a module can define__lazy_modules__as a container of fully qualified module name strings. Regularimportstatements for those modules are then treated as lazy, with the same semantics as thelazykeyword
So the second commit introduces __lazy_modules__.
We can only put top-level imports here, and not those from function or in try/except. And we don't need to put any that used right away at the top-level, like logging in Image.py, but it's not a big deal if we do.
Repeating with 3.15:
| Before: 22 ms | After: 16 ms |
|---|---|
|
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Merging this PR will not alter performance
Comparing Footnotes
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
I think the import should also belong in |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Thanks, a couple extra "icicles" knocked off. With 3.15, on a different machine compared to yesterday:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
It's certainly a bigger and unrelated change, but looks like |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
I made the olefile import eager again: it's an optional dependency and lazy imports affect how its related plugins get registered. I'll have a look at logging. Unfortunately it's not straightforward to
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@hugovk That'd work -- but I'm also thinking... do we need the 4 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@radarhere Do you think we should keep those four log lines? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
I'm inclined not to. The debug message before the import seems unnecessary, as 99% of the time, the import will succeed, and it's not part of an overall consistent use of As for the one on error… I'm not sure under what circumstances it would run for any of our internal plugins. From a big picture perspective though, PIL |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
|
||
| import re | ||
|
|
||
| from . import Image, ImageFile, ImagePalette |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Am I correct in thinking that's there no particular reason why _binary should be lazy, but ImagePalette, isn't? I'm guessing you ran some performance tests, figured out which modules to import lazily based on that, and then applied that list throughout src?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Am I correct in thinking that's there no particular reason why
_binaryshould be lazy, butImagePalette, isn't?
With from ._binary import o8, the module being imported is PIL._binary.
With from . import ImagePalette, the module being imported is the PIL package.
I'm guessing you ran some performance tests, figured out which modules to import lazily based on that, and then applied that list throughout src?
I used https://github.com/henryiii/flake8-lazy and ran something like flake8-lazy src/PIL --apply set (set being faster than list: henryiii/flake8-lazy#50).
Although that does spells it like:
__lazy_modules__ = {f"{__spec__.parent}._binary"}
See henryiii/flake8-lazy#41 for the explanation. I decided to replace the f-strings in this PR.
But perhaps we should go for the f-strings and add flake8-lazy to pre-commit? (There's no Ruff rule yet.)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Updated: re-ran |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.