GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Use lazy imports on Python 3.15 to improve startup speed by hugovk · Pull Request #9733 · python-pillow/Pillow

Conversation

@hugovk

@hugovk hugovk commented Jun 29, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Re: #9330

Defer

The first commit here defers the import of tempfile in four files to the place where it's actually used, so we don't need to pay the import time if we're not using it. This helps for all Python versions.

So for example, only importing PIL.Image on Python 3.14:

pip install tuna
python3.14 -X importtime -c 'import PIL.Image' 2> import.log
tuna import.log
Before: 29 ms After: 23 ms
image image

Lazy

Python 3.15 introduces lazy imports:

  • https://docs.python.org/3.15/whatsnew/3.15.html#whatsnew315-lazy-imports
  • https://peps.python.org/pep-0810/

If you put the new lazy keyword before an import, it won't be actually imported until first use. But lazy is only for 3.15+, it's a syntax error in 3.10-3.14.

For code that cannot use the lazy keyword directly (for example, when supporting Python versions older than 3.15 while still using lazy imports on 3.15+), a module can define __lazy_modules__ as a container of fully qualified module name strings. Regular import statements for those modules are then treated as lazy, with the same semantics as the lazy keyword

So the second commit introduces __lazy_modules__.

We can only put top-level imports here, and not those from function or in try/except. And we don't need to put any that used right away at the top-level, like logging in Image.py, but it's not a big deal if we do.

Repeating with 3.15:

Before: 22 ms After: 16 ms
image image

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

akx, brittanyrey, jquast, and pauloxnet reacted with heart emoji
All reactions
  • ❤️ 4 reactions
@hugovk hugovk added the Performance label Jun 29, 2026
@codspeed

codspeed Bot commented Jun 29, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 603 untouched benchmarks
⏩ 335 skipped benchmarks1


Comparing hugovk:lazy-imports (0150848) with main (3dfc89e)

Open in CodSpeed

Footnotes

  1. 335 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@akx

akx commented Jun 30, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

I think the

ElementTree: ModuleType | None
try:
    from defusedxml import ElementTree
except ImportError:
    ElementTree = None

import should also belong in getxmp()? It'd become a neat try: import; except: warnings.warn() early return too.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk
hugovk force-pushed the lazy-imports branch from 4fd7de6 to b5761eb Compare June 30, 2026 09:44
@hugovk

hugovk commented Jun 30, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Thanks, a couple extra "icicles" knocked off.

With 3.15, on a different machine compared to yesterday:

main: 26 ms PR before: 20 ms PR after: 17 ms
image image image
akx reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@akx

akx commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

It's certainly a bigger and unrelated change, but looks like Pillow only uses logging for a couple debug calls here and there in PIL.Image -- could be worth it to just get rid of it there (though real-world apps will generally probably import logging somehow either way, so additional laziness would need to be put in there in the stdlib...).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk
hugovk force-pushed the lazy-imports branch from 46df57f to 74eba11 Compare June 30, 2026 10:43
@hugovk

hugovk commented Jun 30, 2026

Copy link
Copy Markdown
Member Author

I made the olefile import eager again: it's an optional dependency and lazy imports affect how its related plugins get registered.

I'll have a look at logging.

Unfortunately it's not straightforward to lazy import traceback in the stdib, because if we get an exception during shutdown, the import machinery might already be shutdown: python/cpython#150073.

lazy import re in logging might be more doable, but in any case won't show up until Python 3.16.

akx reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk

hugovk commented Jun 30, 2026

Copy link
Copy Markdown
Member Author

Something like this?

--- a/src/PIL/Image.py
+++ b/src/PIL/Image.py
-import logging
...
-logger = logging.getLogger(__name__)
+class _LazyLogger:
+    """
+    Defer importing `logging` until the logger is first used to avoid slow import.
+    """
+
+    def __getattr__(self, attr: str) -> Any:
+        import logging
+
+        logger = logging.getLogger(__name__)
+        globals()["logger"] = logger
+        return getattr(logger, attr)
+
+
+logger = _LazyLogger()
Before: 16 ms After: 12 ms
image image
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@akx

akx commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

@hugovk That'd work -- but I'm also thinking... do we need the 4 logger.debug()s for importing plugin modules in PIL/Image.py at all?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk

hugovk commented Aug 3, 2026

Copy link
Copy Markdown
Member Author

@radarhere Do you think we should keep those four log lines?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@radarhere

radarhere commented Aug 4, 2026

Copy link
Copy Markdown
Member

I'm inclined not to.

The debug message before the import seems unnecessary, as 99% of the time, the import will succeed, and it's not part of an overall consistent use of logger.debug on our part.

As for the one on error… I'm not sure under what circumstances it would run for any of our internal plugins. From a big picture perspective though, PIL printed the exception on DEBUG and #1207 changed it to use logger. I think PIL.report (#3870) has since become the easiest way to figure out which plugins are working and which aren't.

akx reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk
hugovk force-pushed the lazy-imports branch from 120360c to 0c94308 Compare August 9, 2026 11:57
@hugovk

hugovk commented Aug 9, 2026

Copy link
Copy Markdown
Member Author

logging dropped from Image:

Before: 20 ms After: 10 ms
image image
akx reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

hugovk
hugovk commented Aug 9, 2026
View reviewed changes
Comment thread src/PIL/ImageFilter.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/PIL/ImageOps.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@radarhere radarhere added the Needs Rebase label Aug 22, 2026
@hugovk
hugovk force-pushed the lazy-imports branch from 02f763e to 1f96948 Compare August 24, 2026 02:28
@hugovk hugovk removed the Needs Rebase label Aug 24, 2026
radarhere
radarhere reviewed Aug 24, 2026
View reviewed changes
Comment thread src/PIL/XpmImagePlugin.py

import re

from . import Image, ImageFile, ImagePalette

@radarhere radarhere Aug 24, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I correct in thinking that's there no particular reason why _binary should be lazy, but ImagePalette, isn't? I'm guessing you ran some performance tests, figured out which modules to import lazily based on that, and then applied that list throughout src?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@hugovk hugovk Sep 1, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I correct in thinking that's there no particular reason why _binary should be lazy, but ImagePalette, isn't?

With from ._binary import o8, the module being imported is PIL._binary.

With from . import ImagePalette, the module being imported is the PIL package.

I'm guessing you ran some performance tests, figured out which modules to import lazily based on that, and then applied that list throughout src?

I used https://github.com/henryiii/flake8-lazy and ran something like flake8-lazy src/PIL --apply set (set being faster than list: henryiii/flake8-lazy#50).

Although that does spells it like:

__lazy_modules__ = {f"{__spec__.parent}._binary"}

See henryiii/flake8-lazy#41 for the explanation. I decided to replace the f-strings in this PR.

But perhaps we should go for the f-strings and add flake8-lazy to pre-commit? (There's no Ruff rule yet.)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
hugovk added 4 commits September 1, 2026 14:06
@hugovk
Defer importing tempfile
8a36c5e
@hugovk
Use lazy imports on Python 3.15 to improve startup speed
c8d2165
@hugovk
Defer importing defusedxml
d0f7ae7
@hugovk
Drop debug logging from Image
2c10c4f
@hugovk
hugovk force-pushed the lazy-imports branch from 1f96948 to 2c10c4f Compare September 1, 2026 11:06
@hugovk

hugovk commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Updated: re-ran flake8-lazy src/PIL --apply set with the newer version (but evaluated f-strings), which removes some lazy imports because they're always imported, and others because they get "reified" (imported) immediately, and adds some extra imports the earlier version didn't catch.

akx reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@radarhere
Merge branch 'main' into lazy-imports
0150848
@radarhere
radarhere merged commit 6b5a7db into python-pillow:main Sep 8, 2026
63 of 65 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@hugovk
hugovk deleted the lazy-imports branch September 8, 2026 17:04
@radarhere radarhere mentioned this pull request Oct 15, 2026
Do not lazily import olefile #10213
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@radarhere radarhere radarhere left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

Performance

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@hugovk @akx @radarhere
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.