GeezerVison is showing github.com — Improve import time of various stdlib modules · Issue #109653 · python/cpython
Improve import time of various stdlib modules #109653DescriptionReacted by Daniel Hollas, Ofek Lev, Erlend E. Aasland, Hugo van Kemenade, Pathompong Kwangtong and Brett Holman Reacted by Ofek Lev, Hugo van Kemenade, Pathompong Kwangtong and EDM115 Reacted by Daniel Hollas, Ofek Lev, Erlend E. Aasland, Hugo van Kemenade and Pathompong Kwangtong Activityadded a commit that references this issue on Sep 23, 2023gh-109653: `typing.py`: improve import time by creating soft-deprecat… changed the title [-]Improve import time of `typing.py`[/-] [+]Improve import time of various stdlib modules[/+] on Sep 23, 2023AlexWaygood commented on Sep 23, 2023 Retitling this issue to reflect a change of scope: part of the reason for the slow import time of typing is the slow import time of some of the modules it imports. There may be low-hanging fruit in some other stdlib modules as well. hugovk commented on Sep 23, 2023 An idea: review the output of a linter for "unused" imports.
Some may be false positives due to needing an import's side effects, but I expect there will be some that are no longer needed. Reacted by Alex Waygood, Itamar Oren, Jelle Zijlstra and Eric V. Smith itamaro commented on Sep 23, 2023
Some may be false positives due to needing an import's side effects ...
How do you feel about making an intentional effort to get rid of such reliance on side effects in the stdlib?
Explicit is better than implicit :) Reacted by Hugo van Kemenade AlexWaygood commented on Sep 23, 2023
An idea: review the output of a linter for "unused" imports.
FWIW, I can only find one in the stdlib that I'm 100% confident with, using ruff/pycln:
I think this is because Victor went through them all fairly recently, in #105411. There are a fair few in Lib/test/, though. I'll file a PR in a bit. Reacted by Itamar Oren Reacted by Hugo van Kemenade added a commit that references this issue on Sep 23, 2023gh-109653: Improve `enum` import time by avoiding import of `functool… sobolevn commented on Sep 24, 2023 This is also interesting:
dataclasses copies recursive_repr to avoid a dependency:
|
# This function's logic is copied from "recursive_repr" function in |
|
# reprlib module to avoid dependency. |
|
def _recursive_repr(user_function): |
|
# Decorator to make a repr function return "..." for a recursive |
|
# call. |
|
repr_running = set() |
|
|
|
@functools.wraps(user_function) |
|
def wrapper(self): |
|
key = id(self), _thread.get_ident() |
|
if key in repr_running: |
|
return '...' |
|
repr_running.add(key) |
|
try: |
|
result = user_function(self) |
|
finally: |
|
repr_running.discard(key) |
|
return result |
|
return wrapper |
- But, it imports
functools:
- And it imports
reprlib.recursive_repr:
|
from reprlib import recursive_repr |
sobolevn commented on Sep 24, 2023 One more interesting case:
functools says that import types is deferred to singledispatch():
|
# import types, weakref # Deferred to single_dispatch() |
- But several lines below
types module is imported:
|
from types import GenericAlias |
added a commit that references this issue on May 7, 2024gh-109653: What's new: Note improved import times for several stdlib … layday commented on May 8, 2024 I believe there are futher improvements to be made here - how would you like to handle these? For example, importlib.metadata imports zipfile at the top for a function that won't be called in the vast majority of cases. It also imports importlib.abc, which in turn imports importlib.resources, to subclass an ABC with a single, non-abstract method - I assume redefining the method in importlib.metadata would be harmless. AlexWaygood commented on May 8, 2024
I believe there are futher improvements to be made here - how would you like to handle these? For example, importlib.metadata imports zipfile at the top for a function that won't be called in the vast majority of cases. It also imports importlib.abc, which in turn imports importlib.resources, to subclass an ABC with a single, non-abstract method - I assume redefining the method in importlib.metadata would be harmless.
Sure -- I felt like the low-hanging fruit had been tackled here, and this issue hadn't seen much activity in quite a while, so I felt like it was a good time to close. But more improvements are obviously always welcome. Maybe open a new issue if you have some more targeted improvements you want to propose? Reacted by layday and Hugo van Kemenade ofek commented on May 8, 2024 SponsorContributor More actions Please link the new issue here so we all can start following that as well! layday commented on May 8, 2024 Reacted by Daniel Hollas and Ofek Lev
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|