GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||
GeezerVison is showing github.com — gh-109653: `typing.py`: improve import time by creating soft-deprecated members on demand by AlexWaygood · Pull Request #109651 · python/cpython Conversation
|
|
🤖 New build scheduled with the buildbot fleet by @AlexWaygood for commit 180461f 🤖 If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
typing.pytyping.py
typing.pytyping.py: improve import time by creating soft-deprecated members on demand
|
Would it be going to far to use this approach for all of the If it would be, perhaps we could defer the A |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
The main reason why the changes here save us so much is because we can avoid unconditionally importing
IIRC, |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
I can just about see a path to inlining A |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
FWIW @AA-Turner, here's a diff (relative to this PR branch) that shaves another 0.002s off the import time by avoiding the Detailsdiff --git a/Lib/typing.py b/Lib/typing.py index 84b741bfb0..0218b715f6 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -19,9 +19,7 @@ """ from abc import abstractmethod, ABCMeta -import collections -from collections import defaultdict -import collections.abc +import _collections_abc import copyreg import functools import operator @@ -40,6 +38,11 @@ Generic, ) +try: + from _collections import defaultdict +except ImportError: + from collections import defaultdict + # Please keep __all__ alphabetized within each category. __all__ = [ # Super-special typing primitives. @@ -219,7 +222,7 @@ def _should_unflatten_callable_args(typ, args): we need to unflatten it. """ return ( - typ.__origin__ is collections.abc.Callable + typ.__origin__ is _collections_abc.Callable and not (len(args) == 2 and _is_param_expr(args[0])) ) @@ -1317,7 +1320,7 @@ def _make_substitution(self, args, new_arg_by_param): subargs.append(new_arg_by_param[x]) new_arg = old_arg[tuple(subargs)] - if self.__origin__ == collections.abc.Callable and isinstance(new_arg, tuple): + if self.__origin__ == _collections_abc.Callable and isinstance(new_arg, tuple): # Consider the following `Callable`. # C = Callable[[int], str] # Here, `C.__args__` should be (int, str) - NOT ([int], str). @@ -1885,7 +1888,7 @@ def _proto_hook(cls, other): # ...or in annotations, if it is a sub-protocol. annotations = getattr(base, '__annotations__', {}) - if (isinstance(annotations, collections.abc.Mapping) and + if (isinstance(annotations, _collections_abc.Mapping) and attr in annotations and issubclass(other, Generic) and getattr(other, '_is_protocol', False)): break @@ -2521,18 +2524,18 @@ class Other(Leaf): # Error reported by type checker # Various ABCs mimicking those in collections.abc. _alias = _SpecialGenericAlias -Hashable = _alias(collections.abc.Hashable, 0) # Not generic. -Awaitable = _alias(collections.abc.Awaitable, 1) -Coroutine = _alias(collections.abc.Coroutine, 3) -AsyncIterable = _alias(collections.abc.AsyncIterable, 1) -AsyncIterator = _alias(collections.abc.AsyncIterator, 1) -Iterable = _alias(collections.abc.Iterable, 1) -Iterator = _alias(collections.abc.Iterator, 1) -Reversible = _alias(collections.abc.Reversible, 1) -Sized = _alias(collections.abc.Sized, 0) # Not generic. -Container = _alias(collections.abc.Container, 1) -Collection = _alias(collections.abc.Collection, 1) -Callable = _CallableType(collections.abc.Callable, 2) +Hashable = _alias(_collections_abc.Hashable, 0) # Not generic. +Awaitable = _alias(_collections_abc.Awaitable, 1) +Coroutine = _alias(_collections_abc.Coroutine, 3) +AsyncIterable = _alias(_collections_abc.AsyncIterable, 1) +AsyncIterator = _alias(_collections_abc.AsyncIterator, 1) +Iterable = _alias(_collections_abc.Iterable, 1) +Iterator = _alias(_collections_abc.Iterator, 1) +Reversible = _alias(_collections_abc.Reversible, 1) +Sized = _alias(_collections_abc.Sized, 0) # Not generic. +Container = _alias(_collections_abc.Container, 1) +Collection = _alias(_collections_abc.Collection, 1) +Callable = _CallableType(_collections_abc.Callable, 2) Callable.__doc__ = \ """Deprecated alias to collections.abc.Callable. @@ -2547,15 +2550,15 @@ class Other(Leaf): # Error reported by type checker There is no syntax to indicate optional or keyword arguments; such function types are rarely used as callback types. """ -AbstractSet = _alias(collections.abc.Set, 1, name='AbstractSet') -MutableSet = _alias(collections.abc.MutableSet, 1) +AbstractSet = _alias(_collections_abc.Set, 1, name='AbstractSet') +MutableSet = _alias(_collections_abc.MutableSet, 1) # NOTE: Mapping is only covariant in the value type. -Mapping = _alias(collections.abc.Mapping, 2) -MutableMapping = _alias(collections.abc.MutableMapping, 2) -Sequence = _alias(collections.abc.Sequence, 1) -MutableSequence = _alias(collections.abc.MutableSequence, 1) +Mapping = _alias(_collections_abc.Mapping, 2) +MutableMapping = _alias(_collections_abc.MutableMapping, 2) +Sequence = _alias(_collections_abc.Sequence, 1) +MutableSequence = _alias(_collections_abc.MutableSequence, 1) ByteString = _DeprecatedGenericAlias( - collections.abc.ByteString, 0, removal_version=(3, 14) # Not generic. + _collections_abc.ByteString, 0, removal_version=(3, 14) # Not generic. ) # Tuple accepts variable number of parameters. Tuple = _TupleType(tuple, -1, inst=False, name='Tuple') @@ -2571,20 +2574,15 @@ class Other(Leaf): # Error reported by type checker To specify a variable-length tuple of homogeneous type, use Tuple[T, ...]. """ List = _alias(list, 1, inst=False, name='List') -Deque = _alias(collections.deque, 1, name='Deque') Set = _alias(set, 1, inst=False, name='Set') FrozenSet = _alias(frozenset, 1, inst=False, name='FrozenSet') -MappingView = _alias(collections.abc.MappingView, 1) -KeysView = _alias(collections.abc.KeysView, 1) -ItemsView = _alias(collections.abc.ItemsView, 2) -ValuesView = _alias(collections.abc.ValuesView, 1) +MappingView = _alias(_collections_abc.MappingView, 1) +KeysView = _alias(_collections_abc.KeysView, 1) +ItemsView = _alias(_collections_abc.ItemsView, 2) +ValuesView = _alias(_collections_abc.ValuesView, 1) Dict = _alias(dict, 2, inst=False, name='Dict') -DefaultDict = _alias(collections.defaultdict, 2, name='DefaultDict') -OrderedDict = _alias(collections.OrderedDict, 2) -Counter = _alias(collections.Counter, 1) -ChainMap = _alias(collections.ChainMap, 2) -Generator = _alias(collections.abc.Generator, 3) -AsyncGenerator = _alias(collections.abc.AsyncGenerator, 2) +Generator = _alias(_collections_abc.Generator, 3) +AsyncGenerator = _alias(_collections_abc.AsyncGenerator, 2) Type = _alias(type, 1, inst=False, name='Type') Type.__doc__ = \ """Deprecated alias to builtins.type. @@ -2692,8 +2690,8 @@ def _make_nmtuple(name, types, module, defaults = ()): fields = [n for n, t in types] types = {n: _type_check(t, f"field {n} annotation must be a type") for n, t in types} - nm_tpl = collections.namedtuple(name, fields, - defaults=defaults, module=module) + from collections import namedtuple + nm_tpl = namedtuple(name, fields, defaults=defaults, module=module) nm_tpl.__annotations__ = nm_tpl.__new__.__annotations__ = types return nm_tpl @@ -3432,6 +3430,17 @@ def __getattr__(attr): elif attr in {"ContextManager", "AsyncContextManager"}: import contextlib obj = _alias(getattr(contextlib, f"Abstract{attr}"), 1, name=attr) + elif attr in {"Deque", "DefaultDict", "OrderedDict", "Counter", "ChainMap"}: + import collections + match attr: + case "Deque": + obj = _alias(collections.deque, 1, name="Deque") + case "DefaultDict": + obj = _alias(collections.defaultdict, 2, name="DefaultDict") + case "OrderedDict" | "ChainMap": + obj = _alias(getattr(collections, attr), 2) + case "Counter": + obj = _alias(collections.Counter, 1) else: raise AttributeError(f"Module 'typing' has no attribute {attr!r}") globals()[attr] = obj |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Though still 25% (on the 8ms post-this PR) or 50% total incl. this PR. I agree this PR is both bigger bang-for-buck and should come first (I've no comments save applying Tom's suggestion), but for e.g. CLIs, milli-seconds can matter in feeling responsive, so I think worth at least considering the further improvement! A |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
I'm a bit skeptical of this because it relies on avoiding imports of common standard library modules only: how many real applications are there going to be that want to import On the other hand, the change in this PR is fairly small and self-contained, so I won't oppose it. But we shouldn't go through extremely lengths to avoid importing common stdlib modules. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Co-authored-by: Thomas Grainger <[email protected]>
Well, for one example, from a quick grep, More than that, though, I think that it's generally good to do this kind of thing (where it's not significantly detrimental to code readability). It's best to reduce dependencies between stdlib modules wherever possible, and make it clearer exactly why certain stdlib modules depend on others. It helps reduce the extent to which the stdlib is a massive import cycle.
I definitely agree with you there :) |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
But they do use |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sure, it's unlikely that this PR is going to hugely improve the import time of |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Those are good points. Let's do it.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Nice! I'm fine with this, but #109651 (comment) crosses the line for me.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Thanks all for the helpful reviews! |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…precated members on demand (python#109651) Co-authored-by: Thomas Grainger <[email protected]>
…precated members on demand (python#109651) Co-authored-by: Thomas Grainger <[email protected]>
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.