GeezerVison is showing github.com — inspect not capturing type annotations created by __class_getitem__ · Issue #89601 · python/cpython
inspect not capturing type annotations created by __class_getitem__ #89601Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2021-10-29.08:43:15.006>
created_at = <Date 2021-10-11.22:05:54.548>
labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
title = 'inspect not capturing type annotations created by __class_getitem__'
updated_at = <Date 2021-10-29.08:43:15.004>
user = 'https://github.com/rhettinger' bugs.python.org fields:
activity = <Date 2021-10-29.08:43:15.004>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2021-10-29.08:43:15.006>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2021-10-11.22:05:54.548>
creator = 'rhettinger'
dependencies = []
files = []
hgrepos = []
issue_num = 45438
keywords = ['patch']
message_count = 21.0
messages = ['403692', '403697', '403759', '403760', '403826', '404961', '404966', '404967', '404970', '404974', '404976', '404993', '405034', '405128', '405132', '405133', '405138', '405161', '405163', '405283', '405292']
nosy_count = 8.0
nosy_names = ['gvanrossum', 'rhettinger', 'lukasz.langa', 'serhiy.storchaka', 'levkivskyi', 'miss-islington', 'kj', 'martinitus']
pr_nums = ['29212', '29253', '29254']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45438'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11'] Activityrhettinger commented on Oct 11, 2021 ContributorAuthor More actions In the example below, __annotations__ is correct but not the corresponding Signature object.
-----------------------------------------------------------------------
from typing import List
def f(s: List[float]) -> None: pass
def g(s: list[float]) -> None: pass >>> inspect.signature(f)
<Signature (s: List[float]) -> None>
>>> inspect.signature(g)
<Signature (s: list) -> None> g.__annotations__
{'s': list[float], 'return': None} gvanrossum commented on Oct 11, 2021 Raymond, the bug must be in the Python code in inspect.py. Could you dig a little deeper there? I don't know much about it. Specifically I think the problem may just be in the repr() of class Parameter:
>>> inspect.signature(g).parameters['s']
<Parameter "s: list">
>>> inspect.signature(g).parameters['s'].annotation
list[float]
>>> rhettinger commented on Oct 12, 2021 ContributorAuthor More actions It looks like the error is in inspect.formatannotation().
For instances of type, that function incorrectly returns only annotation.__qualname__.
Instead, it should return repr(annotation) which would include the args.
=================================================================
def formatannotation(annotation, base_module=None):
if getattr(annotation, '__module__', None) == 'typing':
return repr(annotation).replace('typing.', '')
if isinstance(annotation, type): # <== Erroneous case
if annotation.__module__ in ('builtins', base_module):
return annotation.__qualname__
return annotation.__module__+'.'+annotation.__qualname__
return repr(annotation)gvanrossum commented on Oct 12, 2021 Sure. Waiting for your PR. :-) serhiy-storchaka commented on Oct 13, 2021 The cause is that isinstance(list[int], type) returns True. It can cause bugs in other parts of the code which test for instance of type. For example:
>>> types.resolve_bases((typing.List[int],))
(<class 'list'>, <class 'typing.Generic'>)
>>> types.resolve_bases((list[int],))
(list[int],)
>>> types.prepare_class('A', (int,), {'metaclass': typing.Type[int]})
(typing.Type[int], {}, {})
>>> types.prepare_class('A', (int,), {'metaclass': type[int]})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/serhiy/py/cpython/Lib/types.py", line 125, in prepare_class
meta = _calculate_meta(meta, bases)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/serhiy/py/cpython/Lib/types.py", line 139, in _calculate_meta
if issubclass(base_meta, winner):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: issubclass() argument 2 cannot be a parameterized generic
>>> @functools.singledispatch
... def g(a): pass
...
>>> @g.register
... def g2(a: typing.List[int]): pass
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/home/serhiy/py/cpython/Lib/functools.py", line 863, in register
raise TypeError(
^^^^^^^^^^^^^^^^
TypeError: Invalid annotation for 'a'. typing.List[int] is not a class.
>>> @g.register(list[int])
... def g2(a): pass
...
>>> @g.register
... def g3(a: typing.List[int]): pass
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/home/serhiy/py/cpython/Lib/functools.py", line 863, in register
raise TypeError(
^^^^^^^^^^^^^^^^
TypeError: Invalid annotation for 'a'. typing.List[int] is not a class.
>>> @g.register
... def g3(a: list[int]): pass
... And many other examples, too many to show here.
Was it mistake to make isinstance(list[int], type) returning True? martinitus commented on Oct 25, 2021 Can confirm for 3.9.7 as well. martinitus commented on Oct 25, 2021 I just created a PR and signed the contributor agreement. Waiting for it to update :-) Comments welcome! serhiy-storchaka commented on Oct 25, 2021 There are two ways to fix the larger issue.
-
Make issubclass(types.GenericAlias, type) returning True, and also make isinstance(typing.List[int], type) returning True and issubclass(typing._GenericAlias, type) returning True, and analyze every place in the interpreter and the stdlib which calls isinstance(..., type) or issubclass(..., type) and ensure that they work with types.GenericAlias and typing._GenericAlias and their instances as well as with ordinary types, fix them and add tests for them. And perhaps do the same for types.UnionType, typing._UnionGenericAlias, typing.TypeVar, typing.NewType, etc, etc.
-
Make isinstance(list[int], type) returning False. It would be nice to ad also tests for the same cases as in option 1, but it is not so urgent, and I expect that in most cases the current behavior which matches the status quo is expected.
I tried to implement option 1, but it is just too much places, so it would take a large amount of time which I do not have right now. First than invest my time in this I want to make sure which option is desirable in long term.
Guido, Ivan, Ken Jin, what would you say? ilevkivskyi commented on Oct 25, 2021
Was it mistake to make isinstance(list[int], type) returning True?
What was the motivation for this? At first glance returning True looks wrong. gvanrossum commented on Oct 25, 2021 I really don't recall if we even seriously considered what isinstance(list[int], type) should return. PEP-585 doesn't mention it. I presume it falls out of the way it's being tested and the way list[int] passes most attribute requests on to the origin (i.e., to list).
Given that this has returned True for two releases now I'm very reluctant to changing this now. So our best option is pursuing Serhiy's option 1, at least partially. The fix for the OP should be simple enough, right? serhiy-storchaka commented on Oct 25, 2021 Two years is not so long for a bug. We fixed 8-year and 12-year bugs.
The issue is that this feature is internally inconsistent (isinstance() is not consistent with issubclass()), the C implementation of list[int] is not consistent with the Python implementation of List[int], and a lot of code was not tested with this feature, so we perhaps have a lot of bugs in the stdlib and in the third-party libraries. Are we going to backport changes for making code working with GenericAlias as a type to 3.9 and 3.10? If not, we will actually add new features in 3.11 and keep 3.9 and 3.10 broken. If yes, these changes can have larger effect than just making isinstance(list[int], type) returning False, and they can break more code.
Note also that isinstance(List[int], type) was True before 3.7, and we intentionally made it False in 3.7 (it was required significant rewriting of the typing module and introducing __mro_entries__). Do we want to revert this decision? gvanrossum commented on Oct 27, 2021 serhiy-storchaka commented on Oct 28, 2021
issubclass(x, list[int]) rejects the second argument for reasons explained in the PEP.
-
One problem is that isinstance(x, type) != issubclass(type(x), type)
if x is list[int]. It is unprecedented, I cannot recall any other case
in which isinstance() and issubclas() are inconsistent. Ant it breaks
code because these two expressions are often considered equivalent and
interchangeable in refactoring.
-
Other problem is that isinstance(x, type) is used as a guard before
using x as a type. isinstance(obj, type) and issubclass(obj, SomeClass) is a common idiom, because issubclass() raises an exception
if its first argument is not a type. It is now broken for list[int].
What other places are there that are broken because of this?
$ find Lib -name '*.py' \! -path '*/test*' -exec egrep 'isinstance.*,
type\)' '{}' + | wc -l
55
In msg403826 I showed few examples from just two files, but there are
tens more potential examples. I'll show them all if I have enough of
spare time. serhiy-storchaka commented on Oct 28, 2021 gvanrossum commented on Oct 28, 2021 OK, OK, I see your point. I'm curious what Łukasz thinks. serhiy-storchaka commented on Oct 29, 2021 I have opened separate issues for different cases and a meta-issue45665 for general discussion. dss010101 commented on Oct 18, 2022 is there a recommended workaround until this is fixed?
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|