GeezerVison is showing github.com — Typing: improve error messages for invalid `ParamSpec` substitutions · Issue #102725 · python/cpython
Typing: improve error messages for invalid ParamSpec substitutions #102725DescriptionFeature or enhancement
PEP-612 is a complex typing PEP that introduced some complex rules, both for static type checkers and for CPython at runtime. It can be hard to remember the intricacies of all these rules -- both for users and for typing maintainers.
One thing that might help here is if we had better error messages in situations where the runtime raises exceptions. For example, in the following snippet, the runtime is arguably correct in raising TypeErrors -- by the rules laid out in PEP-612, both of these are invalid substitutions for two reasons:
- Parameters can only be prepended to a
ParamSpec in a parameters list using typing(_extensions).Concatenate.
- Appending parameters to a
ParamSpec in a parameters list is disallowed.
However, the error messages don't mention either of these things, and are in fact pretty baffling:
Running PGUpdate|x64 interpreter...
Python 3.12.0a6+ (heads/main:12226bec25, Mar 10 2023, 17:32:23) [MSC v.1932 64 bit (AMD64)] on win32
>>> import typing as t, collections.abc as c
>>> P, T = t.ParamSpec("P"), t.TypeVar("T")
>>> c.Callable[P, T][[P, str], bool][int]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<frozen _collections_abc>", line 501, in __getitem__
File "<frozen _collections_abc>", line 456, in __new__
TypeError: Callable must be used as Callable[[arg, ...], result].
>>> class MyCallable(t.Generic[P, T]): ...
...
>>> MyCallable[P, T][[P, str], bool][int]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\alexw\coding\cpython\Lib\typing.py", line 354, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\alexw\coding\cpython\Lib\typing.py", line 1403, in __getitem__
raise TypeError(f"{self} is not a generic class")
TypeError: __main__.MyCallable[[~P, str], bool] is not a generic classPrevious discussion
This issue is a followup to some points raised in:
Cc. @sobolevn Activitysobolevn commented on Apr 11, 2023 Update: turns out the later substitution does not raise an error anymore on main:
import typing as t, collections.abc as c
P, T = t.ParamSpec("P"), t.TypeVar("T")
class MyCallable(t.Generic[P, T]): ...
print(MyCallable[P, T][[P, str], bool][int])
# __main__.MyCallable[[[int], str], bool]Not good!
I think that this error is helpful and should be restored with a better message. sobolevn commented on Apr 11, 2023 We can now even substitute T with [P, int]:
import typing as t, collections.abc as c
P, T, T1 = t.ParamSpec("P"), t.TypeVar("T"), t.TypeVar("T1")
class MyCallable(t.Generic[T1, T]): ...
print(MyCallable[T, T1][[P, str], bool])
# __main__.MyCallable[[~P, <class 'str'>], bool]Not good at all! sobolevn commented on Apr 11, 2023 Should we fix it? It will require quite a bit of work to get all the things right.
I can do it if we need it. Or we can keep this up to a user to control. AlexWaygood commented on Apr 11, 2023 If it can be fixed without excessively complicating the code of typing.py, and if we can be 100% sure that we're not introducing false-positive errors, then I'll be happy to review a PR. But our general position is that it's better for the runtime to err on the side of leniency, and leave it to type checkers to point out invalid uses of typing constructs. We've had lots of situations in the past where strict checks at runtime have made typing_extensions backports complicated, or interfered with people using type annotations in unusual/experimental ways. Therefore, I view these "false negatives" at runtime as being quite low-priority concerns. sobolevn commented on Apr 11, 2023 Let's start with the second example.
The change itself is quite easy:
if (isinstance(old_arg, ParamSpec)
and isinstance(new_arg, tuple)
and any(isinstance(na, ParamSpec) for na in new_arg)):
raise TypeError(
'Cannot replace a ParamSpec with '
'a tuple containing other ParamSpec, '
'use Concatenate[] instead'
)
elif self.__origin__ == collections.abc.Callable and isinstance(new_arg, tuple):But, it has several flaws.
collections.abc.Callable does not do the same- It solves a very specific problem most users don't have
- Type-checker can catch this just fine: https://mypy-play.net/?mypy=latest&python=3.11&gist=627d3f8590da64c7c9e077067fd1b8e8
main.py:6: error: Invalid location for ParamSpec "P" [valid-type]
main.py:6: note: You can use ParamSpec as the first argument to Callable, e.g., 'Callable[P, int]'
So, I think we should leave this as-is.
Now, let's get back to your first example.
>>> import typing as t, collections.abc as c
>>> P, T = t.ParamSpec("P"), t.TypeVar("T")
>>> c.Callable[P, T][[P, str], bool][int]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<frozen _collections_abc>", line 501, in __getitem__
File "<frozen _collections_abc>", line 456, in __new__
TypeError: Callable must be used as Callable[[arg, ...], result].It is also problematic, because right now typing.Callable does not produce this error:
>>> import typing as t, collections.abc as c
>>> P, T = t.ParamSpec("P"), t.TypeVar("T")
>>> t.Callable[P, T][[P, str], bool][int]
typing.Callable[[int, str], bool]Synchronizing them will be a huge pain.
Because the exception happens on [int] substitution. But, we have to make an invalid substitution here c.Callable[P, T][[P, str], bool] to be able to reach this state.
So, I think we should just close this issue.
There's nothing that can be easily done to solve a hypothetical issue.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|