GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||
GeezerVison is showing github.com — `CallableFunction` as a drop-in replacement for `Callable` · python/typing · Discussion #2191
|
|
Not all I learned about this issue because of a function in my tests. The type below works as a replacement for my one function. See ty playground. But that doesn't mean much. I've mashed-up annotations from typeshed, then "refined" the code by running it through different LLMs and trying to separate the good advice from the bad advice. Frankly, this is beyond my ability to evaluate. The type below may be so flawed that I should start over. If this is a decent start, I don't know how to find the parts that need improvement. Therefore, I would love some feedback. """Protocol for callable functions with full type safety.""" from collections.abc import Callable from types import CellType, CodeType, MethodType from typing import Any, overload, ParamSpec, Protocol, runtime_checkable, Self, TypeVar, TypeVarTuple import sys #======== Stolen, uh, I mean copied from typeshed:stdlib\_typeshed\__init__.pyi ======== type AnnotationForm = Any if sys.version_info >= (3, 14): from annotationlib import Format # NOTE These return annotations, which can be arbitrary objects type AnnotateFunc = Callable[[Format], dict[str, AnnotationForm]] type EvaluateFunc = Callable[[Format], AnnotationForm] #======== End stolen, uh, I mean copied from typeshed:stdlib\_typeshed\__init__.pyi ======== @runtime_checkable class CallableFunction[**P, R](Protocol): """A Protocol representing callable functions with descriptor support. Mimics types.FunctionType while being a drop-in replacement for `collections.abc.Callable`. Includes all standard function attributes and the descriptor protocol for proper method binding. Note: @runtime_checkable only validates attribute presence, not signatures. """ # NOTE: The eehhhh, IDK... section __doc__: str | None __wrapped__: Any # Allegedly, for functools.wraps support # NOTE: End eehhhh, IDK... section @property def __closure__(self) -> tuple[CellType, ...] | None: """Tuple of cells that contain bindings for the function's free variables.""" ... __code__: CodeType __defaults__: tuple[Any, ...] | None __dict__: dict[str, Any] @property def __globals__(self) -> dict[str, Any]: """The global namespace in which the function was defined.""" ... __name__: str __qualname__: str __annotations__: dict[str, AnnotationForm] if sys.version_info >= (3, 14): __annotate__: AnnotateFunc | None __kwdefaults__: dict[str, Any] | None @property def __builtins__(self) -> dict[str, Any]: """The built-in namespace in which the function was defined.""" ... __type_params__: tuple[TypeVar | ParamSpec | TypeVarTuple, ...] __module__: str def __call__(self, *args: P.args, **kwargs: P.kwargs) -> R: """Execute the callable with the given arguments.""" ... @overload def __get__(self, instance: None, owner: type, /) -> Self: ... @overload def __get__(self, instance: object, owner: type | None = None, /) -> MethodType: ... def __get__(self, instance: object | None, owner: type | None = None) -> Self | MethodType: """Descriptor protocol for method binding. Returns self when accessed on the class, or a bound MethodType when accessed on an instance. """ ... |
There was an error while loading. Please reload this page.
Three cases when the type doesn't workty says invalid-assignment in all three cases. See playground. One function is decorated with functools.cache and Pylance says: Type "list[_lru_cache_wrapper[int] | ((dd: int, /) -> int)]" is not assignable to declared type "list[CallableFunction[(int), int]]" "_lru_cache_wrapper[int]" is incompatible with protocol "CallableFunction[(int), int]" "__closure__" is not present "__code__" is not present "__defaults__" is not present "__globals__" is not present "__name__" is not present "__annotate__" is not present "__kwdefaults__" is not present In the other two cases, the functions are not decorated, and Pylance objects to # Code direction: CallableFunction[[int], int] = pos if 0 <=index else neg # Diagnostic message Type "((a: _SupportsPos[_T_co@pos], /) -> _T_co@pos) | ((a: _SupportsNeg[_T_co@neg], /) -> _T_co@neg)" is not assignable to declared type "CallableFunction[(int), int]" Type "((a: _SupportsPos[_T_co@pos], /) -> _T_co@pos) | ((a: _SupportsNeg[_T_co@neg], /) -> _T_co@neg)" is not assignable to type "CallableFunction[(int), int]" "FunctionType" is incompatible with protocol "CallableFunction[(int), int]" "__wrapped__" is not present "CallableFunction[P@CallableFunction, R@CallableFunction]" is not assignable to "FunctionType" "CallableFunction[P@CallableFunction, R@CallableFunction]" is not assignable to "FunctionType" "CallableFunction[P@CallableFunction, R@CallableFunction]" is not assignable to "FunctionType" If I remove |
There was an error while loading. Please reload this page.
|
Python’s static typing system cannot currently infer a generic return type from an “indirect” factory method unless the type variable is directly tied to an input parameter. That’s a limitation of how type variables are propagated in Python’s type checkers and the language’s typing design. In particular: Type variables (TypeVar) are only meaningful to type checkers when they are tied to a function’s signature in a way the checker can follow — usually as function arguments or bound to a class. If you write a factory function that returns a type based on some internal logic without any connection between the return type and the signature’s type parameters, type checkers have no basis to infer the correct return type at call sites. Python’s typing does not support “higher kinded types” or type inference from indirect logic — this is something discussed frequently but unresolved in the core typing design. The canonical pattern that does work and is fully supported today looks like this: T = TypeVar("T") def create_instance(cls: Type[T]) -> T: In this pattern: The cls: Type[T] parameter connects the generic type variable T to an input parameter. Static checkers like mypy or PyCharm can then reason that create_instance(Foo) returns a Foo. Without this connection, no current type checker (including mypy, pyright, or the built-in stubs) can reliably infer the return type. So although Python’s typing library has evolved significantly, there is no built-in mechanism today that allows a factory function to magically infer its generic return type from indirect logic alone. The recommended workaround is to ensure the type variable appears in the function signature in a way type checkers understand. |
There was an error while loading. Please reload this page.
|
I think the main issue is that the protocol is trying to represent two different things: ordinary Python function objects and arbitrary callable objects.
In particular:
If your actual requirement is a callable with a from typing import Protocol class NamedCallable[**P, R](Protocol): @property def __name__(self) -> str: ... def __call__( self, *args: P.args, **kwargs: P.kwargs, ) -> R: ... This preserves the callable's parameter and return types while requiring only the additional attribute you need. For example: def double(value: int) -> int: return value * 2 fn: NamedCallable[[int], int] = double print(fn.__name__) print(fn(5)) However, this deliberately does not accept every If you need to support arbitrary callables, a fallback is more appropriate: from collections.abc import Callable from typing import Any def callable_name(fn: Callable[..., Any]) -> str: return getattr(fn, "__name__", type(fn).__name__) So I would avoid making This avoids depending on implementation-specific function attributes and makes the intended structural contract clearer. |