GeezerVison is showing github.com — Higher-Kinded TypeVars · Issue #548 · python/typing
Higher-Kinded TypeVars #548Descriptionaka type constructors, generic TypeVars
Has there already been discussion about those?
I do a lot of FP that results in impossible situations because of this. Consider an example:
A = TypeVar('A')
B = TypeVar('B')
F = TypeVar('F')
class M(Generic[F[X], A]):
def x(fa: F[A], f: Callable[[A], B]) -> F[B]:
return map(f, fa)
M().x([1], str)I haven't found a way to make this work, does anyone know a trick or is it impossible?
If not, consider the syntax as a proposal.
Reference implementations would be Haskell, Scala.
optimally, the HK's type param would be indexable as well, allowing for F[X[X, X], X[X]]
Summary of current status (by @smheidrich, 2024-02-08):
- @JelleZijlstra has indicated interest in sponsoring a PEP, conditional on a prototype implementation in a major type checker and a well-specified draft PEP.
- Drafting the PEP takes place in @nekitdev's fork of the
peps repo. The stub PEP draft so far contains a few examples of the proposed syntax.
- That same repo's GitHub Discussions forum forum has been designated as the place to discuss the PEP (and presumably the prototype implementation?). Some limited further discussions have taken place there.
- If you want to be notified of new discussion threads, I think you have to set the whole repo as "watched" in GitHub?
Reacted by Taine Zhao, Massimo Redaelli, Ilia Korvigo, William Harvey, Max Kühn, Sascha Desch, ksonj, Aly Cerruti, Poscat, daa and 383 more Reacted by Felipe S. S. Schneider, Harald Husum, Randolf Scholz, Ellis Breen, Santiago Saavedra, milheht, Jonathan Plasse, Darkavid, Pragy Agarwal, Joren Hammudoglu and 12 more Reacted by Taine Zhao, bwo, Rodrigo H. Mota, ksonj, 林煒清(Lin Wei-Ching), Aly Cerruti, Sune Debel, Pradeep Kumar, gsedometov, twosigmajab and 74 more Reacted by Sunghyun Kim, Benoît Dubreuil, Justin Garcia, Las, Rory Hart, Felipe S. S. Schneider, Harald Husum, Lukas Kronsbein, Randolf Scholz, Byeonghoon Yoo and 23 more Reacted by Connor Newton, Justin Garcia, Felipe S. S. Schneider, Alex Waygood, Harald Husum, Randolf Scholz, Riccardo Bergamaschi, milheht, Jonathan Plasse, Julian Fortune and 17 more Activityilevkivskyi commented on Mar 30, 2018 I think this came up few times in other discussions, for example one use case is python/mypy#4395. But TBH this is low priority, since such use cases are quite rare. Reacted by Anton Agestam, martín and Ashrith Sagar Reacted by Matthew Hoffman, Rogério Chaves, Yi Zhou, Rafal Wojdyla, Tamika Nomara, Winston H., kazmi, Romain Yon, Lorenzo, Ivan Klass and 85 more Reacted by Предраг Николић tek commented on Mar 30, 2018 damn, I searched very thoroughly but did not find this one! 😄
So, consider this my +1! Reacted by Taine Zhao, Daniel Davis, Felipe S. S. Schneider, Harald Husum, Redowan Delowar, lapraswastaken, Mi!, Nyuan Zhang, Danipulok, Eldar S and 53 more gvanrossum commented on Mar 31, 2018 Adding your +1 is all fine, but who's going to do the (probably pretty
complicated) implementation work? Reacted by laike9m, Guilherme Caminha, Sebastian Wagner, Michael Panchenko, Danipulok, Jasha Sommer-Simpson, immanelg, Hash, Romir K, Winston H. and 9 more tek commented on Mar 31, 2018 are you approving the feature? gvanrossum commented on Mar 31, 2018 I am neither approving nor disapproving. Just observing that it may be a
lot of work for marginal benefits in most codebases. tek commented on Mar 31, 2018 awfully pragmatic. Where's your sense of adventure? 😄
anyways, I'll work on it, though it's gonna take a while to get into the project. Reacted by Landon Poch, Francisco Castro, Mike McDermott, Balázs Cene, Redowan Delowar, lapraswastaken, Jasha Sommer-Simpson, Ellis Breen, Paweł Rubin, benedikt-bartscher and 11 more landonpoch commented on Aug 14, 2018 Similar to microsoft/TypeScript#1213
Not sure if the discussion over there provides any useful insights to the effort over here. Reacted by Torsten Schmits, Mike McDermott, milheht, Díaz-Esparza and Alexander Schepanovski rcalsaverini commented on Jan 10, 2019 Hi @tek. I'm also very interested in this, so I'd like to ask if you had any progress with this and volunteer to help if you want. tek commented on Jan 10, 2019 @rcalsaverini sorry, I've been migrating my legacy code to haskell and am abandoning python altogether. but I wish you great success! Reacted by William Harvey, Profpatsch, Francisco Castro, hab, Mike McDermott, Dave Nicponski, Alexander Dmitriev, Balázs Cene, Harald Husum, Andrii Dmytruk and 15 more Reacted by Arseny Boykov, Ismail Bello, Feuermurmel, Shahriar Heidrich, Ilya Kamen, immanelg, Alexei Mochalov, Guillaume Desforges, Yann Hamdaoui, Lev Zlotnik and 15 more Reacted by Sunghyun Kim, Tin Tvrtković, Ellis Breen, London Lowmanstone and Axel PASCON rcalsaverini commented on Jan 10, 2019 Oh, sad to hear but I see your point. Thanks. syastrov commented on Oct 10, 2019 Just to add another use case (which I think relates to this issue):
Using Literal types along with overloading __new__, along with higher-kinded typevars could allow implementing a generic "nullable" ORM Field class, using a descriptor to provide access to the appropriate nullable-or-not field values. The descriptor wouldn't have to be reimplemented in subclasses.
It is one step closer to being possible due to the most recent mypy release's support for honoring the return type of __new__ (python/mypy#1020).
Note: this is basically a stripped-down version of Django's Field class:
# in stub file
from typing import Generic, Optional, TypeVar, Union, overload, Type
from typing_extensions import Literal
_T = TypeVar("_T", bound="Field")
_GT = TypeVar("_GT")
class Field(Generic[_GT]):
# on the line after the overload: error: Type variable "_T" used with arguments
@overload
def __new__(cls: Type[_T], null: Literal[False] = False, *args, **kwargs) -> _T[_GT]: ...
@overload
def __new__(cls: Type[_T], null: Literal[True], *args, **kwargs) -> _T[Optional[_GT]]: ...
def __get__(self, instance, owner) -> _GT: ...
class CharField(Field[str]): ...
class IntegerField(Field[int]): ...
# etc...
# in code
class User:
f1 = CharField(null=False)
f2 = CharField(null=True)
reveal_type(User().f1) # Expected: str
reveal_type(User().f2) # Expected: Union[str, None]Reacted by Samuel Colvin, Mike McDermott, Kuza, d-danilkin, Gleb Haranin, lapraswastaken, Théo, Serhii Tereshchenko, Albin Skott, benedikt-bartscher and 6 more samuelcolvin commented on Apr 26, 2020 I wonder if this is what I need or if there's currently a work around for my (slightly simpler) case?:
I'm building an async redis client with proper type hints. I have a "Commands" class with methods for all redis commands (get, set, exists, strlen ... and hundreds more). Normally each of those methods should return a future (actually coroutine) to the result, but in pipeline mode they should all return None - the commands are added to the pipeline to be executed later.
This is easy enough to implement in python, but not so easy to type hint correctly.
Basic example:
class Redis:
def execute(self, command) -> Coroutine[Any, Any, Union[None, str, int, float]]:
return self.connection.execute(...)
def get(self, *args) -> Coroutine[Any, Any, str]:
...
return self.execute(command)
def set(self, *args) -> Coroutine[Any, Any, None]:
...
return self.execute(command)
def exists(self, *args) -> Coroutine[Any, Any, bool]:
...
return self.execute(command)
# ... and many MANY more ...
class RedisPipeline(Redis):
def execute(self, command) -> None:
self.pipeline.append(command)I tried numerous options to make Coroutine[Any, Any, xxx] generic, but nothing seems to work.
Is there any way around this with python 3.8 and latest mypy? If not a solution would be wonderful - as far as I can think, my only other route for proper types is a script which copy and pastes the entire class and changes the return types in code. Reacted by Mike McDermott, Redowan Delowar and chun1 gvanrossum commented on Apr 26, 2020 @samuelcolvin I don't think this question belongs in this issue. The reason for the failure (knowing nothing about Redis but going purely by the code you posted) is that in order to make this work, the base class needs to switch to an Optional result, i.e.
def execute(self, command) -> Optional[Coroutine[Any, Any, Union[None, str, int, float]]]:
samuelcolvin commented on Apr 26, 2020 I get that, but I need all the public methods to definitely return a coroutine. Otherwise, if it returned an optional coroutine, it would be extremely annoying to use.
What I'm trying to do is modify the return type of many methods on the sub-classes, including "higher kind" types which are parameterised.
Hence thinking it related to this issue. RobinSobczyk commented on May 28, 2026 This also leads me to question the meaning of expressions like Concatenate[..., P]. More precisely, Concatenate seems to currently accept any number of Ellipsis and ParamSpecs, without any specified semantic. Am I missing an information there, or is it the currently desired behavior ?
More specifically, from what I understand, Concatenate should be kinded as ...[Type] -> PackedDomain -> PackedDomain (with ...[K] representing a variadic number of elements of kind K), and this should forbid those previously questionned uses. RobinSobczyk commented on Jun 19, 2026 Just a quick progress report, I'm splitting my PEP in 5 PEPs as it was getting really long due to a high number of things being required before properly defining higher kinded typevars. The PEPs will be as follow :
9995 : Defining kinds for actual types (TypeVars would still only represent only concrete types at this level)
9996 : Numerous extensions that naturally comes after defining kinds (like TypedDicts in Concatenate etc)
9997 : Reworking the TypeVar substitution system (defining TypeTemplates etc)
9998 : Adding Higher Kinded TypeVars with only basic features
9999 : Adding advanced higher kinded features (@lift decorator etc)
Again, no ETA, but I can give the current state of things :
9995 : Implementation missing
9996 : Incomplete, lot of things still missing (this is currently what I'm spending most of my time on)
9997 : Specification almost complete, no implementation
9998 : Most of rationale and specification is done, no implementation
9999 : Mostly ideas for now, require more in depth work
I plan on releasing 9995, 9996 and 9997 together at they feel quite connected to me. Then 9998, and finally 9999. Obviously will keep updating here sometimes (feel free to ping me if I don't give updates for too long). Reacted by Stephen Moore, Sebastian Wagner, djsigmann, Philipp A., niqzart, soufianeamini, Lorenzo, João Pedro, benedikt-bartscher and Ashrith Sagar Reacted by Sebastian Wagner, Jasper, João Pedro, Loïc Riegel, Ashrith Sagar and MaloL RobinSobczyk commented on Sep 22, 2026 Sorry for the long silence, I've been facing health problems in August and I'm still recovering, so I got really late in regards of where I wish I would have been currently. Obviously, health is my priority so project is on the back burner for now, but it's still in my plans, just extremely slowed down. I'm just really giving a sign of life here. Reacted by Jelle Zijlstra, Riccardo Bergamaschi, Anh71me, Harald Husum, Krzysztof Jagiello, Philipp A., Loïc Riegel, Jos Verlinde, Charles, niqzart and 8 more
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|