GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-41559: Implement PEP 612 - Add ParamSpec and Concatenate to typing by Fidget-Spinner · Pull Request #23702 · python/cpython

Conversation

@Fidget-Spinner

@Fidget-Spinner Fidget-Spinner commented Dec 8, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Changes:

  • Add ParamSpec and Concatenate to typing.
  • Allow both in Callable
  • Allow ParamSpec in Generic
  • Treat ParamSpec as a TypeVar (add them to __parameters__ ) in:
    • typing generics
    • types.GenericAlias
  • Generics parameterized with ParamSpec and subscripted with lists should cast those lists to tuple:
    • typing.Generic
    • types.GenericAlias
  • Tests for ParamSpec
    • Test specified use cases with typing.Callable.
    • Test specified use cases with collections.abc.Callable.
    • Test user defined generics
    • TypeVar like substitution
  • Tests for Concatenate
  • Loosened Generic and GenericAlias type checks to allow for nearly anything.

Documentation will be another patch.

https://bugs.python.org/issue41559

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

pradeep90 reacted with heart emoji
All reactions
  • ❤️ 1 reaction
Fidget-Spinner added 2 commits December 9, 2020 01:04
@Fidget-Spinner
Add ParamSpec and Concatenate
219b4ee
@Fidget-Spinner
support ParamSpec in generics
a1c0d0a
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Dec 8, 2020
@bedevere-bot bedevere-bot added the awaiting review label Dec 8, 2020
Fidget-Spinner added 2 commits December 10, 2020 00:07
@Fidget-Spinner
Add typing tests, disallow Concatenate in other types
7b3beab
@Fidget-Spinner
Add news
5dd3b44
@Fidget-Spinner
Fidget-Spinner marked this pull request as ready for review December 9, 2020 16:12
@Fidget-Spinner
Fidget-Spinner requested review from gvanrossum and ilevkivskyi as code owners December 9, 2020 16:12
gvanrossum
gvanrossum reviewed Dec 9, 2020
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry, I didn't get to the end of this review (not by far), but I have a few comments. Main thing though is, did you look at all the examples in PEP 612? It seems that for user-defined generic classes that have a ParamSpec parameter, things like X[int, P] are actually valid.

Maybe this indirectly also answers your question: I think that the ParamSpec should be included in __parameters__.

Also, I wonder if we could do less type checking at runtime rather than trying to be strict? I desperately want typing.py to shrink, not grow... :-)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner

Fidget-Spinner commented Dec 9, 2020

Copy link
Copy Markdown
Member Author

Points taken. Sorry X[int, P] slipped my mind since I hadn't fixed __parameters__ yet.

D'oh, I just realised I answered my own question since we need __parameters__ to implement X[int, P] in the first place.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner
Fidget-Spinner marked this pull request as draft December 10, 2020 13:42
Fidget-Spinner and others added 4 commits December 10, 2020 22:34
@Fidget-Spinner @gvanrossum
Address some of Guido's review comments
59c0b20
Co-Authored-By: Guido van Rossum <[email protected]>
@Fidget-Spinner
remove extraneous empty lines
4c381b3
@Fidget-Spinner
Support ParamSpec in __parameters__ of typing and builtin GenericAlias
b36b62d
also reduced runtime checks
@Fidget-Spinner
add tests for user defined generics
d09d088
@gvanrossum

gvanrossum commented Dec 11, 2020

Copy link
Copy Markdown
Member

Let's wait until #23060 has landed and then see what adjustments this might need.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Fidget-Spinner added 6 commits December 14, 2020 22:53
@Fidget-Spinner
Merge remote-tracking branch 'upstream/master' into pep612
0a19f34
@Fidget-Spinner
cast list to tuple done, loosened type checks for Generic
9727e2a
@Fidget-Spinner
loosen generics, allow typevar-like subst, flatten out args if Callable
cc7fc1c
@Fidget-Spinner
fix whitespace issue, cast list to tuple for types.GenericAlias
3e67f23
@Fidget-Spinner
convert list to tuples if substituting paramspecs in types.GenericAlias
d9baa1a
@Fidget-Spinner
done! flattened __args__ in substitutions for collections.abc.Callable
c4155b6
@Fidget-Spinner
Fidget-Spinner marked this pull request as ready for review December 16, 2020 09:45
@Fidget-Spinner
Fidget-Spinner requested a review from rhettinger as a code owner December 16, 2020 09:45
Fidget-Spinner added 3 commits December 16, 2020 22:02
@Fidget-Spinner
fix repr problems, add repr tests
2dbf861
@Fidget-Spinner
Add another test for multiple chaining
87c2d19
@Fidget-Spinner
fix typo
2b09de6
@Fidget-Spinner
Fidget-Spinner marked this pull request as draft December 17, 2020 15:39
@Fidget-Spinner
Clean up some comments
d980702
@Fidget-Spinner
remove stray whitespace
d6f777c
@gvanrossum

gvanrossum commented Dec 23, 2020

Copy link
Copy Markdown
Member

Finally looking at this...

Actually I just realized PEP 612 doesn't explicitly say that ParamSpec should support TypeVar-like substitution. Maybe I over complicated things a little by allowing that in types.GenericAlias (just for collections.abc.Callable)?

Yeah, ParamSpec is super special, and you can't do anything with it that isn't explicitly mentioned in the PEP. So it's debatable whether Callable[P, int] should even have P in its __parameters__. Then again, for the simple case A = Callable[P, int] there's no reason why we couldn't allow A[[str, str]] and get Callable[[str, str], int] as the result. But type checkers wouldn't know what to do with that -- ParamSpec is a magic cookie.

So let's not worry about those clearly wrong cases.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gvanrossum

gvanrossum commented Dec 23, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Sorry, submitted too soon. (UPDATE: Not, I made a mistake -- or GitHub did. :-)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gvanrossum
gvanrossum reviewed Dec 23, 2020
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work! Thanks for your perseverance. My feedback is mostly simple improvements.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/_collections_abc.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/_collections_abc.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/_collections_abc.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Objects/genericaliasobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Objects/genericaliasobject.c Outdated
}
// convert all the lists inside args to tuples to help
// with caching in other libaries if substituting a ParamSpec
if (PyList_CheckExact(subst) && is_paramspec(arg)) {

@gvanrossum gvanrossum Dec 23, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why only do this for ParamSpec?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@Fidget-Spinner Fidget-Spinner Dec 23, 2020

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm afraid of existing code relying on substituting typevars in genericalias with lists, then expecting that list to be inside __args__. Although in Python 3.9 that should be invalid, and the IDE/type checker should warn the users about that already.

If I just convert list to tuples for all cases, the C code will become way simpler :).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@gvanrossum gvanrossum Dec 23, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There can't be much code yet relying about anything for GenericAlias -- it's new in 3.9. We could backport that particular behavior to 3.9.3 to ensure there won't be any more code relying on it. :-)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Objects/genericaliasobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Fidget-Spinner and others added 4 commits December 23, 2020 13:06
@Fidget-Spinner @gvanrossum
Address nearly all of Guido's reviews
9a8176b
Co-Authored-By: Guido van Rossum <[email protected]>
@Fidget-Spinner
more reviews; fix some docstrings, clean up code, cast list to tuple …
fa06838
…by default
@Fidget-Spinner
remove uneeded tests copied over from typevar
b8672cd
@Fidget-Spinner
remove unused variable
51a463c
gvanrossum
gvanrossum reviewed Dec 23, 2020
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re: the 3.9 issues that you discovered: maybe create a PR for those first, land that in master and 3.9 branch, then merge master into this PR?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Fidget-Spinner reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread Lib/_collections_abc.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner

Fidget-Spinner commented Dec 24, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Guido, thank you so much for all the time you've put into reviewing this PR! I learnt quite a bit about the typing module in the process :).

Re: the 3.9 issues that you discovered: maybe create a PR for those first, land that in master and 3.9 branch, then merge master into this PR?

Yeah I agree, I created #23915 for that. Once that one lands, I'll merge the changes in along with some of your suggestions.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Fidget-Spinner added 2 commits December 24, 2020 10:57
@Fidget-Spinner
Merge remote-tracking branch 'upstream/master' into pep612
6d5b754
@Fidget-Spinner
merge length checking into _has_special_args too
c05d5d7
gvanrossum
gvanrossum approved these changes Dec 24, 2020
View reviewed changes
Comment thread Lib/_collections_abc.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added awaiting merge and removed awaiting review labels Dec 24, 2020
@gvanrossum
Update Lib/_collections_abc.py
c49ba30
@gvanrossum

gvanrossum commented Dec 24, 2020

Copy link
Copy Markdown
Member

I think it’s ready!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner Fidget-Spinner mentioned this pull request Dec 24, 2020
PEP 612: Clean up code examples, link to CPython implementation python/peps#1745
Merged
@gvanrossum

gvanrossum commented Dec 24, 2020

Copy link
Copy Markdown
Member

@Fidget-Spinner Let me know if you agree.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner

Fidget-Spinner commented Dec 24, 2020

Copy link
Copy Markdown
Member Author

@Fidget-Spinner Let me know if you agree.

Yes! Sorry, I thought you were going to merge it in so I was waiting too 😆 .

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gvanrossum

gvanrossum commented Dec 24, 2020

Copy link
Copy Markdown
Member

I figured that was the case. :-)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gvanrossum
gvanrossum merged commit 73607be into python:master Dec 24, 2020
@bedevere-bot

bedevere-bot commented Dec 24, 2020

Copy link
Copy Markdown

@gvanrossum: Please replace # with GH- in the commit message next time. Thanks!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting merge label Dec 24, 2020
@Fidget-Spinner

Fidget-Spinner commented Dec 24, 2020

Copy link
Copy Markdown
Member Author

@gvanrossum Oh dang I knew I had a hunch I missed something :/. I forgot to convert all lists to tuples in ga_new/setup_ga too (I did it only for getitem and co.). Well on the bright side, having that as a separate PR makes backporting to 3.9 easier if anything :), and anyways lists aren't valid in genericalias other than the Callable case (which we already have covered), so we won't have anything broken even without that in right now.

gvanrossum reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner

Fidget-Spinner commented Dec 24, 2020

Copy link
Copy Markdown
Member Author

I'll fix that some other time, right now I'm celebrating 🥳

gvanrossum and askurihin reacted with hooray emoji
All reactions
  • 🎉 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner
Fidget-Spinner deleted the pep612 branch December 24, 2020 09:19
@Fidget-Spinner Fidget-Spinner mentioned this pull request Dec 30, 2020
bpo-41559: Documentation for PEP 612 #24000
Merged
@Fidget-Spinner Fidget-Spinner mentioned this pull request Feb 14, 2021
bpo-43224: Initial implementation of PEP 646 in typing.py #24527
Closed
adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 13, 2021
@Fidget-Spinner @adorilson
bpo-41559: Implement PEP 612 - Add ParamSpec and Concatenate to typing (
41127ae
python#23702)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gvanrossum gvanrossum gvanrossum approved these changes
@ilevkivskyi ilevkivskyi Awaiting requested review from ilevkivskyi

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@Fidget-Spinner @gvanrossum @bedevere-bot @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.