GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-46333: Honor `module` parameter in ForwardRef by aha79 · Pull Request #30536 · python/cpython

Conversation

@aha79

@aha79 aha79 commented Jan 11, 2022 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

The module parameter carries semantic information about the forward ref.
Forward refs are different if they refer to different module even if they
have same name. This affects eq, repr and hash and methods,
which were previously not honoring the module parameter.

https://bugs.python.org/issue46333

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@andreas-h-sie
bpo-46333: Honor module parameter in ForwardRef
980ea28
The `module` parameter carries semantic information about the forward ref.
Forward refs are different if they refer to different module even if they
have same name. This affects __eq__, __repr__ and __hash__ and methods.
@aha79
aha79 requested review from Fidget-Spinner and gvanrossum as code owners January 11, 2022 15:28
@the-knights-who-say-ni the-knights-who-say-ni added the CLA not signed label Jan 11, 2022
@the-knights-who-say-ni

This comment was marked as resolved.

Sign in to view
@bedevere-bot bedevere-bot added the awaiting review label Jan 11, 2022
AlexWaygood
AlexWaygood reviewed Jan 11, 2022
View reviewed changes

@AlexWaygood AlexWaygood left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome to CPython! You can add a NEWS entry to your PR using https://blurb-it.herokuapp.com/ :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

aha79 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread Lib/test/test_typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

blurb-it Bot and others added 3 commits January 11, 2022 15:54
@blurb-it
📜🤖 Added by blurb_it.
82fa061
@aha79 @AlexWaygood
Update Lib/test/test_typing.py
ac51d98
Co-authored-by: Alex Waygood <[email protected]>
@aha79 @AlexWaygood
Update Lib/typing.py
d643861
Co-authored-by: Alex Waygood <[email protected]>
AlexWaygood
AlexWaygood reviewed Jan 11, 2022
View reviewed changes
Comment thread Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@aha79 @AlexWaygood
Update Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst
ac16c5c
Co-authored-by: Alex Waygood <[email protected]>
@AlexWaygood

AlexWaygood commented Jan 11, 2022

Copy link
Copy Markdown
Member

Since this is your first contribution, you should also feel free to add your name to https://github.com/python/cpython/blob/main/Misc/ACKS (this is entirely optional, only if you'd like to).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@andreas-h-sie
Update ACKS
a2e1377
@gvanrossum
gvanrossum removed their request for review January 11, 2022 18:19
@gvanrossum

gvanrossum commented Jan 11, 2022

Copy link
Copy Markdown
Member

@Fidget-Spinner I hope you can review and merge this PR?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner

Fidget-Spinner commented Jan 12, 2022

Copy link
Copy Markdown
Member

@Fidget-Spinner I hope you can review and merge this PR?

@gvanrossum Sure. Leave it to me.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Fidget-Spinner
Fidget-Spinner reviewed Jan 12, 2022
View reviewed changes

@Fidget-Spinner Fidget-Spinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! We're almost ready to merge, I just have some minor concerns.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_typing.py
fr = typing.ForwardRef('int')
self.assertEqual(fr, typing.ForwardRef('int'))
self.assertNotEqual(List['int'], List[int])
self.assertNotEqual(fr, typing.ForwardRef('int', module=__name__))

@Fidget-Spinner Fidget-Spinner Jan 12, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I wonder if this test is run directly, will __name__ be `main``? Wouldn't that make the test pass incorrectly? (Throwing this question out there because I'm not too sure myself). Maybe we should be safe and give it a distinct name?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@aha79 aha79 Jan 12, 2022

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test is to ensure that ForwardRef with module set is different from ForwardRef with module not set (That was previsously not the case). The actual value of module (or __name__) is not relevant. Under the hood the comparison is None vs str

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@Fidget-Spinner Fidget-Spinner Jan 12, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad, thanks for pointing that out. I mistakenly assumed it would grab a default value from the current scope to populate module in things like ClassVar['int'] due to the _type_check/_type_convert function (I'm now shocked that it doesn't and only does that for TypedDict).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@aha79 aha79 Jan 12, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, unfortunately the current module is not captured automatically.
The current scope/module is typicallly obtained by get_type_hints because a class knows its module (and so all ForwardRefs in member annotations can be correctly resolved. But each forward Ref does originally not know its module).
However for type aliases this does not work, and there one needs the explicit module.

For example

Json = Union[ List['Json'], Dict[str,'Json'], int, float, bool, None ]

cannot be exported (and expect to work) unless one sets the module for each forward ref.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@aha79 @Fidget-Spinner
Update Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst
c084fdb
Co-authored-by: Ken Jin <[email protected]>
@AlexWaygood

AlexWaygood commented Jan 12, 2022

Copy link
Copy Markdown
Member

@Fidget-Spinner, do you see this one as more of a bugfix that should be backported, or more of an enhancement/behaviour change? (This is just for my clarification, @aha79 — I'm fairly new to the triage team 🙂)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner

Fidget-Spinner commented Jan 12, 2022

Copy link
Copy Markdown
Member

@Fidget-Spinner, do you see this one as more of a bugfix that should be backported, or more of an enhancement/behaviour change? (This is just for my clarification, @aha79 — I'm fairly new to the triage team 🙂)

@AlexWaygood this is a grey area.

On one hand, generally, anything dealing with __eq__ and __hash__ in typing.py == bug. typing.py relies heavily on an LRU cache when subscripting things (see _tp_cache), so in the past, any hash/equality comparisons would mean incorrect values when using almost anything in typing.py, as Andreas rightly pointed out.

However, this specific module parameter is used only for differentiating different-module TypedDicts. TypedDict doesn't use the type cache, and different module TypedDict will automatically have different hash() values. So I don't know of anyone who's affected unless you use ForwardRef directly. But that's explicitly discouraged as an internal detail as I mentioned on the bug tracker.

I'm inclined to accept the PR without the __repr__ change, and I will likely backport it like that since it fixes something (albeit in a use case not really supported by us).

AlexWaygood reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood

AlexWaygood commented Jan 12, 2022

Copy link
Copy Markdown
Member

@AlexWaygood this is a grey area.

Thanks, that's really clarifying! It might be worth noting that @sobolevn proposes in #29158 (not yet merged) to use the module parameter of typing.ForwardRef to solve a bug relating to dataclasses and get_type_hints.

sobolevn reacted with eyes emoji
All reactions
  • 👀 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood

AlexWaygood commented Jan 14, 2022

Copy link
Copy Markdown
Member

Btw @aha79 — could I ask if you've signed the CLA? For legal reasons, it may be difficult for CPython to merge a PR from you if you haven't 🙂

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@aha79

aha79 commented Jan 14, 2022

Copy link
Copy Markdown
Contributor Author

Btw @aha79 — could I ask if you've signed the CLA? For legal reasons, it may be difficult for CPython to merge a PR from you if you haven't 🙂

Yes, sorry. I completely understand that. I clarify with my organization (This will be ok, but may take a few more days)

AlexWaygood reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood

AlexWaygood commented Jan 14, 2022

Copy link
Copy Markdown
Member

Yes, sorry. I completely understand that. I clarify with my organization (This will be ok, but may take a few more days)

Of course! No rush :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@the-knights-who-say-ni the-knights-who-say-ni added CLA signed and removed CLA not signed labels Feb 1, 2022
@aha79

aha79 commented Feb 9, 2022

Copy link
Copy Markdown
Contributor Author

Yes, sorry. I completely understand that. I clarify with my organization (This will be ok, but may take a few more days)

Of course! No rush :)

@AlexWaygood The CLA is now signed. Sorry that it took a while.
Should I undo the __ repr__ change as @Fidget-Spinner suggests? Not sure if this is a final descision.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood

AlexWaygood commented Feb 9, 2022

Copy link
Copy Markdown
Member

@AlexWaygood The CLA is now signed. Sorry that it took a while.

Thanks, no worries!

Should I undo the __ repr__ change as Fidget-Spinner suggests?

To me it makes sense to split off the __repr__ changes into a separate PR: the rest of the PR is clearly backportable, but the __repr__ changes seem like more of an enhancement.

Fidget-Spinner is currently on a break from open source — @JelleZijlstra, if you've got the time, do you have any thoughts on this PR?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@JelleZijlstra
JelleZijlstra self-requested a review February 9, 2022 15:06
aha79 and others added 2 commits February 11, 2022 09:01
@aha79 @AlexWaygood
Update Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst
42fc26b
Co-authored-by: Alex Waygood <[email protected]>
@andreas-h-sie
add tests for ForwardRef.hash
9dd2f8f
@aha79

aha79 commented Feb 11, 2022

Copy link
Copy Markdown
Contributor Author

Could you possibly add some tests for the changes to __hash__, as well as the changes to __eq__? (test_forward_equality_hash() looks like a good candidate for where tests might be added.)

There were already tests for __eq__. But I added some for __hash__ as well (even it would not strictly be required that __hash__ changes with module - inclusion of module in __hash__ just avoids hash collisions but does not change behavior)

AlexWaygood reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood

AlexWaygood commented Feb 11, 2022

Copy link
Copy Markdown
Member

Sorry, I think I was unclear — my bad :)

We now have two test methods for testing ForwardRef.__hash__ — the one you've just added, and the existing one here:

cpython/Lib/test/test_typing.py

Line 2648 in 9dd2f8f

def test_forward_equality_hash(self):
.

Could you move the new tests for ForwardRef.__hash__ that you've just added (which look good) into the existing test method?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood

AlexWaygood commented Feb 11, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There were already tests for __eq__.

I didn't mean to imply otherwise!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@andreas-h-sie
move ForwardRef.__hash__ test to correct function
94c9a69
@aha79

aha79 commented Feb 11, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Sorry, I think I was unclear — my bad :)

We now have two test methods for testing ForwardRef.__hash__ — the one you've just added, and the existing one here:

cpython/Lib/test/test_typing.py

Line 2648 in 9dd2f8f

def test_forward_equality_hash(self):

.
Could you move the new tests for ForwardRef.__hash__ that you've just added (which look good) into the existing test method?

Sorry I misunderstood. I move/adapted the tests to the right place

AlexWaygood reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

AlexWaygood
AlexWaygood approved these changes Feb 11, 2022
View reviewed changes

@AlexWaygood AlexWaygood left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Feel free to open a new PR for the __repr__ changes (you can link to the same BPO issue).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

aha79 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@AlexWaygood AlexWaygood added needs backport to 3.9 labels Feb 11, 2022
@AlexWaygood
AlexWaygood requested a review from JelleZijlstra February 11, 2022 10:17
JelleZijlstra
JelleZijlstra approved these changes Feb 11, 2022
View reviewed changes
@AlexWaygood

AlexWaygood commented Feb 11, 2022

Copy link
Copy Markdown
Member

@gvanrossum could we possibly get the full test suite run on this PR, please? :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@aha79 aha79 mentioned this pull request Feb 11, 2022
bpo-46333: include module in ForwardRef.__repr__ #31283
Merged
@gvanrossum

gvanrossum commented Feb 11, 2022

Copy link
Copy Markdown
Member

@AlexWaygood / @JelleZijlstra Is this ready to merge?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood

AlexWaygood commented Feb 11, 2022

Copy link
Copy Markdown
Member

@AlexWaygood / @JelleZijlstra Is this ready to merge?

Looks that way to me :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@JelleZijlstra

JelleZijlstra commented Feb 11, 2022

Copy link
Copy Markdown
Member

Yes I think it's good. It can wait though, hopefully I can merge it myself next week :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@JelleZijlstra
JelleZijlstra merged commit 6e7b813 into python:main Feb 17, 2022
@miss-islington

miss-islington commented Feb 17, 2022

Copy link
Copy Markdown
Contributor

Thanks @aha79 for the PR, and @JelleZijlstra for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9, 3.10.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting core review label Feb 17, 2022
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Feb 17, 2022
@aha79 @miss-islington
bpo-46333: Honor module parameter in ForwardRef (pythonGH-30536)
c1898ee
The `module` parameter carries semantic information about the forward ref.
Forward refs are different if they refer to different module even if they
have the same name. This affects the `__eq__`, `__repr__` and `__hash__` methods.

Co-authored-by: Andreas Hangauer <[email protected]>
Co-authored-by: Alex Waygood <[email protected]>
Co-authored-by: Ken Jin <[email protected]>
(cherry picked from commit 6e7b813)

Co-authored-by: aha79 <[email protected]>
@bedevere-bot bedevere-bot removed the needs backport to 3.10 label Feb 17, 2022
@bedevere-bot

bedevere-bot commented Feb 17, 2022

Copy link
Copy Markdown

GH-31379 is a backport of this pull request to the 3.10 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Feb 17, 2022
@aha79 @miss-islington
bpo-46333: Honor module parameter in ForwardRef (pythonGH-30536)
817fdc1
The `module` parameter carries semantic information about the forward ref.
Forward refs are different if they refer to different module even if they
have the same name. This affects the `__eq__`, `__repr__` and `__hash__` methods.

Co-authored-by: Andreas Hangauer <[email protected]>
Co-authored-by: Alex Waygood <[email protected]>
Co-authored-by: Ken Jin <[email protected]>
(cherry picked from commit 6e7b813)

Co-authored-by: aha79 <[email protected]>
@bedevere-bot bedevere-bot removed the needs backport to 3.9 label Feb 17, 2022
@bedevere-bot

bedevere-bot commented Feb 17, 2022

Copy link
Copy Markdown

GH-31380 is a backport of this pull request to the 3.9 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington added a commit that referenced this pull request Feb 17, 2022
@miss-islington
[3.10] bpo-46333: Honor module parameter in ForwardRef (GH-30536) (G…
a17d59a
…H-31379)

The `module` parameter carries semantic information about the forward ref.
Forward refs are different if they refer to different module even if they
have the same name. This affects the `__eq__`, `__repr__` and `__hash__` methods.

Co-authored-by: Andreas Hangauer <[email protected]>
Co-authored-by: Alex Waygood <[email protected]>
Co-authored-by: Ken Jin <[email protected]>
(cherry picked from commit 6e7b813)


Co-authored-by: aha79 <[email protected]>

Automerge-Triggered-By: GH:JelleZijlstra
miss-islington added a commit that referenced this pull request Feb 17, 2022
@miss-islington @aha79
bpo-46333: Honor module parameter in ForwardRef (GH-30536)
29ae7d3
The `module` parameter carries semantic information about the forward ref.
Forward refs are different if they refer to different module even if they
have the same name. This affects the `__eq__`, `__repr__` and `__hash__` methods.

Co-authored-by: Andreas Hangauer <[email protected]>
Co-authored-by: Alex Waygood <[email protected]>
Co-authored-by: Ken Jin <[email protected]>
(cherry picked from commit 6e7b813)

Co-authored-by: aha79 <[email protected]>
hello-adam pushed a commit to hello-adam/cpython that referenced this pull request Jun 2, 2022
@miss-islington @aha79 @hello-adam
bpo-46333: Honor module parameter in ForwardRef (pythonGH-30536)
c724729
The `module` parameter carries semantic information about the forward ref.
Forward refs are different if they refer to different module even if they
have the same name. This affects the `__eq__`, `__repr__` and `__hash__` methods.

Co-authored-by: Andreas Hangauer <[email protected]>
Co-authored-by: Alex Waygood <[email protected]>
Co-authored-by: Ken Jin <[email protected]>
(cherry picked from commit 6e7b813)

Co-authored-by: aha79 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Fidget-Spinner Fidget-Spinner Fidget-Spinner left review comments
@JelleZijlstra JelleZijlstra JelleZijlstra approved these changes
@AlexWaygood AlexWaygood AlexWaygood approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

9 participants

@aha79 @the-knights-who-say-ni @AlexWaygood @gvanrossum @Fidget-Spinner @JelleZijlstra @miss-islington @bedevere-bot @andreas-h-sie
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.