GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-82129: Provide `__annotate__` method for dataclasses from `make_dataclass` by sobolevn · Pull Request #122262 · python/cpython

Conversation

@sobolevn

@sobolevn sobolevn commented Jul 25, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This is my plan for python3.14+

While #122232 can be backported to older versions and solve their problems, this one is actually a correct way to solve this problem for the future.

This is a WIP, because I think that we should first decide on #122232

  • Issue: get_type_hints fails if there are un-annotated fields in a dataclass #82129

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@sobolevn
pythongh-82128: Provide __annotate__ method for dataclasses from `m…
c2d85b7
…ake_dataclass`
@sobolevn
sobolevn requested review from AlexWaygood and JelleZijlstra July 25, 2024 07:19
@bedevere-app bedevere-app Bot mentioned this pull request Jul 25, 2024
symtable_handle_namedexpr does not adjust correctly the recursion level #82128
Closed
@sobolevn

sobolevn commented Jul 25, 2024

Copy link
Copy Markdown
Member Author

Test failure:

 ======================================================================
FAIL: test_no_types (test.test_dataclasses.TestMakeDataclass.test_no_types)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\a\cpython\cpython\Lib\test\test_dataclasses\__init__.py", line 4111, in test_no_types
    self.assertEqual(C.__annotations__, {'x': 'typing.Any',
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                         'y': 'typing.Any',
                                         ^^^^^^^^^^^^^^^^^^
                                         'z': 'typing.Any'})
                                         ^^^^^^^^^^^^^^^^^^^
AssertionError: {'x': typing.Any, 'y': typing.Any, 'z': typing.Any} != {'x': 'typing.Any', 'y': 'typing.Any', 'z': 'typing.Any'}
- {'x': typing.Any, 'y': typing.Any, 'z': typing.Any}
+ {'x': 'typing.Any', 'y': 'typing.Any', 'z': 'typing.Any'}
?       +          +       +          +       +          +

is expected, I just don't want to touch tests at this point.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@sobolevn sobolevn changed the title gh-82128: Provide __annotate__ method for dataclasses from make_dataclass gh-82129: Provide __annotate__ method for dataclasses from make_dataclass Jul 25, 2024
@bedevere-app bedevere-app Bot mentioned this pull request Jul 25, 2024
get_type_hints fails if there are un-annotated fields in a dataclass #82129
Closed
@sobolevn sobolevn mentioned this pull request Jul 25, 2024
gh-82129: Fix NameError on get_type_hints in dataclasses #122232
Closed
@DavidCEllis

DavidCEllis commented Jul 25, 2024

Copy link
Copy Markdown
Contributor

I mentioned it on the issue but with this change typing is imported on class creation because the dataclass call will inspect the annotations and hence trigger the import. It also imports typing even if there are no untyped fields.

Python 3.14.0a0 (heads/main:5592399313, Jul 24 2024, 19:11:33) [MSC v.1938 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> from dataclasses import make_dataclass
>>> print(sys.modules.get("typing"))
None
>>> A = make_dataclass('A', [('a_var', str)])
>>> print(sys.modules.get("typing"))
<module 'typing' from '...\\cpython\\Lib\\typing.py'>

If you want to avoid the typing import this way you need to put some placeholder for dataclass to use and replace the attribute afterwards. Possibly also modifying Field.type to be a property that does the same replacement.

This doesn't happen on the 3.13 version proposed because dataclasses doesn't evaluate the strings so the import doesn't get called by dataclasses when it looks at the annotations.

sobolevn reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

JelleZijlstra
JelleZijlstra reviewed Jul 25, 2024
View reviewed changes
Comment thread Lib/dataclasses.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@JelleZijlstra

JelleZijlstra commented Jul 25, 2024

Copy link
Copy Markdown
Member

You might be able to get around @DavidCEllis's point by adding some internal-only flag to tell @dataclass to skip introspecting the annotations.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@DavidCEllis

DavidCEllis commented Jul 25, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Perhaps you can take advantage of dataclasses calling annotate with format.FORWARDREF and return something that looks like a ForwardRef there (or a real ForwardRef if there is a way to make this evaluate correctly if typing has been imported).

If it's called with format.VALUE you could then do the import and return the typing.Any object.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@sobolevn

sobolevn commented Jul 25, 2024

Copy link
Copy Markdown
Member Author

Perhaps you can take advantage of dataclasses calling annotate with format.FORWARDREF and return something that looks like a ForwardRef there (or a real ForwardRef if there is a way to make this evaluate correctly if typing has been imported).

Yes, this is my go-to idea right now :)

DavidCEllis reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

sobolevn added 2 commits September 17, 2024 22:45
@sobolevn
Merge branch 'main' into issue-82129-2
de409ad
@sobolevn
Fix NameError
642f849
@sobolevn
sobolevn marked this pull request as ready for review September 18, 2024 06:18
@sobolevn
sobolevn requested a review from ericvsmith as a code owner September 18, 2024 06:18
@bedevere-app bedevere-app Bot added the awaiting core review label Sep 18, 2024
@sobolevn
sobolevn requested a review from carljm September 18, 2024 06:18
@sobolevn

sobolevn commented Sep 18, 2024

Copy link
Copy Markdown
Member Author

This PR still does not account for the problem mentioned by @DavidCEllis
It still imports typing when dataclass is created, because dataclasses accesses .__annotations__ inside prepare_class function.

I will try to work around this problem.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@sobolevn

sobolevn commented Sep 18, 2024

Copy link
Copy Markdown
Member Author

But, since dataclasses.prepare_class sets __annotations__ in the created class, __annotate__ won't ever be called again, so I am not sure what to do.

cc @JelleZijlstra about my question in #122285

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

DavidCEllis
DavidCEllis reviewed Sep 18, 2024
View reviewed changes
Comment thread Lib/dataclasses.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

sobolevn added 2 commits September 20, 2024 10:17
@sobolevn
Merge branch 'main' into issue-82129-2
69251ac
@sobolevn
Address review
5ec3a08
@sobolevn

sobolevn commented Sep 20, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Now make_dataclass does not import typing on dataclass creation 🎉
And everything else seem to work just fine!

Thanks everyone for your help and ideas! 🤝

serjflint reacted with hooray emoji
All reactions
  • 🎉 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@sobolevn
sobolevn requested a review from JelleZijlstra September 20, 2024 07:22
JelleZijlstra
JelleZijlstra reviewed Sep 23, 2024
View reviewed changes
Comment thread Lib/dataclasses.py

def annotate_method(format):
typing = sys.modules.get("typing")
if typing is None and format == annotationlib.Format.FORWARDREF:

@JelleZijlstra JelleZijlstra Sep 23, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also avoid importing typing for the SOURCE format here I think; is that worth it?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

carljm and MandeepSinghPB08 reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

@sobolevn sobolevn Sep 24, 2024

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure we can. I need _convert_to_source, there can be complex annotations that should be formatted properly. I will open a new issue about converting annotations to string with public API though. Right now I don't see a clear way.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@sobolevn sobolevn Sep 24, 2024

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened #124412 for that.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@JelleZijlstra JelleZijlstra mentioned this pull request Sep 23, 2024
__annotate__ does not get called if __annotations__ exist #122285
Closed
carljm
carljm reviewed Sep 24, 2024
View reviewed changes

@carljm carljm left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks reasonable to me.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Misc/NEWS.d/next/Library/2024-09-18-09-15-40.gh-issue-82129.GQwt3u.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@sobolevn @carljm
Update Misc/NEWS.d/next/Library/2024-09-18-09-15-40.gh-issue-82129.GQ…
43caa3f
…wt3u.rst

Co-authored-by: Carl Meyer <[email protected]>
@sobolevn sobolevn mentioned this pull request Sep 24, 2024
How to convert annotations to Format.SOURCE in __annotate__? #124412
Closed
ericvsmith
ericvsmith reviewed Sep 24, 2024
View reviewed changes
Comment thread Lib/dataclasses.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@sobolevn
Introduce _ANY_MARKER
33b279d
@sobolevn
sobolevn requested review from JelleZijlstra and ericvsmith October 15, 2024 08:47
@sobolevn
Merge branch 'main' into issue-82129-2
d5ddb16
@sobolevn

sobolevn commented Oct 15, 2024

Copy link
Copy Markdown
Member Author

I realized there's now new way to convert annotations to source. Fixing!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@sobolevn sobolevn mentioned this pull request Oct 15, 2024
Incorrect format passed to __annotate__ function #125507
Closed
@sobolevn
WIP
3b26e36
@sobolevn
sobolevn marked this pull request as draft October 15, 2024 09:47
@bedevere-app bedevere-app Bot removed the awaiting core review label Oct 15, 2024
@sobolevn

sobolevn commented Oct 15, 2024

Copy link
Copy Markdown
Member Author

Blocked by #125507

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@JelleZijlstra
Merge branch 'main' into issue-82129-2
2c4d618
@JelleZijlstra JelleZijlstra mentioned this pull request May 4, 2025
gh-82129: Improve annotations for make_dataclass() #133406
Merged
@JelleZijlstra JelleZijlstra closed this in #133406 May 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@carljm carljm carljm left review comments
@DavidCEllis DavidCEllis DavidCEllis left review comments
@AlexWaygood AlexWaygood Awaiting requested review from AlexWaygood
@ericvsmith ericvsmith Awaiting requested review from ericvsmith ericvsmith is a code owner
@JelleZijlstra JelleZijlstra Awaiting requested review from JelleZijlstra

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@sobolevn @DavidCEllis @JelleZijlstra @carljm @ericvsmith
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.