GeezerVison is showing github.com — Cross-module dataclass inheritance breaks get_type_hints · Issue #89687 · python/cpython
Cross-module dataclass inheritance breaks get_type_hints #89687Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/ericvsmith'
closed_at = None
created_at = <Date 2021-10-19.14:37:47.448>
labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
title = 'Cross-module dataclass inheritance breaks get_type_hints'
updated_at = <Date 2022-01-22.08:37:13.515>
user = 'https://bugs.python.org/aidanbclark' bugs.python.org fields:
activity = <Date 2022-01-22.08:37:13.515>
actor = 'AlexWaygood'
assignee = 'eric.smith'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-10-19.14:37:47.448>
creator = 'aidan.b.clark'
dependencies = []
files = []
hgrepos = []
issue_num = 45524
keywords = ['patch']
message_count = 13.0
messages = ['404307', '404312', '404313', '404314', '404321', '404740', '404752', '404754', '404812', '404822', '404823', '404907', '404918']
nosy_count = 8.0
nosy_names = ['gvanrossum', 'eric.smith', 'JelleZijlstra', 'slebedev', 'sobolevn', 'kj', 'AlexWaygood', 'aidan.b.clark']
pr_nums = ['29158']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45524'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11'] Linked PRs
Reacted by Tuukka Mustonen Activityaidanbclark commented on Oct 19, 2021 MannequinAuthor More actions [I believe this is fundamentally a dataclass version of https://bugs.python.org/issue41249]
When using from __future__ import annotations, calling get_type_hints on the constructor of a dataclass B which inherits from a dataclass A defined in another module will error if dataclass A has type hints which are not imported in the module where dataclass B is defined.
This is best shown by example, if you have foo.py:
from __future__ import annotations
import collections
import dataclasses
@dataclasses.dataclass
class A:
x: collections.OrderedDict
and then in bar.py:
from __future__ import annotations
import foo
import dataclasses
import typing
@dataclasses.dataclass
class B(foo.A):
pass
typing.get_type_hints(B)
the final line will raise "NameError: name 'collections' is not defined".
This code will not error if you do either of the following:
- add
import collections to bar.py.
- remove the future annotations import from both files.
I am not confident enough on the internals of dataclass to suggest a fix, but potentially a similar approach to that which solved the TypedDict equivalent https://bugs.python.org/issue41249 would work? AlexWaygood commented on Oct 19, 2021 I can't reproduce this on Python 3.8.3, 3.9.6, 3.10.0 or 3.11.0a1+. Which versions of Python have you tried this on? (I'm able to reproduce the typing.TypedDict bug on Python 3.8, since the fix was only backported to 3.9, but not this.) slebedev commented on Oct 19, 2021 I think the example has a minor typo.
The crash is reproducible on 3.9 if the last line in bar.py is
typing.get_type_hints(B.__init__) instead of
Reacted by Tuukka Mustonen AlexWaygood commented on Oct 19, 2021 Thanks @sergei. With that alteration, I have reproduced this on Python 3.9, 3.10 and 3.11. aidanbclark commented on Oct 19, 2021 MannequinAuthor More actions Ah yes, I completely apologize about that typo, the __init__ is definitely needed (serves me right for trying to clean up a repo before posting it).
One other comment to make, perhaps obvious; manually passing a namespace to get_type_hints' globalns which contains collections does indeed solve the problem (in case anyone else sees this and is blocked). Of course determining the right values to pass is in general difficult for a user (and shouldn't be necessary). sobolevn commented on Oct 22, 2021 I had some time to debug this. It happens because we treat classes and functions differently.
The difference between get_type_hints(B) and get_type_hints(B.__init__) is that we use different global contexts there:
- For
B we use __mro__ entries and extract globals and locals from there:
|
for base in reversed(obj.__mro__): |
|
if globalns is None: |
|
base_globals = getattr(sys.modules.get(base.__module__, None), '__dict__', {}) |
|
else: |
|
base_globals = globalns |
- For
B.__init__ we use simplier logic
|
nsobj = obj |
|
# Find globalns for the unwrapped object. |
|
while hasattr(nsobj, '__wrapped__'): |
|
nsobj = nsobj.__wrapped__ |
|
globalns = getattr(nsobj, '__globals__', {}) |
|
if localns is None: |
|
localns = globalns |
It does not know anything about __mro__ and super contexts
Funny thing, this problem goes away if we remove @dataclass decorator and convert our examples into regular classes:
# a.py
from __future__ import annotations
import collections
class A:
x: collections.OrderedDict
def __init__(self, x: collections.OrderedDict) -> None:
...
and
# b.py
from __future__ import annotations
import a
import typing
class B(a.A):
pass
print(typing.get_type_hints(B))
# {'x': <class 'collections.OrderedDict'>}
print(typing.get_type_hints(B.__init__))
# {'x': <class 'collections.OrderedDict'>, 'return': <class 'NoneType'>}
I am going to try to solve this with something really simple (if no one else is working on it). AlexWaygood commented on Oct 22, 2021 @nikita, I had a go at writing some more rigorous tests regarding this issue. I found the same thing you did -- the issue seems:
- Isolated to dataclasses specifically (doesn't occur with TypedDicts, standard classes or NamedTuples)
- Isolated to the __init__ method of dataclasses
- Only occurs when you *subclass* a dataclass defined in another module.
My tests are in these two files on my cpython fork:
(I'm not proposing adding two new files to the cpython test suite -- just put the tests in new files so that I could isolate the new tests from the rest of the test suite and understand the problem better.) AlexWaygood commented on Oct 22, 2021 If you try running my test script with the from __future__ import annotations line at the top commented out, however, the error is the same. (from __future__ import annotations is obviously still there in the module that's being imported by the test script.) slebedev commented on Oct 22, 2021 Is it worth also addressing the case where a @dataclass/typing.TypeDict class is defined within a function?
from __future__ import annotations
import typing
from dataclasses import dataclass
def make_A():
import collections
@dataclass
class A:
x: collections.defaultdict
return A
A = make_A()
@dataclass
class B(A):
y: int
# NameError: name 'collections' is not defined
print(typing.get_type_hints(B.__init__))
AlexWaygood commented on Oct 22, 2021 Tishka17 commented on Jul 28, 2025 I've tried python3.14rc1 with annotationlib.get_annotations. It is still broken in the same way as it is with get_type_hints
Traceback (most recent call last):
File "/src/tmp/dtc/child.py", line 21, in <module>
print(annotationlib.get_annotations(Child.__init__, eval_str=True))
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.14/annotationlib.py", line 998, in get_annotations
key: value if not isinstance(value, str) else eval(value, globals, locals)
~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File "<string>", line 1, in <module>
NameError: name 'LocalDep' is not defined
JelleZijlstra commented on Jul 28, 2025 You are not showing a full code sample so I can't tell, but the problem may be that you're using from __future__ import annotations. The behavior under the future import is largely unchanged in Python 3.14. If you want good introspection, don't use from __future__ import annotations. Tishka17 commented on Jul 28, 2025 The current problem is:
a.py:
b.py:
import a
from annotationlib import ForwardRef
class A: pass
x = ForwardRef("A", module='a')
cls = x.evaluate(globals=globals(), locals=locals())
assert cls is a.AThis raises an assertion error JelleZijlstra commented on Jul 28, 2025 Tishka17 commented on Jul 28, 2025 It should not, I believe, because there is no other way to make forward ref bounded to module JelleZijlstra commented on Jul 28, 2025 It works if you just call x.evaluate() in your example without passing explicit globals().
I'm not sure why you'd expect passing a different module's globals would help evaluate the ForwardRef. Tishka17 commented on Jul 28, 2025 Because it's what's happening inside get_type_hints JelleZijlstra commented on Jul 28, 2025 I don't think so; its globals arguments defaults to None, not the calling module's globals(). Indeed, typing.evaluate_forward_ref(x) succeeds for your example too.
I realize this might cause problems though if you call get_type_hints() and need to pass globals=globals() for some ForwardRefs in there but not others. Not sure when that would happen though; I'd love self-contained examples where the current behavior doesn't work well. Tishka17 commented on Jul 28, 2025 a.py:
from dataclasses import dataclass
class A: pass
@dataclass
class D:
a: Ab.py:
import a
from typing import ForwardRef, evaluate_forward_ref, get_type_hints
from annotationlib import get_annotations, Format
from dataclasses import dataclass
class A: pass
@dataclass
class D2(a.D):
pass
print(get_type_hints(a.D.__init__))
print(a.D.__init__.__annotations__)
print(get_annotations(a.D.__init__, format=Format.VALUE))
print("D2")
print(get_type_hints(D2.__init__))
print(D2.__init__.__annotations__)
print(get_annotations(D2.__init__, format=Format.VALUE))Playing with order of A and D, if the annotation is string or not, I got different behavior, but it should not be different.
To fix this we should have a type hint holding module, but there isn't anything exceptForwardRef Tishka17 commented on Jul 29, 2025 What I see now:
- Attributes of ForwardRef are uncodumented https://docs.python.org/3.14/library/annotationlib.html#annotationlib.ForwardRef
- In cases ForwardRef has
owner or module set ( ForwardRef('A', is_class=True, owner=<class 'a.D'>)) it is resolved by get_type_hints ignoring information about it's origin.
- string annotations in class are not converted to
ForwardRef and __annotations__ strores them as is
TypeAliasType doesn't expect module argument and is immutable so cannot be used as a lazy proxy to another module- dataclasses copy annotations of all fields (including parent ones) to
__init__
So, the question is:
How can I create an annotation (in runtume) which is lazy and references to the same type even if used or resolved in another place? JelleZijlstra commented on Jul 30, 2025 As far as I can tell, things work correctly when creating a ForwardRef with an appropriate owner= and calling .evaluate(). Things also work correctly if you don't use string annotations.
What doesn't work well is get_type_hints(), because it's prone to passing in the wrong globals when it evaluates ForwardRefs. Unfortunately, that function has a lot of legacy behavior and is difficult to change. In user code, I'd recommend performing a simplified version of get_type_hints() directly, i.e., calling annotationlib.get_annotations(), converting any raw strings into ForwardRefs with the appropriate owner, and then calling ForwardRef.evaluate() as appropriate.
Dataclasses could do better at preserving the owner of field types in the case of inheritance. I think this patch would be an improvement:
diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py
index 53b3b54cfb3..e3d25fb0840 100644
--- a/Lib/dataclasses.py
+++ b/Lib/dataclasses.py
@@ -789,7 +789,10 @@ def _get_field(cls, a_name, a_type, default_kw_only):
# Only at this point do we know the name and the type. Set them.
f.name = a_name
- f.type = a_type
+ if isinstance(a_type, str):
+ f.type = annotationlib.ForwardRef(a_type, owner=cls)
+ else:
+ f.type = a_type
# Assume it's a normal field until proven otherwise. We're next
# going to decide if it's a ClassVar or InitVar, everything else
Tishka17 commented on Jul 31, 2025 Ok, setting a ForwardRef with owner right on field type sounds better, but anyway it does nothing if globals/locals are passed. Looks like the encapuslation here is broken.
I still have no idea about the idea behind the module/owner. From my perspecitve any ForwardRef should be bounded to a scope, so it can still access the same value in that scope. If ForwardRef is created within a specific module, it is expected to be solved using objects in that module. It is not a matter of solver but an origin where it is created.
Talking about this, I can say that I do not see much difference between TypeAliasType and ForwardRef, but you cannot create TypeAliasType instance providing another module so it cannot be used here.
If you have any links related the design of ForwardRef attributes, please share
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|