GeezerVison is showing github.com — annotationlib: `ref.evaluate(format=Format.FORWARDREF)` returns a ForwardRef with a copied `__globals__` that no longer updates · Issue #137969 · python/cpython
annotationlib: ref.evaluate(format=Format.FORWARDREF) returns a ForwardRef with a copied __globals__ that no longer updates #137969DescriptionBug report
Bug description:
This can happen internally in get_annotations and means subsequent attempts to evaluate it will fail even if the names have since been defined.
Underlying logic:
from annotationlib import get_annotations, Format
class Demo:
x: Sequence[undefined]
annos = get_annotations(Demo, format=Format.FORWARDREF)
x_anno = annos['x']
# Try to evaluate the reference, but just give a forwardref if it fails
x_repeat_anno = x_anno.evaluate(format=Format.FORWARDREF)
# The resulting reference no longer shares the globals namespace with the annotate function
print(f"{x_anno.__globals__ is Demo.__annotate__.__globals__ = }") # True
print(f"{x_repeat_anno.__globals__ is Demo.__annotate__.__globals__ = }") # False
# Define the previously undefined attributes
from collections.abc import Sequence
undefined = str
# This means evaluation fails in the second case
print(f"{x_anno.evaluate() = }") # collections.abc.Sequence[str]
print(f"{x_repeat_anno.evaluate() = }") # NameErrorThis evaluate call happens internally if get_annotations has to rely on the fallback behaviour for an unexpected exception, such as an AttributeError:
from annotationlib import get_annotations, Format
import typing
class Works:
a: Sequence[undefined]
b: unknowable
# Intentionally set up an annotation that will raise AttributeError on evaluation
class Fails:
a: Sequence[undefined]
b: typing.doesnotexist
a_works = get_annotations(Works, format=Format.FORWARDREF)['a']
a_fails = get_annotations(Fails, format=Format.FORWARDREF)['a']
# Realise the references
from collections.abc import Sequence
undefined = str
print(f"{a_works.evaluate() = }") # collections.abc.Sequence[str]
print(f"{a_fails.evaluate() = }") # NameErrorThis appears to be caused by the creation of a new globals dict here:
|
if type_params is not None: |
|
globals = dict(globals) |
|
for param in type_params: |
|
globals[param.__name__] = param |
Commenting this out makes these examples succeed, but obviously breaks type parameters.
CPython versions tested on:
CPython main branch, 3.14
Operating systems tested on:
No response
Linked PRs
Activitydr-carlos commented on Aug 22, 2025 A simple (possibly naive?) fix for this is just to add a new dictionary to store type param values instead of copying the globals.
Then, a check against this dict is added where globals is accessed later in evaluate(), logically between the locals and globals.
See dr-carlos@f8ac2c0
This seemingly fixes the provided example and works with type parameters (as far as my simple tests have shown).
Happy to open a PR, just wanting to make sure I haven't missed the problem (or created a new one). DavidCEllis commented on Aug 22, 2025 ContributorAuthor More actions Make sure you run the annotationlib tests with:
./python -m test -v test_annotationlib
It looks like your commit breaks the tests as you shadow the types module with your new variable which breaks the check a few lines up for isinstance(owner, types.ModuleType).
I thought I'd tried putting the type_params at the start of the locals and had test failures I didn't have time to investigate, but I think I had some logic in the wrong order. dr-carlos commented on Aug 22, 2025 Thanks for the heads up re the test command!
I've renamed the variable to something much more sensible (forgot I was going to do that before posting earlier 🤦 )
All tests are now passing. A new test would possibly be warranted for this use case, but that should probably be discussed in some future PR. JelleZijlstra commented on Aug 22, 2025 That sounds reasonable, please send a PR and I'll take a look. added a commit that references this issue on Nov 3, 2025gh-137969: Fix evaluation of `ref.evaluate(format=Format.FORWARDREF)`… JelleZijlstra commented on Nov 3, 2025 @dr-carlos your fix had to be reverted because of failures on some buildbots. Are you able to make another PR and try to fix the issues? dr-carlos commented on Nov 3, 2025
@dr-carlos your fix had to be reverted because of failures on some buildbots. Are you able to make another PR and try to fix the issues?
Yep, I'll take a look! Reacted by Jelle Zijlstra dr-carlos commented on Nov 4, 2025
@dr-carlos your fix had to be reverted because of failures on some buildbots. Are you able to make another PR and try to fix the issues?
It seems that the issue was with the test:
test_re_evaluate_generics tests globals, so uses a global declaration, but this shadows a name used in other test functions' scopes.- Renaming this variable to something unique fixed the other tests breaking
- For some reason, if you run the tests with the same command as the buildbots use, it seems that the global scope is shared across test runs? Anyway, if you just check whether the global is still set and then delete it, it fixes the issue.
Anyway, I've made a PR which re-implements the fix now that #138164 is implemented and includse a working test.
It's a slightly different approach to last time, but I haven't found any issues with it so far, so let me know if you'd prefer it implemented with a separate dict instead of injecting into locals. JelleZijlstra commented on Nov 4, 2025 Thanks, I think the difference is that buildbots run the same test twice in the same process. Should be able to repro with something like ./python -m test test_annotationlib test_annotationlib. added a commit that references this issue on Nov 13, 2025gh-137969: Fix double evaluation of `ForwardRef`s which rely on globa…
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|