GeezerVison is showing github.com — 3.14 regression: `Union[int, str] is Union[int, str]` does not hold · Issue #131933 · python/cpython
3.14 regression: Union[int, str] is Union[int, str] does not hold #131933Labels 3.14bugs and security fixesbugs and security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytopic-typingtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error Activityhugovk commented on Mar 31, 2025 Reacted by Miro Hrončok and Srinivas Reddy Thatiparthy (తాటిపర్తి శ్రీనివాస్ రెడ్డి) changed the title [-]Python 3.14 regression: Union[int, str] is Union[int, str] no more[/-] [+]3.14 regression: `Union[int, str] is Union[int, str]` does not hold[/+] on Mar 31, 2025picnixz commented on Mar 31, 2025 JelleZijlstra commented on Mar 31, 2025 I think that test is incorrect and I posted python-attrs/cattrs#642 to fix it. We never guaranteed anything about the behavior of is with Union.
This does indicate a possible real-world issue that I hadn't considered, though. The Union[A, B] is Union[A, B] property held on earlier versions because we had a cache that deduplicated these objects, and that cache was added to avoid excessive memory usage if a program contains many instances of the same union. So it's possible that removing this cache will cause increased memory usage for some programs.
We could re-add a cache to the C implementation of typing.Union now. But here's a few reasons why I'm not sure that's necessary:
- Python 3.14 will also have deferred evaluation of annotations, meaning many programs will contain a lot fewer instances of Union objects.
- Adding the cache will have some cost too. It will obviously make union construction slower (we'd have to construct the cache key and check the cache dict). It will also increase memory usage because we have to put the object in a cache dictionary. A Union object is 72 bytes; to store it in the cache we'll have to make a weakref to it (80 bytes) and add an entry to the dictionary (which appears to add about 33 bytes per key on average). We also need to save the cache key, which likely means a new tuple because we have to use
@lru_cache(typed=True) or a moral equivalent; that's likely another 72 bytes or so. So if a particular union only appears in a program three or fewer times, we likely lose memory by adding the cache.
- New-style unions (
int | str) don't do the caching and never did (int | str is int | str was False even on 3.13). As far as I know nobody has complained about this, and several users running large Python codebases have already migrated to this style of union.
This makes me think that it's not worth re-adding the cache system that would make it so running Union[int, str] twice gives you the same object.
We can add a note to the What's New for 3.14 though pointing out this behavior change. Reacted by Miro Hrončok, Martin DeMello, Edgar Ramírez Mondragón and Victor Stinner JelleZijlstra commented on Apr 4, 2025 One more point around memory usage: new-style union objects are themselves much smaller. Old typing.Union had a __dict__, which was at least 272 bytes on 3.13; now on 3.14 the whole object is 72 bytes. Though admittedly the size of int | str got bigger from 3.13 to 3.14 (48 to 72 bytes) because we had to add some new pointers. JelleZijlstra commented on Apr 4, 2025 Oh and one more thing (in case someone else complains about this later): the Union[int, str] is Union[int, str] identity wasn't very reliable in 3.13 and earlier anyway, because it was based on an LRU cache. Therefore, if you create enough other unions in between, you'll get two different objects:
>>> from typing import Literal, Union
>>> lst = [Union[int, str], *[Union[str, Literal[i]] for i in range(1000)], Union[int, str]]
>>> lst[0]
typing.Union[int, str]
>>> lst[-1]
typing.Union[int, str]
>>> lst[0] is lst[-1]
False Reacted by Alex Waygood and Petr Viktorin
MetadataMetadata3.14bugs and security fixesbugs and security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytopic-typingtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error DevelopmentNo branches or pull requestsIssue actions
|