GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — `typing.Protocol` implementation means that `isinstance([], collections.abc.Mapping)` can sometimes evaluate to `True` · Issue #105280 · python/cpython

typing.Protocol implementation means that isinstance([], collections.abc.Mapping) can sometimes evaluate to True #105280

New issue
Copy link
New issue
Copy link
Closed
Closed
typing.Protocol implementation means that isinstance([], collections.abc.Mapping) can sometimes evaluate to True#105280
Copy link
Assignees
AlexWaygood
Labels
3.12only security fixesonly security fixes3.13only security fixesonly security fixesstdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytopic-typingtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Description

@AlexWaygood
AlexWaygood
opened on Jun 4, 2023
Issue body actions

(In very specific circumstances)

Bug report

Due to some changes that have been made to the implementation of typing.Protocol in Python 3.12, whether or not isinstance([], collections.abc.Mapping) evaluates to True or False now depends on precisely when and if garbage collection happens.

Minimal repro:

>>> import gc, collections.abc, typing
>>> gc.disable()
>>> try:
...     class Foo(collections.abc.Mapping, typing.Protocol): pass
... except TypeError:
...     pass
...
>>> isinstance([], collections.abc.Mapping)
True

Why does this happen?! It's because Foo is an illegal class, so TypeError is raised during class initialisation here:

cpython/Lib/typing.py

Lines 1905 to 1912 in ce558e6

# ... otherwise check consistency of bases, and prohibit instantiation.
for base in cls.__bases__:
if not (base in (object, Generic) or
base.__module__ in _PROTO_ALLOWLIST and
base.__name__ in _PROTO_ALLOWLIST[base.__module__] or
issubclass(base, Generic) and getattr(base, '_is_protocol', False)):
raise TypeError('Protocols can only inherit from other'
' protocols, got %r' % base)

TypeError being raised here means that Foo never has __protocol_attrs__ set on it. But, the error is raised after class construction, meaning that if garbage collection happens at the wrong time, a reference to Foo is still retained in collections.abc.Mapping.__subclasses__(). This then creates problems in the isinstance() check, because of some behaviour in the abc module where it iterates through all of the subclasses of collections.abc.Mapping in order to determine whether an object is an instance of collections.abc.Mapping.

>>> import gc, collections.abc, typing
>>> gc.disable()
>>> try:
...     class Foo(collections.abc.Mapping, typing.Protocol): pass
... except TypeError:
...     pass
...
>>> x = collections.abc.Mapping.__subclasses__()
>>> x
[<class 'collections.abc.MutableMapping'>, <class '__main__.Foo'>]
>>> x[-1].__protocol_attrs__
set()

The fix is to raise TypeError before the class has actually been created, rather than during class initialisation.

Why does this matter?

This might seem like an absurdly specific bug report, but it would be good to see it fixed. It was causing bizarre test failures in the typing_extensions backport. The issue was that we did this in one test method:

    def test_protocols_bad_subscripts(self):
        T = TypeVar('T')
        S = TypeVar('S')
        with self.assertRaises(TypeError):
            class P(typing.Mapping[T, S], Protocol[T]): pass

...And that final P class wasn't being cleaned up by the garbage collector immediately after that test having been run. That then caused an unrelated test elsewhere to fail due to the fact that isinstance([], collections.abc.Mapping) was evaluating to True, and it was very hard to figure out why.

Linked PRs

  • gh-105280: Ensure isinstance([], collections.abc.Mapping) always evaluates to False #105281
  • [3.12] gh-105280: Ensure isinstance([], collections.abc.Mapping) always evaluates to False (GH-105281) #105318
Reacted by James Hilton-Balfe

Activity

  1. AlexWaygood
    added
    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory
    topic-typing
    3.12only security fixesonly security fixes
    3.13only security fixesonly security fixes
    on Jun 4, 2023
  2. AlexWaygood
    self-assigned this
    on Jun 4, 2023
  3. AlexWaygood
    added a commit that references this issue on Jun 5, 2023

    gh-105280: Ensure `isinstance([], collections.abc.Mapping)` always ev…

    08756db
  4. miss-islington
    added a commit that references this issue on Jun 5, 2023

    pythongh-105280: Ensure `isinstance([], collections.abc.Mapping)` alw…

    c004e38
  5. AlexWaygood
    added a commit that references this issue on Jun 5, 2023

    [3.12] gh-105280: Ensure `isinstance([], collections.abc.Mapping)` al…

    2031238
  6. AlexWaygood commented on Jun 5, 2023

    @AlexWaygood
    AlexWaygood
    on Jun 5, 2023
    MemberAuthor
    More actions

    Let us never again talk about the brief period in time when a list could sometimes be a mapping

    Reacted by Jelle Zijlstra, Kirill Podoprigora, Bénédikt Tran, Simeon Visser and William Sjöblom
  7. AlexWaygood
    closed this as completedon Jun 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

  • @AlexWaygood
    AlexWaygood

Labels

3.12only security fixesonly security fixes3.13only security fixesonly security fixesstdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytopic-typingtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    You are visitor 01402503 — GeezerVison does not store the pages you look up.
    Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.