GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-113320: Reduce the number of dangerous `getattr()` calls when constructing protocol classes by AlexWaygood · Pull Request #113401 · python/cpython

Conversation

@AlexWaygood

@AlexWaygood AlexWaygood commented Dec 22, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member
  • Only attempt to figure out whether protocol members are "method members" or not if the class is marked as a runtime protocol. This information is irrelevant for non-runtime protocols; we can safely skip the risky introspection for them.
  • Only do the risky getattr() calls in one place (the runtime_checkable class decorator), rather than in three places (_ProtocolMeta.__init__, _ProtocolMeta.__instancecheck__ and _ProtocolMeta.__subclasscheck__). This reduces the number of locations in typing.py where the risky introspection could go wrong.
  • For runtime protocols, if determining whether a protocol member is callable or not fails, give a better error message. I think it's reasonable for us to reject runtime protocols that have members which raise strange exceptions when you try to access them. PEP-544 clearly states that all protocol member must be callable for issubclass() calls against the protocol to be valid -- and if a member raises when we try to access it, there's no way for us to figure out whether it's a callable member or not!

Cc. @Gobot1234

  • Issue: Should typing be calling getattr on arbitrary classes?  #113320

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Gobot1234 reacted with heart emoji
All reactions
  • ❤️ 1 reaction
@AlexWaygood
pythongh-113320: Reduce the number of dangerous getattr() calls whe…
7b9f8a1
…n constructing protocol classes
@AlexWaygood AlexWaygood added topic-typing needs backport to 3.12 only security fixes labels Dec 22, 2023
@AlexWaygood
AlexWaygood requested a review from carljm December 22, 2023 11:36
@AlexWaygood
AlexWaygood requested a review from JelleZijlstra as a code owner December 22, 2023 11:36
@bedevere-app bedevere-app Bot added the awaiting core review label Dec 22, 2023
@bedevere-app bedevere-app Bot mentioned this pull request Dec 22, 2023
Should typing be calling getattr on arbitrary classes? #113320
Closed
AlexWaygood
AlexWaygood commented Dec 23, 2023
View reviewed changes
Comment thread Lib/test/test_typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood
Better comment
2d8d40a
@AlexWaygood

AlexWaygood commented Dec 23, 2023

Copy link
Copy Markdown
Member Author

Runtime protocols with members that raised when accessed on the class object have never worked — not even on Python <=3.11 (see #113320 (comment)) — so I don't think there are any backwards-compatibility issues here w.r.t. raising an error if someone tries to create one

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

carljm
carljm approved these changes Jan 3, 2024
View reviewed changes

@carljm carljm left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thank you!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/test/test_typing.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Jan 3, 2024
AlexWaygood added 2 commits January 5, 2024 00:40
@AlexWaygood
Merge branch 'main' into classproperties-protocols-2
484f0d0
@AlexWaygood
Address review + add a couple of comments
e0e108b
@AlexWaygood

AlexWaygood commented Jan 5, 2024

Copy link
Copy Markdown
Member Author

Thanks for the review!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood
AlexWaygood enabled auto-merge (squash) January 5, 2024 00:45
@AlexWaygood
AlexWaygood merged commit ed6ea3e into python:main Jan 5, 2024
@AlexWaygood
AlexWaygood deleted the classproperties-protocols-2 branch January 5, 2024 01:01
@bedevere-app bedevere-app Bot removed the awaiting merge label Jan 5, 2024
@miss-islington-app

miss-islington-app Bot commented Jan 5, 2024

Copy link
Copy Markdown

Thanks @AlexWaygood for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington-app

miss-islington-app Bot commented Jan 5, 2024

Copy link
Copy Markdown

Sorry, @AlexWaygood, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker ed6ea3ea79fac68b127c7eb457c7ecb996461010 3.12
AlexWaygood reacted with thumbs down emoji
All reactions
  • 👎 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington-app miss-islington-app Bot assigned AlexWaygood Jan 5, 2024
@bedevere-app

bedevere-app Bot commented Jan 5, 2024

Copy link
Copy Markdown

GH-113722 is a backport of this pull request to the 3.12 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.12 only security fixes label Jan 5, 2024
AlexWaygood added a commit to AlexWaygood/cpython that referenced this pull request Jan 5, 2024
@AlexWaygood
[3.12] pythongh-113320: Reduce the number of dangerous getattr() ca…
1dbee4d
…lls when constructing protocol classes (python#113401)

- Only attempt to figure out whether protocol members are "method members" or not if the class is marked as a runtime protocol. This information is irrelevant for non-runtime protocols; we can safely skip the risky introspection for them.
- Only do the risky getattr() calls in one place (the runtime_checkable class decorator), rather than in three places (_ProtocolMeta.__init__, _ProtocolMeta.__instancecheck__ and _ProtocolMeta.__subclasscheck__). This reduces the number of locations in typing.py where the risky introspection could go wrong.
- For runtime protocols, if determining whether a protocol member is callable or not fails, give a better error message. I think it's reasonable for us to reject runtime protocols that have members which raise strange exceptions when you try to access them. PEP-544 clearly states that all protocol member must be callable for issubclass() calls against the protocol to be valid -- and if a member raises when we try to access it, there's no way for us to figure out whether it's a callable member or not!

(cherry-picked from commit ed6ea3e)
AlexWaygood added a commit that referenced this pull request Jan 5, 2024
@AlexWaygood
[3.12] gh-113320: Reduce the number of dangerous getattr() calls wh…
8435fbf
…en constructing protocol classes (#113401) (#113722)

- Only attempt to figure out whether protocol members are "method members" or not if the class is marked as a runtime protocol. This information is irrelevant for non-runtime protocols; we can safely skip the risky introspection for them.
- Only do the risky getattr() calls in one place (the runtime_checkable class decorator), rather than in three places (_ProtocolMeta.__init__, _ProtocolMeta.__instancecheck__ and _ProtocolMeta.__subclasscheck__). This reduces the number of locations in typing.py where the risky introspection could go wrong.
- For runtime protocols, if determining whether a protocol member is callable or not fails, give a better error message. I think it's reasonable for us to reject runtime protocols that have members which raise strange exceptions when you try to access them. PEP-544 clearly states that all protocol member must be callable for issubclass() calls against the protocol to be valid -- and if a member raises when we try to access it, there's no way for us to figure out whether it's a callable member or not!

(cherry-picked from commit ed6ea3e)
@AlexWaygood AlexWaygood mentioned this pull request Jan 20, 2024
Backport recent improvements to the implementation of Protocol python/typing_extensions#324
Merged
kulikjak pushed a commit to kulikjak/cpython that referenced this pull request Jan 22, 2024
@AlexWaygood @kulikjak
pythongh-113320: Reduce the number of dangerous getattr() calls whe…
98bee40
…n constructing protocol classes (python#113401)

- Only attempt to figure out whether protocol members are "method members" or not if the class is marked as a runtime protocol. This information is irrelevant for non-runtime protocols; we can safely skip the risky introspection for them.
- Only do the risky getattr() calls in one place (the runtime_checkable class decorator), rather than in three places (_ProtocolMeta.__init__, _ProtocolMeta.__instancecheck__ and _ProtocolMeta.__subclasscheck__). This reduces the number of locations in typing.py where the risky introspection could go wrong.
- For runtime protocols, if determining whether a protocol member is callable or not fails, give a better error message. I think it's reasonable for us to reject runtime protocols that have members which raise strange exceptions when you try to access them. PEP-544 clearly states that all protocol member must be callable for issubclass() calls against the protocol to be valid -- and if a member raises when we try to access it, there's no way for us to figure out whether it's a callable member or not!
aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024
@AlexWaygood @aisk
pythongh-113320: Reduce the number of dangerous getattr() calls whe…
1d0f713
…n constructing protocol classes (python#113401)

- Only attempt to figure out whether protocol members are "method members" or not if the class is marked as a runtime protocol. This information is irrelevant for non-runtime protocols; we can safely skip the risky introspection for them.
- Only do the risky getattr() calls in one place (the runtime_checkable class decorator), rather than in three places (_ProtocolMeta.__init__, _ProtocolMeta.__instancecheck__ and _ProtocolMeta.__subclasscheck__). This reduces the number of locations in typing.py where the risky introspection could go wrong.
- For runtime protocols, if determining whether a protocol member is callable or not fails, give a better error message. I think it's reasonable for us to reject runtime protocols that have members which raise strange exceptions when you try to access them. PEP-544 clearly states that all protocol member must be callable for issubclass() calls against the protocol to be valid -- and if a member raises when we try to access it, there's no way for us to figure out whether it's a callable member or not!
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Feb 26, 2024
adam
py-typing-extensions: updated to 4.10.0
141f6d4
Release 4.10.0 (February 24, 2024)

This feature release adds support for PEP 728 (TypedDict with extra
items) and PEP 742 (``TypeIs``).

There are no changes since 4.10.0rc1.

Release 4.10.0rc1 (February 17, 2024)

- Add support for PEP 728, supporting the `closed` keyword argument and the
  special `__extra_items__` key for TypedDict. Patch by Zixuan James Li.
- Add support for PEP 742, adding `typing_extensions.TypeIs`. Patch
  by Jelle Zijlstra.
- Drop runtime error when a read-only `TypedDict` item overrides a mutable
  one. Type checkers should still flag this as an error. Patch by Jelle
  Zijlstra.
- Speedup `issubclass()` checks against simple runtime-checkable protocols by
  around 6% (backporting python/cpython#112717, by Alex
  Waygood).
- Fix a regression in the implementation of protocols where `typing.Protocol`
  classes that were not marked as `@runtime_checkable` would be unnecessarily
  introspected, potentially causing exceptions to be raised if the protocol had
  problematic members. Patch by Alex Waygood, backporting
  python/cpython#113401.
@ibakirov ibakirov mentioned this pull request Feb 26, 2024
Bump typing-extensions from 4.9.0 to 4.10.0 IBX-EDU/sanic-app#10
Merged
Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024
@AlexWaygood @Glyphack
pythongh-113320: Reduce the number of dangerous getattr() calls whe…
52f97c8
…n constructing protocol classes (python#113401)

- Only attempt to figure out whether protocol members are "method members" or not if the class is marked as a runtime protocol. This information is irrelevant for non-runtime protocols; we can safely skip the risky introspection for them.
- Only do the risky getattr() calls in one place (the runtime_checkable class decorator), rather than in three places (_ProtocolMeta.__init__, _ProtocolMeta.__instancecheck__ and _ProtocolMeta.__subclasscheck__). This reduces the number of locations in typing.py where the risky introspection could go wrong.
- For runtime protocols, if determining whether a protocol member is callable or not fails, give a better error message. I think it's reasonable for us to reject runtime protocols that have members which raise strange exceptions when you try to access them. PEP-544 clearly states that all protocol member must be callable for issubclass() calls against the protocol to be valid -- and if a member raises when we try to access it, there's no way for us to figure out whether it's a callable member or not!
msk pushed a commit to msk/pkgsrc that referenced this pull request May 11, 2026
adam
py-typing-extensions: updated to 4.10.0
a281da8
Release 4.10.0 (February 24, 2024)

This feature release adds support for PEP 728 (TypedDict with extra
items) and PEP 742 (``TypeIs``).

There are no changes since 4.10.0rc1.

Release 4.10.0rc1 (February 17, 2024)

- Add support for PEP 728, supporting the `closed` keyword argument and the
  special `__extra_items__` key for TypedDict. Patch by Zixuan James Li.
- Add support for PEP 742, adding `typing_extensions.TypeIs`. Patch
  by Jelle Zijlstra.
- Drop runtime error when a read-only `TypedDict` item overrides a mutable
  one. Type checkers should still flag this as an error. Patch by Jelle
  Zijlstra.
- Speedup `issubclass()` checks against simple runtime-checkable protocols by
  around 6% (backporting python/cpython#112717, by Alex
  Waygood).
- Fix a regression in the implementation of protocols where `typing.Protocol`
  classes that were not marked as `@runtime_checkable` would be unnecessarily
  introspected, potentially causing exceptions to be raised if the protocol had
  problematic members. Patch by Alex Waygood, backporting
  python/cpython#113401.
jperkin pushed a commit to TritonDataCenter/pkgsrc that referenced this pull request May 14, 2026
adam
py-typing-extensions: updated to 4.10.0
68eedf2
Release 4.10.0 (February 24, 2024)

This feature release adds support for PEP 728 (TypedDict with extra
items) and PEP 742 (``TypeIs``).

There are no changes since 4.10.0rc1.

Release 4.10.0rc1 (February 17, 2024)

- Add support for PEP 728, supporting the `closed` keyword argument and the
  special `__extra_items__` key for TypedDict. Patch by Zixuan James Li.
- Add support for PEP 742, adding `typing_extensions.TypeIs`. Patch
  by Jelle Zijlstra.
- Drop runtime error when a read-only `TypedDict` item overrides a mutable
  one. Type checkers should still flag this as an error. Patch by Jelle
  Zijlstra.
- Speedup `issubclass()` checks against simple runtime-checkable protocols by
  around 6% (backporting python/cpython#112717, by Alex
  Waygood).
- Fix a regression in the implementation of protocols where `typing.Protocol`
  classes that were not marked as `@runtime_checkable` would be unnecessarily
  introspected, potentially causing exceptions to be raised if the protocol had
  problematic members. Patch by Alex Waygood, backporting
  python/cpython#113401.
@coderabbitai coderabbitai Bot mentioned this pull request Sep 22, 2026
feat: parser 6.0 release bundle FluffyBrudy/tilemap-parser#22
Merged
tsutsui pushed a commit to tsutsui/pkgsrc that referenced this pull request Oct 17, 2026
adam
py-typing-extensions: updated to 4.10.0
3e46d90
Release 4.10.0 (February 24, 2024)

This feature release adds support for PEP 728 (TypedDict with extra
items) and PEP 742 (``TypeIs``).

There are no changes since 4.10.0rc1.

Release 4.10.0rc1 (February 17, 2024)

- Add support for PEP 728, supporting the `closed` keyword argument and the
  special `__extra_items__` key for TypedDict. Patch by Zixuan James Li.
- Add support for PEP 742, adding `typing_extensions.TypeIs`. Patch
  by Jelle Zijlstra.
- Drop runtime error when a read-only `TypedDict` item overrides a mutable
  one. Type checkers should still flag this as an error. Patch by Jelle
  Zijlstra.
- Speedup `issubclass()` checks against simple runtime-checkable protocols by
  around 6% (backporting python/cpython#112717, by Alex
  Waygood).
- Fix a regression in the implementation of protocols where `typing.Protocol`
  classes that were not marked as `@runtime_checkable` would be unnecessarily
  introspected, potentially causing exceptions to be raised if the protocol had
  problematic members. Patch by Alex Waygood, backporting
  python/cpython#113401.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@carljm carljm carljm approved these changes
@JelleZijlstra JelleZijlstra Awaiting requested review from JelleZijlstra JelleZijlstra is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@AlexWaygood AlexWaygood

Labels

topic-typing

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@AlexWaygood @carljm
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.