GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — gh-74690: Improve performance when testing against protocols with many attributes. by randolf-scholz · Pull Request #112157 · python/cpython Conversation
|
| Benchmark | baseline | baseline_repro | test |
|---|---|---|---|
| one_int | 34.9 us | not significant | 12.8 us: 2.73x faster |
| two_frac | 505 ns | 501 ns: 1.01x faster | 502 ns: 1.01x faster |
| three_float | 39.2 us | not significant | 10.7 us: 3.67x faster |
| Geometric mean | (ref) | 1.05x slower | 2.16x faster |
one_int and three_float are negative examples, and we see a 2.7x-3.7x speedup, but no measurable slowdown for the true positive two_frac.
Are there any other benchmarks to test against?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
|
||
| # quick true negative check for protocols with many attributes. | ||
| if not hasattr(instance, '__getattr__'): | ||
| if not (cls.__protocol_attrs__ <= set(dir(instance))): |
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
What about classes that define custom __dir__() methods? What about classes that define custom __getattribute__ methods? I think this approach is very risky
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Well, is there any C-implemented alternative to dir, that returns some iterable of the attribute names? I saw inspect.getmembers_static, but it is mostly plain python and way too slow. If custom __dir__ is a worry, one could maybe use object.__dir__(instance).
dir itself is even inefficient since it sorts the names, here we really only need something that iterates over the names.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
In regard to the custom __getattribute__ stuff, I think the current code already suffers from that, no? The docs say about getattr_static:
Note: this function may not be able to retrieve all attributes that getattr can fetch (like dynamically created attributes) and may find attributes that getattr can’t (like descriptors that raise AttributeError).
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
is there any C-implemented alternative to
dir, that returns some iterable of the attribute names?
Not that I know of
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
In regard to the custom
__getattribute__stuff, I think the current code already suffers from that, no?
Ah indeed; good point
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
So really the issue here seems that
getattr_static is just really slow (I re-ran the test, replacing it with plain getattr and got 11x speedup)
getattr can be slow if it triggers some computation.isinstance(x, proto) only checks statically known attributes and can give incorrect results if certain attributes are defined dynamically.
__hasattr__ to communicate which attributes are set dynamically.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Your analysis is correct.
getattr_staticis just really slow (I re-ran the test, replacing it with plaingetattrand got 11x speedup)
getattrcan be slow if it triggers some computation.
isinstance(x, proto)only checks statically known attributes and can give incorrect results if certain attributes are defined dynamically.
Note that we changed these two bullet points in Python 3.12 after long discussions in python/typing#1363 and #102433. Previously, isinstance() checks didn't use inspect.getattr_static, and did resolve dynamically defined attributes. But it was decided that that behaviour was bad, so we decided to accept the performance penalty in order to have more correct and less unexpected behaviour. (And through various other performance optimisations to compensate for that, isinstance() checks against runtime-checkable protocols are still overall much faster in Python 3.12 than in Python 3.11, despite the slowdown from switching to inspect.getattr_static.)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Wait for |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.