Description

eendebakpt

opened on May 3, 2022

Issue body actions

Bug report

The performance of attribute lookup for type objects is worse than for other objects. A benchmark

import pyperf
runner=pyperf.Runner()

setup="""
class Class:
    def all(self):
        pass

x=Class()
"""

runner.timeit('hasattr x.all', "hasattr(x, 'all')", setup=setup)
runner.timeit('hasattr x.__array_ufunc__', "hasattr(x, '__array_ufunc__')", setup=setup)
runner.timeit('hasattr Class.all', "hasattr(Class, 'all')", setup=setup)
runner.timeit('hasattr Class.__array_ufunc__', "hasattr(Class, '__array_ufunc__')", setup=setup) # worse performance

Results:

hasattr x.all: Mean +- std dev: 68.1 ns +- 1.1 ns
hasattr x.__array_ufunc__: Mean +- std dev: 40.4 ns +- 0.3 ns
hasattr Class.all: Mean +- std dev: 38.1 ns +- 0.6 ns
hasattr Class.__array_ufunc__: Mean +- std dev: 255 ns +- 2 ns

The reason seems to be that the type_getattro always executes PyErr_Format, wheras for the "normal" attribute lookup this is avoided (see here and here)

Notes:

• The benchmark is from the python side, but we are working with the C-API
• The performance is important for numpy, see PERF: Improve performance of special attribute lookups numpy/numpy#21423
• Another location where this is a bottleneck: https://github.com/python/cpython/blob/v3.12.0a2/Lib/dataclasses.py#L1301

Your environment

• CPython versions tested on: Python 3.11.0a7+
• Operating system and architecture: Linux Ubuntu

Linked PRs

• gh-92216: improve performance of hasattr for type objects #99977
• gh-92216: improve performance of hasattr for type objects #99979

Reacted by Sebastian Berg

Activity

1. eendebakpt

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on May 3, 2022
2. hugovk

   added

   performancePerformance or resource usagePerformance or resource usage

   on May 3, 2022
3. seberg commented on May 3, 2022

   

   seberg

   on May 3, 2022

   Contributor

   More actions

   Related to this issue, I am wondering if we can consider _PyObject_LookupAttr either as stable (removing the _) or as "semi stable"?
   Of course that is only faster currently for the default tp_getattro, but I suspect that is a huge amount of calls.

   EDIT: Ah, I half thought it was not fully internal API right now, but "semi" exposed.
4. eendebakpt commented on May 3, 2022

   

   eendebakpt

   on May 3, 2022

   ContributorAuthor

   More actions

   Also see: #76752 (PRs where _PyObject_LookupAttr is introduced)
5. serhiy-storchaka commented on May 4, 2022

   

   serhiy-storchaka

   on May 4, 2022

   Member

   More actions

   _PyObject_LookupAttr is an internal API.

   I'll look if it is possible to speed up lookup of the absent attribute in a class object.

   Reacted by Sebastian Berg and Pieter Eendebak
6. seberg

   added a commit that references this issue on May 11, 2022

   PERF: Skip probing `__array_ufunc__` for NumPy builtin scalars (#21470)

   7987557
7. iritkatriel

   added

   3.12only security fixesonly security fixes

   on Sep 12, 2022
8. Fidget-Spinner

   added a commit that references this issue on Dec 23, 2022

   gh-92216: improve performance of `hasattr` for type objects (GH-99979)

   7fc7909
9. Fidget-Spinner commented on Dec 23, 2022

   

   Fidget-Spinner

   on Dec 23, 2022

   Member

   More actions

   @eendebakpt thanks for the improvement. Can I close this issue now or do you plan to submit more PRs?
10. eendebakpt commented on Dec 23, 2022

    

    eendebakpt

    on Dec 23, 2022

    ContributorAuthor

    More actions

    The issue is solved, i will close
11. eendebakpt

    closed this as completedon Dec 23, 2022
12. iritkatriel

    added a commit that references this issue on Dec 28, 2022

    Merged main into regmachine branch (#50)

    fdc73cb

Issue actions