GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-38337: Change getattr to inspect.getattr_static by 3j14 · Pull Request #16521 · python/cpython

Conversation

@3j14

@3j14 3j14 commented Oct 1, 2019 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

When calling inspect.getmembers on a class that has a property (@property), the property will be called by the getattr call in getmembers.

Example:

import inspect

class Example:
    def __init__(self, var):
        self._var = var
        print('__init__')

    def foo(self, bar):
        print(bar)
        print('foo')

    @property
    def var(self):
        print('Access property')
        return self._var


if __name__ == '__main__':
    ex = Example('Hello')
    print('--- getmembers from instance ---')
    print(inspect.getmembers(ex))
    print('--- getmembers from class    ---')
    print(inspect.getmembers(Example))

Result:

__init__
--- getmembers from instance ---
Access property
[('__class__', <attribute '__class__' of 'object' objects>), ..., ('var', 'Hello')]
--- getmembers from class    ---
[('__class__', <attribute '__class__' of 'object' objects>), ..., ('var', <property object at 0x...>)]

Expected (new result):

__init__
--- getmembers from instance ---
[('__class__', <attribute '__class__' of 'object' objects>), ..., ('var', <property object at 0x...>)]
--- getmembers from class    ---
[('__class__', <attribute '__class__' of 'object' objects>), ..., ('var', <property object at 0x...>)]

https://bugs.python.org/issue38337

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@3j14
Change getattr to inspect.getattr_static
ba746bc
@the-knights-who-say-ni the-knights-who-say-ni added the CLA not signed label Oct 1, 2019
@the-knights-who-say-ni

the-knights-who-say-ni commented Oct 1, 2019

Copy link
Copy Markdown

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

CLA Missing

Our records indicate the following people have not signed the CLA:

@jnsdrtlf

For legal reasons we need all the people listed to sign the CLA before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

If you have recently signed the CLA, please wait at least one business day
before our records are updated.

We also demand... A SHRUBBERY!

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added the awaiting review label Oct 1, 2019
@blurb-it
📜🤖 Added by blurb_it.
80fcfb4
@s-sanjay

s-sanjay commented Oct 1, 2019

Copy link
Copy Markdown
Contributor

should we add a test case ?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher brandtbucher added needs backport to 3.7 type-bug An unexpected behavior, bug, or error labels Oct 1, 2019
brandtbucher
brandtbucher requested changes Oct 1, 2019
View reviewed changes

@brandtbucher brandtbucher left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jnsdrtlf, and welcome to CPython! 😎

I have a minor change to suggest for the NEWS entry, but otherwise this looks like a good improvement.

I also agree that adding regression test is probably a good idea.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Misc/NEWS.d/next/Library/2019-10-01-12-23-37.bpo-38337.QmLSXW.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added awaiting core review and removed awaiting review labels Oct 1, 2019
@brandtbucher

brandtbucher commented Oct 1, 2019

Copy link
Copy Markdown
Member

Actually, from looking at the test failures... I'm not sure if this is the desired behavior (and the docs are ambiguous). Maybe @1st1 can shed some light on the "correct" behavior of this function?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher brandtbucher removed needs backport to 3.7 labels Oct 1, 2019
@1st1

1st1 commented Oct 1, 2019

Copy link
Copy Markdown
Member

Since inspect.getmembers is an introspection function it shouldn't run any actual code (i.e. executing getters). It should compute its result statically. Using getattr_static is the right idea here, for sure.

brandtbucher, 3j14, and jaymegordo reacted with thumbs up emoji
All reactions
  • 👍 3 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher

brandtbucher commented Oct 1, 2019 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Okay then. So this probably doesn't need a new test, but the ~4 failing ones should be updated!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@1st1

1st1 commented Oct 1, 2019

Copy link
Copy Markdown
Member

So this probably doesn't need a new test, but the ~4 failing ones should be updated!

I think it needs a new test. Basically an object with a property and a test ensuring that the property was discovered by getmembers, but the getter code wasn't run.

kernc reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@3j14 @brandtbucher
Update NEWS entry
434a3e6
As suggested by @brandtbucher

Co-Authored-By: Brandt Bucher <[email protected]>
@the-knights-who-say-ni the-knights-who-say-ni added CLA signed and removed CLA not signed labels Oct 1, 2019
@3j14

3j14 commented Oct 1, 2019

Copy link
Copy Markdown
Author

I think it needs a new test. Basically an object with a property and a test ensuring that the property was discovered by getmembers, but the getter code wasn't run.

Sounds good! I will get back to work and write a test later today.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

3j14 added 2 commits October 1, 2019 23:46
@3j14
Add test for inspect.getmembers
db038c5
Make sure inspect.getmembers won't call properties or other dynamic attributes
@3j14
Assert whether the property is in the list
8a95946
@3j14

3j14 commented Oct 1, 2019

Copy link
Copy Markdown
Author

The test test_getmembers_VirtualAttribute expects this method

@types.DynamicClassAttribute
    def eggs(self):
        return 'spam'

to be called. It asserts whether ('eggs', 'spam') is in inspect.getmembers. But this is exactly the behavior that I tried to avoid with my PR.

There is also some wierd behaviour when overwriting __getattr__ in a metaclass. This probably needs more insight on how inspect.getmembers should behave.

Failing tests:

  • test_get_slot_members
  • test_getmembers_method
  • test_getmembers_VirtualAttribute
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

hongweipeng
hongweipeng reviewed Oct 3, 2019
View reviewed changes
Comment thread Lib/test/test_inspect.py Outdated

def test_getmembers_static(self):
class Foo:
def __init__(bar):

@hongweipeng hongweipeng Oct 3, 2019

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use def __init__(self, bar): please.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@3j14 3j14 Oct 3, 2019

Copy link
Copy Markdown
Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oups 🤦‍♂ Fixed in commit b415224

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@3j14
Fix constructor without self
b415224
@hongweipeng

This comment has been minimized.

Sign in to view
@3j14
bpo-38337: Fix whitespace in test_inspect
64f5954
@3j14

3j14 commented Oct 7, 2019 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Author

Another inconsistent behaviour: inspect.isdatadescriptor should return True for properties:

Return true if the object is a data descriptor.

Data descriptors have both a get and a set method. Examples are properties (defined in Python), getsets, and members

(Python Docs)

However, the following code does not return any properties:

class Foo:
    def __init__(self, data):
        self._bar = data
    
    @property
    def bar(self):
        print('--BAR--')
        return self._bar

inspect.getmembers(Foo('test'), inspect.isdatadescriptor)

Result

--BAR--
[]

Expected (new result)

[('__class__', <attribute '__class__' of 'object' objects>), ('__dict__', <attribute '__dict__' of 'Foo' objects>), ('__weakref__', <attribute '__weakref__' of 'Foo' objects>), ('bar', <property object at 0x...>)]

Is this correct so far? I am still confused about how getmembers should behave. See test_getmembers_VirtualAttribute, this test expects a different behaviour.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@3j14

3j14 commented Oct 7, 2019

Copy link
Copy Markdown
Author

Additionally,

class Foo:
    pass

inspect.getattr_static(Foo('test'), '__class__')

returns <attribute '__class__' of 'object' objects> whereas getattr would return <class '__main__.Foo'>. What should getmembers return? If it returns <attribute '__class__' of 'object' objects>, the inspect.isclass predicate would not work:

class Foo:
    pass

# Will return [] if getattr_static is used
inspect.getmembers(Foo('test'), inspect.isclass)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@3j14
bpo-38337: Only use getattr_static for properties
3ac2093
1st1
1st1 reviewed Oct 7, 2019
View reviewed changes
Comment thread Lib/inspect.py
# looking in the __dict__.
try:
value = getattr_static(object, key)
if isinstance(getattr_static(object, key), property):

@1st1 1st1 Oct 7, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't we always use getattr_static? If there's a reason for not using it always, please describe that in a comment.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@3j14 3j14 Oct 7, 2019

Copy link
Copy Markdown
Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is related to my latest comment. Sorry, I didn't make this clear:

getattr_static(...,'__class__') returns <attribute '__class__' of 'object' objects> whereas getattr would return <class '__main__.Foo'>. What should getmembers return? Using getattr_static for every attribute doesn't seem to be the solution. It works fine with methods, functions and properties, but some edge cases seem to require getattr (such as __class__). It is not clear what the expected behaviour is.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

kernc reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@1st1 1st1 Oct 7, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, well, describe this in a comment :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@3j14

3j14 commented Oct 8, 2019 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Author

In 3ac2093, I've changed inspect.getmembers to only use inspect.getattr_static for properties, as other members (e.g. __class__) are expected to be returned using getattr(obj,'__class__'):

getattr_static(obj,'__class__') returns <attribute '__class__' of 'object' objects> whereas getattr would return <class '__main__.Foo'>

This feels more like a temporary fix - could be reverted to the first fix I've committed - although I am just not sure which is better.

At this point, I am not sure what the actual expected behaviour should be. Someone should review this and the test case that is currently failing, as it expects a different behaviour (return the actual value of the property).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

1st1
1st1 requested changes Oct 8, 2019
View reviewed changes

@1st1 1st1 left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Jonas, I thought about this PR for some time and I think we should reject it. Modifying getmembers in any way would break backwards compatibility in a very non-obvious way for many users. The ship to fix it has sailed.

We can add a new function getmembers_static that would use getattr_static instead of getattr. It must be a new distinct function, though.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting changes and removed awaiting core review labels Oct 8, 2019
@bedevere-bot

bedevere-bot commented Oct 8, 2019

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@3j14

3j14 commented Oct 10, 2019 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Author

@1st1 I share your thoughts on that. It does not seem to be obvious, what getmembers should return - and changing it in any way will break existing code. This has already caused me a lot of headaches.

As for introducing a function getmembers_static: I still have some questions that should be discussed before doing so:

  1. What should getmembers_static return? Is getattr_static appropriate for every member? This would break some predicates (e.g. isclass, see comment above). There does not seem to be a "simple" solution.

  2. A technical question: The new method should pass most of the same tests as getmembers. Would one just copy and paste the old ones? It would also share most of the code - except for maybe 1 line - with getmembers.

  3. Is there a reason for getmembers to return a list of tuples? I would have suggested to use a dict instead, which feels more intuitive on first though.

Is there a better place to discuss such questions?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@1st1

1st1 commented Oct 24, 2019

Copy link
Copy Markdown
Member

Is there a better place to discuss such questions?

I think it would be better if you could post this to the python-ideas mailing list. I've very limited time and would hate to get this thing staled because of me.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@1st1

1st1 commented Oct 24, 2019

Copy link
Copy Markdown
Member

Or post it here: https://discuss.python.org/c/ideas

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@csabella

csabella commented Jan 26, 2020

Copy link
Copy Markdown
Contributor

I added a link to the discussion on the bug tracker.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@kernc kernc mentioned this pull request Jun 16, 2020
bpo-30533:Add function inspect.getmembers_static that does not call properties or dynamic properties. #20911
Merged
@jaymegordo

jaymegordo commented Aug 30, 2020

Copy link
Copy Markdown

This would be very helpful, running into the same issue with inspect.getmembers evaluating @Property decorated methods.

getmembers_static would solve it.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner vstinner closed this May 3, 2021
@vstinner
vstinner deleted the branch python:master May 3, 2021 21:29
@jnsdrtlf jnsdrtlf mannequin mentioned this pull request Apr 10, 2022
inspect: getmembers calls properties #82518
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@1st1 1st1 1st1 requested changes
@brandtbucher brandtbucher brandtbucher requested changes
+1 more reviewer
@hongweipeng hongweipeng hongweipeng left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting changes type-bug An unexpected behavior, bug, or error

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

10 participants

@3j14 @the-knights-who-say-ni @s-sanjay @brandtbucher @1st1 @hongweipeng @bedevere-bot @csabella @jaymegordo @vstinner
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.