GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-106307: Add PyMapping_GetOptionalItem() by serhiy-storchaka · Pull Request #106308 · python/cpython

Conversation

@serhiy-storchaka

@serhiy-storchaka serhiy-storchaka commented Jul 1, 2023 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Replacement of PyObject_GetItem() which doesn't raise KeyError.

int _PyMapping_LookupItem(PyObject *obj, PyObject *key, PyObject **result)

Return 1 and set *result != NULL if a key is found.
Return 0 and set *result == NULL if a key is not found; a KeyError is silenced.
Return -1 and set *result == NULL if an error other than KeyError is raised.

  • Issue: Add PyMapping_GetOptionalItem() #106307

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@serhiy-storchaka serhiy-storchaka added the skip news label Jul 1, 2023
@serhiy-storchaka
serhiy-storchaka requested review from brettcannon, ericsnowcurrently, ncoghlan and warsaw as code owners July 1, 2023 12:31
@bedevere-bot bedevere-bot added the awaiting core review label Jul 1, 2023
@bedevere-bot bedevere-bot mentioned this pull request Jul 1, 2023
Add PyMapping_GetOptionalItem() #106307
Closed
@serhiy-storchaka
serhiy-storchaka requested review from markshannon and vstinner July 1, 2023 20:39
@serhiy-storchaka
pythongh-106307: Add _PyMapping_LookupItem()
078ea48
Replacement of PyObject_GetItem() which doesn't raise KeyError.

  int _PyMapping_LookupItem(PyObject *obj, PyObject *key, PyObject **result)

Return 1 and set *result != NULL if a key is found.
Return 0 and set *result == NULL if a key is not found;
a KeyError is silenced.
Return -1 and set *result == NULL if an error other than KeyError
is raised.
@serhiy-storchaka
serhiy-storchaka force-pushed the _PyMapping_LookupItem branch from cfc5851 to 078ea48 Compare July 1, 2023 20:46
vstinner
vstinner reviewed Jul 2, 2023
View reviewed changes
Comment thread Include/cpython/object.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

erlend-aasland
erlend-aasland reviewed Jul 2, 2023
View reviewed changes
Comment thread Python/import.c Outdated
_PyErr_Clear(tstate);
}
}
(void)_PyMapping_LookupItem(modules, name, &m);

@erlend-aasland erlend-aasland Jul 2, 2023

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of interest, why not explicitly _PyErr_Clear(tstate) if _PyMapping_LookupItem returns -1?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@serhiy-storchaka serhiy-storchaka Jul 6, 2023

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the current code does not clear arbitrary errors (and doing this would be wrong). It only clears KeyError, what is done internally by _PyMapping_LookupItem.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@serhiy-storchaka
Make them public C API.
00a3743
@serhiy-storchaka
serhiy-storchaka requested review from a team and encukou as code owners July 7, 2023 17:15
@serhiy-storchaka serhiy-storchaka changed the title gh-106307: Add _PyMapping_LookupItem() gh-106307: Add PyMapping_GetOptionalItem() Jul 7, 2023
carljm
carljm reviewed Jul 7, 2023
View reviewed changes

@carljm carljm left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bytecodes.c cleanups alone (i.e. the existing cases where we cared enough about perf to special-case the exact-dict fast path) clearly show the value of this API! Now all call sites can get the same perf benefit, with much less verbosity.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

erlend-aasland reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread Doc/c-api/mapping.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/mapping.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/whatsnew/3.13.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Include/abstract.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Include/abstract.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/mapping.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/mapping.rst Outdated
Comment on lines +40 to +43
Return ``1`` and set ``*result != NULL`` if a key is found.
Return ``0`` and set ``*result == NULL`` if a key is not found;
a :exc:`KeyError` is silenced.
Return ``-1`` and set ``*result == NULL`` if an error other than

@carljm carljm Jul 7, 2023 •
edited by serhiy-storchaka
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To me, "KeyError is silenced" suggests internal implementation details that the caller doesn't need to care about, and b) may or may not be true (in the exact-dict case, no KeyError is silenced because it isn't set in the first place). IMO "no KeyError is raised" is a clearer statement of the contract that is relevant to the caller.

Suggested change
Return ``1`` and set ``*result != NULL`` if a key is found.
Return ``0`` and set ``*result == NULL`` if a key is not found;
a :exc:`KeyError` is silenced.
Return ``-1`` and set ``*result == NULL`` if an error other than
Return ``1`` and set ``*result != NULL`` if the key is found.
Return ``0`` and set ``*result == NULL`` if the key is not found;
the :exc:`KeyError` is silenced.
Return ``-1`` and set ``*result == NULL`` if an error other than

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@carljm carljm Jul 7, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, it may be important to specify "the KeyError is silenced" to clarify the behavior with a custom __getitem__ method that raises KeyError explicitly. I take back that part of the suggestion; I would just replace all three instances of a with the.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Jul 7, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dislike specify that *pvalue is set to NULL on error. I would prefer to leave it undefined.

Code like (void)PyMapping_GetOptionalItem(modules, name, &m); would be unsafe: I prefer to handle explicitly the error case and no rely on m value.

In the implementation, I prefer to set it to NULL to avoid... the undefined behavior :-) Maybe in debug mode we could even set the value to a marker value to detect the usage of the undefined value.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@serhiy-storchaka serhiy-storchaka Jul 8, 2023

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I disagree. It will require explicitly setting result = NULL before every call of this function, and if you forgot to do this, it will crash your code in rare case of error, which may be not covered by tests. The risk of writing errorenous code is too high even if there would be a benefit from not setting the result to NULL.

Also, it makes the code simpler. Instead of saving the return code into a variable and checking it in two if\ s

int rc = PyMapping_GetOptionalItem(...);
if (rc < 0) {
    goto error;
}
if (rc == 0) {
    // handle default
}

you can get rid of that variable and insert the call directly in the if.

if (PyMapping_GetOptionalItem(...) < 0) {
    goto error;
}
if (value == NULL) {
    // handle default
}

And there are many other cases in which it is convenient that you have two ways to check for result. I suggest you to make PyMapping_GetOptionalItem() not setting the result to NULL on error and to look how much code it will require to rewrite and how much uglier it will become. It is the same for PyObject_GetOptionalAttr.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

carljm reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@erlend-aasland erlend-aasland Jul 11, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I disagree. It will require explicitly setting result = NULL before every call of this function, and if you forgot to do this, it will crash your code in rare case of error, which may be not covered by tests.

+1 and slightly related, I have a PR for the devguide repo regarding explicitly clearing output parameters: python/devguide#1129 python/devguide#1128

cc. @serhiy-storchaka, @carljm

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Doc/whatsnew/3.13.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

vstinner
vstinner reviewed Jul 7, 2023
View reviewed changes
Comment thread Objects/abstract.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Objects/abstract.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Objects/abstract.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_pickle.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/mapping.rst Outdated
Comment on lines +40 to +43
Return ``1`` and set ``*result != NULL`` if a key is found.
Return ``0`` and set ``*result == NULL`` if a key is not found;
a :exc:`KeyError` is silenced.
Return ``-1`` and set ``*result == NULL`` if an error other than

@vstinner vstinner Jul 7, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dislike specify that *pvalue is set to NULL on error. I would prefer to leave it undefined.

Code like (void)PyMapping_GetOptionalItem(modules, name, &m); would be unsafe: I prefer to handle explicitly the error case and no rely on m value.

In the implementation, I prefer to set it to NULL to avoid... the undefined behavior :-) Maybe in debug mode we could even set the value to a marker value to detect the usage of the undefined value.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@brettcannon
brettcannon removed their request for review July 7, 2023 22:01
serhiy-storchaka and others added 3 commits July 8, 2023 11:45
@serhiy-storchaka @carljm @vstinner
Apply suggestions from code review
4b3fc37
Co-authored-by: Carl Meyer <[email protected]>
Co-authored-by: Victor Stinner <[email protected]>
@serhiy-storchaka @carljm
Apply suggestions from code review
a176323
Co-authored-by: Carl Meyer <[email protected]>
@serhiy-storchaka
Cleanup.
0f6cb97
@serhiy-storchaka serhiy-storchaka removed the skip news label Jul 8, 2023
@serhiy-storchaka
Add a news entry.
0c2244a
vstinner
vstinner approved these changes Jul 8, 2023
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Jul 8, 2023
encukou
encukou reviewed Jul 10, 2023
View reviewed changes
Comment thread Misc/stable_abi.toml Resolved

Uh oh!

There was an error while loading. Please reload this page.

@serhiy-storchaka
Move new entries to the end of Misc/stable_abi.toml.
576bcf5
serhiy-storchaka added 3 commits July 10, 2023 15:53
@serhiy-storchaka
Merge branch 'main' into _PyMapping_LookupItem
a9f41bd
@serhiy-storchaka
Rewrite documentation according to review in python#106522.
d6a9204
@serhiy-storchaka
Merge branch 'main' into _PyMapping_LookupItem
07a169f
@serhiy-storchaka
serhiy-storchaka merged commit 4bf4371 into python:main Jul 11, 2023
@bedevere-bot bedevere-bot removed the awaiting merge label Jul 11, 2023
@serhiy-storchaka
serhiy-storchaka deleted the _PyMapping_LookupItem branch July 11, 2023 20:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@carljm carljm carljm left review comments
@encukou encukou encukou left review comments
@erlend-aasland erlend-aasland erlend-aasland left review comments
@vstinner vstinner vstinner approved these changes
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner
@ncoghlan ncoghlan Awaiting requested review from ncoghlan ncoghlan is a code owner
@warsaw warsaw Awaiting requested review from warsaw warsaw is a code owner
@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@serhiy-storchaka @carljm @vstinner @encukou @erlend-aasland @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.