GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-94936: C getters: co_varnames, co_cellvars, co_freevars by Fidget-Spinner · Pull Request #95008 · python/cpython

Conversation

@Fidget-Spinner

@Fidget-Spinner Fidget-Spinner commented Jul 19, 2022 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member
  • Issue: Public C API for accessing code object fields #94936

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@Fidget-Spinner
C getters: co_varnames, co_cellvars, co_freevars
d287d7c
@Fidget-Spinner
Fidget-Spinner requested a review from markshannon as a code owner July 19, 2022 14:27
@bedevere-bot bedevere-bot added the awaiting core review label Jul 19, 2022
@Fidget-Spinner
Add news entry
a1450f8
@Fidget-Spinner
Fidget-Spinner requested a review from vstinner July 19, 2022 14:38
markshannon
markshannon reviewed Jul 19, 2022
View reviewed changes
Comment thread Modules/_testcapimodule.c
return NULL;
}
/* co_varnames */
{

@markshannon markshannon Jul 19, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make more sense to expose the functions here, and test them in test/test_capi.py, checking that the C version returns the same as the Python version?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Aug 3, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be better, but I'm fine with these tests.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@gvanrossum gvanrossum mentioned this pull request Jul 20, 2022
Public C API for accessing code object fields #94936
Closed
gvanrossum
gvanrossum reviewed Jul 20, 2022
View reviewed changes
Comment thread Objects/codeobject.c
PyObject *
PyCode_GetVarnames(PyCodeObject *code)
{
return _PyCode_GetVarnames(code);

@gvanrossum gvanrossum Jul 20, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the advantage of separate wrapper functions rather than just removing the leading _ from the wrapped functions?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@vstinner vstinner Aug 3, 2022 •
edited by gvanrossum
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I concur wuth @gvanrossum, unless it's implemented as a static inline function, I see no advantage, but that's not the case here. Just rename _PyCode_GetVarnames() to PyCode_GetVarnames(). Same remark for the 2 other functions.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
vstinner
vstinner reviewed Aug 3, 2022
View reviewed changes
Comment thread Include/cpython/code.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Objects/codeobject.c
PyObject *
PyCode_GetVarnames(PyCodeObject *code)
{
return _PyCode_GetVarnames(code);

@vstinner vstinner Aug 3, 2022 •
edited by gvanrossum
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I concur wuth @gvanrossum, unless it's implemented as a static inline function, I see no advantage, but that's not the case here. Just rename _PyCode_GetVarnames() to PyCode_GetVarnames(). Same remark for the 2 other functions.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Modules/_testcapimodule.c
return NULL;
}
/* co_varnames */
{

@vstinner vstinner Aug 3, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be better, but I'm fine with these tests.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner

vstinner commented Aug 3, 2022

Copy link
Copy Markdown
Member

This change targets Python 3.11 which no longer accepts new features (after beta1). But it seems like the lack of public C API functions to the removed PyCodeObject members is an issue, see: https://discuss.python.org/t/getting-the-class-name-of-a-code-frame-object-in-cpython-3-11-c-api/17396

Should we target Python 3.11 or is it ok to leave Python 3.11 with no public C API for that?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gvanrossum

gvanrossum commented Aug 3, 2022

Copy link
Copy Markdown
Member

@pablogsal -- These three new C getter APIs would be useful for some and don't really add any risk for 3.11, but at the same time I don't think they are essential. What do you think?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Aug 3, 2022

Copy link
Copy Markdown
Member

but at the same time I don't think they are essential

A datapoint: when I proposed to replace a direct access to a structure member in C with PyObject_GetAttrString(), @nedbat reported a serious slowdown in his https://github.com/nedbat/coveragepy/ project. PyFrame_GetLasti() was added to Python 3.11 to solve this performance issue: https://docs.python.org/dev/c-api/frame.html#c.PyFrame_GetLasti

I failed to find the discussion about the performance regression caused by PyObject_GetAttrString() but it's obviously way more work than just reading directly a C structure member... The function decodes a bytes string from UTF-8, creates a temporary Unicode object, lookup in a dictionary, delete the temporary Unicode code... But then you get a Python object, you need to unbox its content and delete the second (temporary?) result object...

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gvanrossum

gvanrossum commented Aug 3, 2022

Copy link
Copy Markdown
Member

FWIW one suggestion (see #94936 (comment)) to avoid the perf regression of PyObject_GetAttrString() is to just include internal headers and call the _Py versions of the functions. That would have to be done just for 3.11. Of course a shortcut would also be to just duplicate the extern declarations for these from the headers. Again just for 3.11.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal

pablogsal commented Aug 4, 2022

Copy link
Copy Markdown
Member

@pablogsal -- These three new C getter APIs would be useful for some and don't really add any risk for 3.11, but at the same time I don't think they are essential. What do you think?

I would be ok landing this in 3.11 but we are one day from an already delayed release candidate so if these APIs are not essential I would prefer delaying them to 3.12 because they won't be tested when we release them (before the RC) and I'm not sure of we can properly review the implementation in one day.

If you think is going to be much better to land these in 3.11 then let's do it, but we are going to need at least 2 core Devs to review and approve the PR.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

vstinner
vstinner approved these changes Aug 4, 2022
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. @gvanrossum: Would you mind to review this PR as well?

I can rename _PyCode_GetVarnames() to PyCode_GetVarnames() in a following PR. But I would prefer to see these functions added to Python 3.11. Avoiding the internal API to access code members in C is way more convenient to port extensions to Python 3.11!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Aug 4, 2022
@vstinner vstinner added the needs backport to 3.11 only security fixes label Aug 4, 2022
gvanrossum
gvanrossum approved these changes Aug 4, 2022
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it’s urgent I’m fine with this as is. We can clean up later.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@gvanrossum
gvanrossum merged commit 42b102b into python:main Aug 4, 2022
@miss-islington

miss-islington commented Aug 4, 2022

Copy link
Copy Markdown
Contributor

Thanks @Fidget-Spinner for the PR, and @gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting merge label Aug 4, 2022
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Aug 4, 2022
@Fidget-Spinner @miss-islington
pythongh-94936: C getters: co_varnames, co_cellvars, co_freevars (pyt…
333a642
…honGH-95008)

(cherry picked from commit 42b102b)

Co-authored-by: Ken Jin <[email protected]>
@bedevere-bot

bedevere-bot commented Aug 4, 2022

Copy link
Copy Markdown

GH-95653 is a backport of this pull request to the 3.11 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Aug 4, 2022
@vstinner

vstinner commented Aug 4, 2022

Copy link
Copy Markdown
Member

I wrote python/pythoncapi-compat#44 to add the 3 functions to pythoncapi-compat (for Python 3.10 and older).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington added a commit that referenced this pull request Aug 4, 2022
@miss-islington @Fidget-Spinner
gh-94936: C getters: co_varnames, co_cellvars, co_freevars (GH-95008)
f292635
(cherry picked from commit 42b102b)

Co-authored-by: Ken Jin <[email protected]>
@Fidget-Spinner

Fidget-Spinner commented Aug 4, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Oops I missed the convo on this one.I planned to direct extension authors to use PyObject_GetAttrString. I want a better/more performant implementation for 3.12. It will cache the debug info in a lazy debug struct much like how we do it for PyInterpreterFrame and PyFrameObject now.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner
Fidget-Spinner deleted the code_getters branch August 4, 2022 15:38
@joerick

joerick commented Aug 5, 2022

Copy link
Copy Markdown
Contributor

Thank you so much for getting this in! 🙏

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner vstinner mentioned this pull request Aug 5, 2022
gh-91248: Add PyFrame_GetVar() function #95712
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@markshannon markshannon markshannon left review comments
@vstinner vstinner vstinner approved these changes
@gvanrossum gvanrossum gvanrossum approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

8 participants

@Fidget-Spinner @vstinner @gvanrossum @pablogsal @miss-islington @bedevere-bot @joerick @markshannon
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.