GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-98586: Add vector call APIs to the Limited API by wjakob · Pull Request #98587 · python/cpython

Conversation

@wjakob

@wjakob wjakob commented Oct 24, 2022 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

This PR constains tentative changes needed to expose the facilities for making PEP-590-style vector calls through Python's limited API.

  • Issue: Add vector call functions to the limited API #98586

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

erlend-aasland reacted with hooray emoji
All reactions
  • 🎉 1 reaction
@wjakob
wjakob requested a review from a team as a code owner October 24, 2022 08:33
@bedevere-bot bedevere-bot added the awaiting review label Oct 24, 2022
@wjakob wjakob mentioned this pull request Oct 24, 2022
Add vector call functions to the limited API #98586
Closed
@wjakob
wjakob force-pushed the vectorcall-public branch from abc4964 to 85d09f5 Compare October 24, 2022 08:47
@erlend-aasland
erlend-aasland requested review from encukou, erlend-aasland and vstinner October 24, 2022 10:49
@erlend-aasland

erlend-aasland commented Oct 24, 2022

Copy link
Copy Markdown
Contributor

@markshannon might be interested in reviewing this as the author of PEP 590.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland erlend-aasland linked an issue Oct 24, 2022 that may be closed by this pull request
Add vector call functions to the limited API #98586
Closed
erlend-aasland
erlend-aasland reviewed Oct 24, 2022
View reviewed changes

@erlend-aasland erlend-aasland left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add tests to Modules/_testcapimodule/vectorcall_limited.c and also adjust the NEWS entry to use Sphinx directives for linking to the relevant parts of the C-API docs and PEP-590?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@wjakob
expose more of the PEP-590 vector call API
7acde40
@wjakob
wjakob force-pushed the vectorcall-public branch from 85d09f5 to 8732fb0 Compare October 24, 2022 13:34
@wjakob

wjakob commented Oct 24, 2022

Copy link
Copy Markdown
Contributor Author

@erlend-aasland Done.

erlend-aasland reacted with hooray emoji erlend-aasland reacted with heart emoji
All reactions
  • 🎉 1 reaction
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

wjakob added 2 commits October 24, 2022 15:36
@wjakob
add testcases to test_capi
68e17a9
@wjakob
adjusted NEWS entry to use Sphinx formatting directives
9934529
@wjakob
wjakob force-pushed the vectorcall-public branch from 8732fb0 to 9934529 Compare October 24, 2022 13:36
@markshannon
markshannon self-requested a review October 24, 2022 14:10
@wjakob
rename internal test methods to avoid them being picked up as indepen…
35685b0
…dent tests
@erlend-aasland

erlend-aasland commented Oct 24, 2022

Copy link
Copy Markdown
Contributor

What is the reason for not adding PyVectorcall_Function to the Limited API?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@wjakob

wjakob commented Oct 24, 2022

Copy link
Copy Markdown
Contributor Author

What is the reason for not adding PyVectorcall_Function to the Limited API?

It seemed like sort of an implementation detail. I am happy to add it (& tests) if there is consensus to do so.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland

erlend-aasland commented Oct 24, 2022

Copy link
Copy Markdown
Contributor

It seemed like sort of an implementation detail. I am happy to add it (& tests) if there is consensus to do so.

I've never had use for it myself, so I have no opinion about this; I was just curious. Let's hear what the others say when they chime in.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland erlend-aasland changed the title gh-98586: expose more of the PEP-590 vector call API gh-98586: Add vector call APIs to the Limited API Oct 25, 2022
erlend-aasland
erlend-aasland reviewed Oct 25, 2022
View reviewed changes
Comment thread Misc/NEWS.d/next/Core and Builtins/2022-10-24-10-30-30.gh-issue-98586.Tha5Iy.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

erlend-aasland
erlend-aasland reviewed Oct 25, 2022
View reviewed changes
Comment thread Modules/_testcapi/vectorcall_limited.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@markshannon

markshannon commented Oct 25, 2022

Copy link
Copy Markdown
Member

Two questions.

First question

Why include PyObject_VectorcallDict? It seem to be an internal function for efficiently supporting the old calling convention for callables that support vectorcall.

Second question

The vectorcall API has two parts.

  1. Calling through PyObject_Vectorcall
  2. Declaring that an object can be called as "vector callable" by setting the Py_TPFLAGS_HAVE_VECTORCALL flag and type.tp_vectorcall_offset.

Any plans to add the ability to declare classes as having "vector callable" instances?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@wjakob

wjakob commented Oct 25, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Thanks for the review @markshannon.

Why include PyObject_VectorcallDict? It seem to be an internal function for efficiently supporting the old calling convention for callables that support vectorcall.

I am happy to hear it. I have never found use for this function myself. I included in the PR for reasons of symmetry, and because it was prominently advertised in the docs next to PyObject_Vectorcall and PyObject_VectorcallMethod. I will remove it in the next update of the PR.

Any plans to add the ability to declare classes as having "vector callable" instances?

I think that this was already done in PR #93274. Please let me know if I misunderstood.

Finally, do you have any thoughts on PyVectorcall_Function brought up by @erlend-aasland ?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon

markshannon commented Oct 25, 2022

Copy link
Copy Markdown
Member

One other question.

Why add PyObject_VectorcallMethod, rather than exposing the lower level method-getting machinery?
PyObject_VectorcallMethod forces the attribute lookup to happen after the arguments are evaluated which is different from Python.
A PyObject *LoadMethod(PyObject *obj, PyObject *name, PyObject **self) function would allow C code to use Python evaluation order. PyObject_VectorcallMethod could be added as a utility function if desirable.

This is a sketch and is probably missing some error handling and/or incref/decrefs.

PyObject *PyObject_VectorcallMethod(
    PyObject *name, PyObject *const *args,
    size_t nargsf, PyObject *kwnames)
{
      /* Error if nargs < 1 */
      PyObject *obj = args[0];
      PyObject *callable = LoadMethod(obj, name, &args[0]);
      /* TO DO --- Handle error if callable == NULL */
      if (args[0] == NULL) {
            /* No self */
           return PyObject_Vectorcall(callable, args+1, (nargsf-1) | PY_VECTORCALL_ARGUMENTS_OFFSET, kwnames);
      }
      return PyObject_Vectorcall(callable, args, nargsf, kwnames);
}
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon

markshannon commented Oct 25, 2022

Copy link
Copy Markdown
Member

Finally, do you have any thoughts on PyVectorcall_Function brought up by @erlend-aasland ?

I'd rather not add it.
Unless it is specified in such a way as to be useless, its existence would unnecessarily constrain the implementation.

erlend-aasland and wjakob reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@wjakob

wjakob commented Oct 25, 2022

Copy link
Copy Markdown
Contributor Author

Why add PyObject_VectorcallMethod, rather than exposing the lower level method-getting machinery?
PyObject_VectorcallMethod forces the attribute lookup to happen after the arguments are evaluated which is different from Python.

It is convenient to use a 1-function-does-it-all interface like PyObject_VectorcallMethod for performing method calls from binding projects. This is particularly true for the Py_LIMITED_API that is already heavy on CPython API calls due to the opaque nature of the interfaces. The difference in attribute lookup ordering is an interesting point. I was not aware of this, but it strikes me as a quite minor/subtle issue. (Maybe others will disagree though!)

Altogether, it sounds to me more like a general question about why PyObject_VectorcallMethod was designed the way it was? My goal with this PR was to expose something that is already in general use.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@wjakob
incorporate feedback
550df59
@bedevere-bot bedevere-bot added the awaiting changes label Oct 26, 2022
@wjakob
fix potential leaks in test suite
8d05d91
@wjakob

wjakob commented Oct 26, 2022

Copy link
Copy Markdown
Contributor Author

Is this okay @markshannon? (with a goto to avoid lots of if blocks with different numbers of Py_DECREFs)

I have made the requested changes; please review again

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added awaiting change review and removed awaiting changes labels Oct 26, 2022
@bedevere-bot

bedevere-bot commented Oct 26, 2022

Copy link
Copy Markdown

Thanks for making the requested changes!

@erlend-aasland, @markshannon: please review the changes made to this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot
bedevere-bot requested review from erlend-aasland and markshannon October 26, 2022 12:54
@wjakob
remove superflous assignment
2c893e0
@wjakob

wjakob commented Oct 26, 2022

Copy link
Copy Markdown
Contributor Author

The test_asyncio test fails on Windows (Azure Pipelines). Potentially a fluke/unrelated issue? I don't see how it could be related.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood

AlexWaygood commented Oct 26, 2022

Copy link
Copy Markdown
Member

The test_asyncio test fails on Windows (Azure Pipelines). Potentially a fluke/unrelated issue? I don't see how it could be related.

It's a known issue that will be fixed by #98704

wjakob and erlend-aasland reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

erlend-aasland
erlend-aasland reviewed Oct 26, 2022
View reviewed changes
Comment thread Modules/_testcapi/vectorcall_limited.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_testcapi/vectorcall_limited.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@wjakob
formatting tweaks
7968a83
@encukou
encukou dismissed markshannon’s stale review October 27, 2022 08:55

The changes were made.

@encukou
Remove outdated commit
72169aa
encukou
encukou approved these changes Oct 27, 2022
View reviewed changes

@encukou encukou left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also have a nitpick, otherwise LGTM.
I'll push a commit to your branch to avoid another round of review ping-pong.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Modules/_testcapi/vectorcall_limited.c Outdated
Comment on lines 157 to 159
* (Empty list left here as template/example, since using
* PyModule_AddFunctions isn't very common.)
*/

@encukou encukou Oct 27, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment can go away now.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting change review labels Oct 27, 2022
@encukou
encukou merged commit e60892f into python:main Oct 27, 2022
@bedevere-bot bedevere-bot removed the awaiting merge label Oct 27, 2022
@encukou

encukou commented Oct 27, 2022

Copy link
Copy Markdown
Member

Thank you for the PR!

wjakob reacted with hooray emoji
All reactions
  • 🎉 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@wjakob
wjakob deleted the vectorcall-public branch October 27, 2022 10:53
@wjakob

wjakob commented Oct 27, 2022

Copy link
Copy Markdown
Contributor Author

Awesome, many thanks for your help in getting it merged!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gvanrossum pushed a commit to gvanrossum/cpython that referenced this pull request Oct 28, 2022
@wjakob @gvanrossum
pythongh-98586: Add vector call APIs to the Limited API (pythonGH-98587)
1ec22b5
Expose the facilities for making vector calls through Python's limited API.
@wjakob

wjakob commented Nov 3, 2022

Copy link
Copy Markdown
Contributor Author

I realized that I forgot to add an entry to Doc/whatsnew/3.12.rst. Should I open a separate PR, or will somebody from the maintainers team do this?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon

markshannon commented Nov 3, 2022

Copy link
Copy Markdown
Member

Open a new PR please.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

wjakob added a commit to wjakob/cpython that referenced this pull request Nov 3, 2022
@wjakob
pythongh-98586: add whatsnew entry, related documentation fix
11b4bca
PR python#98587 addressing issue python#98586 lacked a "what's new" entry.

While making those changes, I noticed an inconsistency in how
``PY_VECTORCALL_ARGUMENTS_OFFSET`` is declared in the underlying Sphinx
markup when compared to other macro constants like
``Py_TPFLAGS_HAVE_VECTORCALL``. This PR fixes that as well, which should
connect a few currently broken cross-references
@wjakob wjakob mentioned this pull request Nov 3, 2022
gh-98586: add whatsnew entry covering limited-API outgoing vector calls, related documentation fix #99056
Merged
@erlend-aasland erlend-aasland mentioned this pull request Nov 11, 2022
gh-96002: Add functional test for Argument Clinic #96178
Merged
@wjakob wjakob mentioned this pull request Mar 5, 2024
Implement stable ABI builds nicholasjng/nanobind-bazel#5
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@erlend-aasland erlend-aasland erlend-aasland approved these changes
@encukou encukou encukou approved these changes
@vstinner vstinner Awaiting requested review from vstinner
@markshannon markshannon Awaiting requested review from markshannon
+1 more reviewer
@Cammiec91 Cammiec91 Cammiec91 approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add vector call functions to the limited API

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@wjakob @erlend-aasland @markshannon @bedevere-bot @AlexWaygood @encukou @Cammiec91
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.