GeezerVison is showing github.com — Receiving vector calls in the Py_LIMITED_API · Issue #93274 · python/cpython
Receiving vector calls in the Py_LIMITED_API #93274DescriptionFeature or enhancement
The PyType_FromSpec family of functions provides a convenient and forward-compatible way mechanism for creating new types. It could in principle also be used to create callables providing a PEP-590 vector call interface, which has significant performance benefits for binding libraries (see the related discussion here).
One can already specify a member named __vectorcalloffset__ in PyType_FromSpec. This isn't fully working in the limited API, however. I encountered the following problems.
- The
Py_TPFLAGS_HAVE_VECTORCALL flag is not part of the limited API.
- The
PyVectorcall_NARGS() helper function is not part of the limited API.
- One would normally set
tp_call to the compatibility dispatch routine PyVectorcall_Call. It is, however, also not part of the public ABI.
- Leaving
tp_call unspecified is not an option. PyType_Ready() even throws an exception in type_ready_pre_checks() when tp_call is unspecified.
Pitch
I propose the following changes:
- Adding
Py_TPFLAGS_HAVE_VECTORCALL, PyVectorcall_NARGS(), and PyVectorcall_Call() to the limited API.
- Redundant, but nice: Setting
tp_call to PyVectorcall_Call when PyType_Ready encounters a type that doesn't have this field set.
Note that vector calls can be received and performed. This issue is just about the receiving end.
Previous discussion
See the discord thread https://discuss.python.org/t/ideas-for-forward-compatible-and-fast-extension-libraries-in-python-3-12/15993/12. Reacted by Erlend E. Aasland and C.A.M. Gerlach Reacted by Erlend E. Aasland Activityencukou commented on Jun 7, 2022 encukou commented on Jun 7, 2022 changed the title [-]Receiving vector calls in the Py_LIMITED_API[/-] [+]PEP: Receiving vector calls in the Py_LIMITED_API[/+] on Jun 7, 2022changed the title [-]PEP: Receiving vector calls in the Py_LIMITED_API[/-] [+]Receiving vector calls in the Py_LIMITED_API[/+] on Jun 7, 2022encukou commented on Jun 7, 2022 Combined with exposing the call side of vectorcall (PyObject_Vectorcall, PyObject_VectorcallDict, PyObject_VectorcallMethod), it looks like there are enough changes here for a small PEP. markshannon commented on Jun 16, 2022 PEP 590 makes it clear that tp_call and the vectorcall function must be consistent.
That can be achieved two ways:
- Make the class immutable, so that
__call__ cannot be set.
- If
__call__ is set, then set the vectorcall function pointer to NULL.
encukou commented on Jun 17, 2022 The issue is that if the class is mutable, __call__ can be set from arbitrary Python code. markshannon commented on Jun 17, 2022 Not a problem.
Somewhere in the bowels of typeobject.c there is code that sets cls->tp_call, so
becomes
cls->tp_call = ...
cls->tp_vectorcall = NULL;encukou commented on Jun 17, 2022 Setting __call__ on a class defines how the instances of that class are called.
tp_vectorcall defines how a class object is called (the tp_vectorcall_offset of a metaclass is usually set to offsetof(PyTypeObject, tp_vectorcall)).
It would need to become something like:
cls->tp_call = ...
for inst in all instances of cls:
if isinstance(inst, type):
inst->tp_vectorcall = NULL;
else:
???
markshannon commented on Jun 17, 2022 Indeed, as Victor points out, it isn't tp_vectorcall that needs to change, but whether it is valid.
The code above should read:
cls->tp_call = ...
cls->tp_flags &= ~Py_TPFLAGS_HAVE_VECTORCALL;
cls->tp_version = 0; /* Just to be safe? */markshannon commented on Jun 17, 2022 We might need to invalidate sub-classes as well, it depends on whether PyType_FromSpec allows sub-classing. encukou commented on Aug 5, 2022 PR: #95717
As for setting tp_call automatically, after reviewing the past discussions I prefer leaving this as is: it needs to be set it explicitly. erlend-aasland commented on Aug 8, 2022 AFAICS, this is now implemented. Unless you have further changes planned, let's close this issue. encukou commented on Aug 8, 2022 Next up in this bucket is making vectorcalls. Or see Discourse for a bigger bucket. encukou commented on Aug 8, 2022 The new tests broke buildbots. I'll revert & test more thoroughly. encukou commented on Aug 9, 2022
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|