GeezerVison is showing github.com — [C API] Private functions used by mypyc · Issue #121489 · python/cpython
[C API] Private functions used by mypyc #121489DescriptionI've been working on adding support for 3.13 to mypyc. After updating the codebase, there are still some functions left we depend on which are now private after #106320. Would it make sense to export these again? That would make adopting 3.13 a bit easier. /CC @vstinner
_PyBytes_Join defined in pycore_bytesobject.h_PyObject_CallMethodIdObjArgs defined in pycore_call.h_PyType_CalculateMetaclass defined in pycore_object.h_PyUnicode_EQ defined in pycore_unicodeobject.h_PyUnicode_FastCopyCharacters defined in pycore_unicodeobject.h (this could be replaced with PyUnicode_CopyCharacters but it's used in a simplified copy of _PyUnicode_JoinArray which also uses _PyUnicode_FastCopyCharacters).
Other functions include these two. However, as they are inlined, it isn't a problem to include them.
_PyObject_CallMethodIdNoArgs and _PyObject_CallMethodIdOneArg defined in pycore_call.h
It would be fine to keep all of them in internal/pycore_* and just add PyAPI_FUNC.
/CC @hauntsaninja
Linked PRs
Activityvstinner commented on Jul 8, 2024
_PyBytes_Join defined in pycore_bytesobject.h
You can call something like:
PyObject *empty = PyBytes_FromString("");
PyObject *res = PyObject_CallMethod(empty, "join", ...);
Py_DECREF(empty);
_PyObject_CallMethodIdObjArgs defined in pycore_call.h
We are trying to get rid of _Py_IDENTIFIER() API. You should use your own mecanism for constant strings.
You can call PyObject_CallMethodObjArgs().
_PyType_CalculateMetaclass defined in pycore_object.h
For this one, I don't know. You might reimplement it, use the internal C API, or we should expose it as a public C API.
_PyUnicode_EQ defined in pycore_unicodeobject.h
You can use PyUnicode_Compare(). Maybe we should expose it.
_PyUnicode_FastCopyCharacters defined in pycore_unicodeobject.h (this could be replaced with PyUnicode_CopyCharacters but it's used in a simplified copy of _PyUnicode_JoinArray which also uses _PyUnicode_FastCopyCharacters).
Would you mind to elaborate your use case? Can you use the new https://docs.python.org/dev/c-api/unicode.html#pyunicodewriter API instead? cdce8p commented on Jul 8, 2024 ContributorAuthor More actions vstinner commented on Jul 8, 2024 vstinner commented on Jul 12, 2024 Reacted by Erlend E. Aasland vstinner commented on Jul 12, 2024
_PyBytes_Join defined in pycore_bytesobject.h
I created issue gh-121645: [C API] Add PyBytes_Join() function. vstinner commented on Aug 2, 2024 @cdce8p: Python 3.13rc1 has been released. What's the mypyc status? Which functions do you still need? cdce8p commented on Aug 2, 2024 ContributorAuthor More actions
@cdce8p: Python 3.13rc1 has been released. Which functions do you still need?
I believe we've now resolved all issues here. There is one open PR on the mypy side which needs to get merged but that should be it python/mypy#17526.
What's the mypyc status?
Not strictly related to this issue. There are still two test issues with mypyc before we can support 3.13. Maybe you've an idea.
- mypyc still uses some functions from
internal/pycore_*.h. I've added #define Py_CORE_BUILD which works. However, there seems to be an issue with linking mimalloc. This is one of the error messages. pythonsupport.h is part of mypyc.
/usr/include/python3.13/internal/mimalloc/mimalloc.h:433:1: error: template with C linkage
433 | template<class T> struct _mi_stl_allocator_common {
| ^~~~~~~~
In file included from /usr/include/python3.13/internal/pycore_runtime.h:17,
from /usr/include/python3.13/internal/pycore_pystate.h:12,
from /usr/include/python3.13/internal/pycore_call.h:12,
from pythonsupport.h:21,
from CPy.h:12,
from test_capi.cc:5:
/usr/include/python3.13/internal/pycore_interp.h:4:1: note: ‘extern "C"’ linkage started here
4 | extern "C" {
| ^~~~~~~~~~
In file included from /usr/include/python3.13/internal/pycore_mimalloc.h:39,
from /usr/include/python3.13/internal/pycore_interp.h:31,
from /usr/include/python3.13/internal/pycore_runtime.h:17,
from /usr/include/python3.13/internal/pycore_pystate.h:12,
from /usr/include/python3.13/internal/pycore_call.h:12,
from pythonsupport.h:21,
from CPy.h:12,
from test_capi.cc:5:
- The other category of errors is
extension module '...' is already cached. I've bisected that to gh-117953: Track Extra Details in Global Extensions Cache #118532. It might be something which can be fixed in the mypyc test runner.
Traceback (most recent call last): (diff)
File "driver.py", line 1, in <module> (diff)
from native import f1 (diff)
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load (diff)
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked (diff)
File "<frozen importlib._bootstrap>", line 921, in _load_unlocked (diff)
File "<frozen importlib._bootstrap>", line 819, in module_from_spec (diff)
File "<frozen importlib._bootstrap>", line 782, in _init_module_attrs (diff)
SystemError: extension module 'native' is already cached (diff)
vstinner commented on Aug 5, 2024
error: template with C linkage
The problem is that mimalloc is included inside extern "C" { ... }.
Ah, another issue was already created: issue gh-122584. vstinner commented on Aug 5, 2024
SystemError: extension module 'native' is already cached (diff)
Please open a separated issue for that. vstinner commented on Aug 27, 2024 @cdce8p: Can you close the issue? Or are there still issues not tracked by other open issues? cdce8p commented on Aug 27, 2024 ContributorAuthor More actions
@cdce8p: Can you close the issue? Or are there still issues not tracked by other open issues?
Yes. Thanks again for your help! vstinner commented on Sep 25, 2024
_PyUnicode_EQ defined in pycore_unicodeobject.h
I created #124502 to add PyUnicode_Equal() function.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|