Description

vstinner

opened on Oct 30, 2023

Issue body actions

In Python 3.13, the private _Py_Identifier API was removed: see PR #108593.

Replacements:

• Use a literal bytes string: replace &PyId_abc with "abc", use "String" variant functions. Slower, but works on all Python versions. This solution can be used on code which is not performance sensitive.
• Initialize a global or "static" variable to only create a Python str object once using PyUnicode_FromString(): less convenient, not compatible with multi-phase init API (PEP 489), not compatible with PEP 684 – A Per-Interpreter GIL.

If we want to add a public API for this, I would prefer to only add the bare minimum API, make it as small as possible:

• Py_IDENTIFIER(name) macro.
• PyUnicode_FromId(&PyId_name) function.
• Py_Identifier structure with public members (needed by Py_IDENTIFIER() macro).
• Maybe also: Py_static_string(name, "...") macro for strings which are not valid C identifiers.

It means that _PyObject_GetAttrId(obj, &PyId_abc) must be replaced with:

PyObject *name = PyUnicode_FromId(&Py_abc);
if (name == NULL) { ... handle error... }
attr = PyObject_GetAttr(obj, name);

Since functions like PyObject_GetAttr() don't accept NULL, so you cannot simply write:

attr = PyObject_GetAttr(obj, PyUnicode_FromId(&Py_abc));

See discussion:

• https://discuss.python.org/t/c-api-how-much-private-is-the-private-py-identifier-api/29190 (July 2023)
• https://mail.python.org/archives/list/[email protected]/thread/DNMZAMB4M6RVR76RDZMUK2WRLI6KAAYS/ (Feb 2022)

Activity

1. vstinner

   added

   topic-C-API

   on Oct 30, 2023
2. encukou commented on Oct 30, 2023

   

   encukou

   on Oct 30, 2023

   Member

   More actions

   Would these be interpreter-specific, or shared? Interned? Immortal?
3. vstinner commented on Oct 30, 2023

   

   vstinner

   on Oct 30, 2023

   MemberAuthor

   More actions

   Would these be interpreter-specific, or shared? Interned? Immortal?

   If possible, it should work in all cases: multiple instances of an extension per interpreter and support multiple interpreters per process with one GIL per interpreter.

   Currently, _PyUnicode_FromId() interns strings, but interned strings are per interpreter which makes it problematic with Per-Interpreter GIL.

   Maybe it's easier to implement if strings are made immortal.
4. vstinner commented on Dec 15, 2023

   

   vstinner

   on Dec 15, 2023

   MemberAuthor

   More actions

   The following private functions removed in Python 3.13 alpha1 were restored in Python 3.13 alpha2:

   • _PyObject_CallMethodId()
   • _PyObject_GetAttrId()
   • _PyUnicode_FromId()
   • _Py_IDENTIFIER()
   • _Py_static_string()
   • _Py_static_string_init()
5. vstinner commented on Dec 20, 2023

   

   vstinner

   on Dec 20, 2023

   MemberAuthor

   More actions

   Since the private API is back and no one managed to propose a replacement so far, I suppose that we can keep this API a little bit longer until a replacement is designed (if needed).
6. vstinner

   closed this as not plannedon Dec 20, 2023

Issue actions