GeezerVison is showing github.com — C API: _PyObject_VisitManagedDict() function should be public · Issue #107073 · python/cpython
C API: _PyObject_VisitManagedDict() function should be public #107073Descriptiontl; dr: _PyObject_VisitManagedDict() and _PyObject_ClearManagedDict() function should be public.
If a C extension implements a type as a heap type, the type must support the GC protocol: it must implement visit and clear functions, and the dealloc function must DECREF the type.
It seems like if a heap type uses the new Py_TPFLAGS_MANAGED_DICT flag, the visit function must call _PyObject_VisitManagedDict() and the clear function must call _PyObject_ClearManagedDict(). Correct me if I'm wrong.
Problem: _PyObject_VisitManagedDict() and _PyObject_ClearManagedDict() functions are private. IMO they must be public.
Either Py_TPFLAGS_MANAGED_DICT flag must be private, or it should be public and all related helper functions should be public as well.
Py_TPFLAGS_MANAGED_DICT was added to Python 3.11.
_PyObject_VisitManagedDict() and _PyObject_ClearManagedDict() functions were added to Python 3.12.
In Python 3.12, typing.TypeVar, typing.ParamSpec, typing.TypeVarTuple are implemented with Py_TPFLAGS_MANAGED_DICT (and use the 2 helper functions).
In Python 3.13, _asyncio.Future and _asyncio.Task are also implemented with Py_TPFLAGS_MANAGED_DICT (and use the 2 helper functions).
Note: In Objects/typeobject.c, subtype_traverse() calls _PyObject_VisitManagedDict() in some cases, and subtype_clear() calls _PyObject_ClearManagedDict() in some cases.
Linked PRs
Activityvstinner commented on Jul 22, 2023 vstinner commented on Jul 22, 2023 The doc doesn't say much about how a heap type should be implemented:
vstinner commented on Jul 22, 2023 vstinner commented on Jul 23, 2023 vstinner commented on Sep 1, 2023 I'm not sure if _PyObject_GetDictPtr() should be made public or not. markshannon commented on Oct 2, 2023 Just to be clear, the root problem here is that if a class is defined using PyType_FromSpec() and Py_TPFLAGS_MANAGED_DICT is set, there is no way to implement tp_visit and tp_clear.
That can be fixed by adding _PyObject_VisitManagedDict() and _PyObject_ClearManagedDict() (or something with equivalent functionality) to the API so that tp_visit and tp_clear can be implemented.
However, PyType_FromSpec() inherits many of the flaws of statically declaring PyTypeObjects.
We should offer a better way to build classes, so maybe add a PyUnstable_ prefix, as we want to come up with a better approach? markshannon commented on Oct 2, 2023 _PyObject_GetDictPtr() should not be made public.
We should remove it, but not while anyone still uses tp_dictoffset.
markshannon commented on Oct 2, 2023 If the PyUnstable_ prefix had existed when these functions were added, I would have used it.
Having managed dicts is stable, so the functions will continue to have meaning if even the implementation changes.
In that sense they are stable.
However, we want to move the responsibility for visiting and clearing from C extensions to the VM. Once we have done that we will no longer need _PyObject_VisitManagedDict() and _PyObject_ClearManagedDict(), so there is an argument for making them PyUnstable.
On balance, I think they should be public. Making object layout the full responsibility of the VM won't happen for a few releases and even when it does it will need a longish deprecation to remove all the old API. So, we might as well add _PyObject_VisitManagedDict() and _PyObject_ClearManagedDict() to the stable API. vstinner commented on Oct 3, 2023
However, we want to move the responsibility for visiting and clearing from C extensions to the VM.
Py_VISIT() should only be called once by the GC. So either calling PyObject_VisitManagedDict() should be the responsibility of the C extension, or the VM. But it cannot be both. And it cannot change without adding a new type flag, otherwise it would break the backward compatibility, since PyObject_VisitManagedDict() is public in Python 3.13. No?
On balance, I think they should be public. Making object layout the full responsibility of the VM won't happen for a few releases
That sounds like a good practical trade-off. vstinner commented on Oct 3, 2023 I added PyObject_VisitManagedDict() and PyObject_ClearManagedDict() functions to pythoncapi-compat for Python 3.11 and Python 3.12 using _PyObject_GetDictPtr(): python/pythoncapi-compat@a594354 added a commit that references this issue on Oct 3, 2023gh-107073: Mention pythoncapi-compat for PyObject_VisitManagedDict() (#…
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|