GeezerVison is showing github.com — Limited API way of assigning to immutable types and/or making a class immutable · Issue #121654 · python/cpython
Limited API way of assigning to immutable types and/or making a class immutable #121654DescriptionFeature or enhancement
Proposal:
This is one of the uglier bits of Cython's limited API implementation.
When we create our extension type ("cdef classes") we want them to be presented to users as immutable (mainly because that's what they are in non-limited-API modes and we want it to remain the same).
However, we do want to assign things to the type dict just after type creation (i.e. before it's presented to the user). That's for 2 things:
cdef class Example:
class_attribute = 1 # needs assigning
def f(self): # also needs assigning
passWe make f our own function type (to get better introspection that the default Python C function), so need to be able to assign those functions into the class namespace (i.e. we don't want to just use PyMethodDef).
What doesn't work:
PyObject_SetAttr - fails because the type is immutable- getting
__dict__ - returns a read-only proxy
PyType_GetDict - not in the Limited API.
My current workaround is to use PyObject_GenericSetAttr. This is supposed to be used as a good initial value of tp_setattro for mutable types. However, it doesn't check the mutability of the type, and does succeed in modifying immutable types. However, it feels like I'm misusing non-documented unintended behaviour in something.
Solutions I think could work:
- expose
PyType_GetDict,
- some official way to set an attribute of an immutable type (that could just be deciding my
GenericGetAttr way is actually fine),
- some way to change a mutable type to immutable (so we create it as mutable, make the additions we need, then change it to immutable).
- something else I haven't thought of?
@markshannon @encukou @vstinner tagging you as people who might have an interest in this. Not urgent though - we have a functioning workaround.
Has this already been discussed elsewhere?
No response given
Links to previous discussion of this feature:
No response
Linked PRs
Activityencukou commented on Jul 18, 2024 Hmm. Would it work to have a slot like tp_getset, tp_members or tp_methods, but with a name and value, used to populate for class attributes?
IOW, can you create the values you're assigning before you have the class object?
API for changing a mutable type to immutable sounds doable. It might even be a better way to create types: create the object, then set slots/attributes (using setattr, or an “apply slots” function to reduce API calls), then “freeze” it if it's to be immutable. See markshannon/New-C-API-for-Python#4 (comment) for some previous discussion. da-woods commented on Jul 19, 2024 ContributorAuthor More actions
IOW, can you create the values you're assigning before you have the class object?
I think that would work. It's not how we currently do it, but I don't think anything we do relies on the class existing first.
Would it work to have a slot like tp_getset, tp_members or tp_methods, but with a name and value, used to populate for class attributes?
I'm not completely sure - we currently use statically allocated specs, and tables of slots for these (which wouldn't work, for this case). However I don't think it's a strict requirement. vstinner commented on Jul 29, 2024
some way to change a mutable type to immutable (so we create it as mutable, make the additions we need, then change it to immutable).
I would prefer this approach, rather than leaving "holes" where an "immutable" type can be modified on purpose. Can you propose an API for that? encukou commented on Jul 29, 2024 Looks like the API could be simple, but I haven't started trying to implement it:
int PyType_Freeze(PyTypeObject* type): Turn a mutable class into an immutable one.
To make it more usable, we should add:
int PyType_ApplySlots(PyTypeObject* type, PyType_Slot *slots, Py_ssize_t nslots): apply a PyType_Slot[]: array to a mutable class. (nslots=-1 for a zero-terminated array.) Can only handle some slot types, for example it can't set tp_base or (at least in the initial implementation) tp_members.
and perhaps some parts of “new type creation” and GUPOL. da-woods commented on Jul 29, 2024 ContributorAuthor More actions
int PyType_Freeze(PyTypeObject* type): Turn a mutable class into an immutable one.
That was going to be my suggestion. I think this makes more sense than a general mechanism for changing type flags.
PyType_ApplySlots
I don't think Cython would need this - everything we want to do before immortalization can be done with PyObject_SetAttr. That doesn't mean it isn't useful - just that I don't have a use-case :) vstinner commented on Jul 30, 2024 I wrote PR gh-122457 to implement PyType_Freeze(). da-woods commented on Aug 4, 2024 ContributorAuthor More actions I've moved away from the PyObject_GenericSetAttr-hack because it turns out that someone's already broken it in the main branch of Python. I've found a different (slightly less hacky) way to get access to the type's __dict__ and modify it.
I still think PyType_Freeze is the nicer long-term solution though, so that remains useful to have.
But if you're ever considering modifying PyObject_GenericSetAttr and are wondering if you should do it because it might break Cython then don't worry - someone else got there first. wjakob commented on Sep 3, 2024 @da-woods — I'm curious about the motivation of making Cython's classes immutable by default. Is this to benefit from performance improvements related to the specializing interpreter? Or simply to avoid bugs due to accidental modification of a type. da-woods commented on Sep 3, 2024 ContributorAuthor More actions
Or simply to avoid bugs due to accidental modification of a type.
Mostly this.
Also because the existing non-Limited API version uses static types which are automatically immutable and we find it simpler if the two modes of compilation have the same behaviour. Reacted by Wenzel Jakob, Victor Stinner, Erlend E. Aasland and AraHaan vstinner commented on Oct 25, 2024 PyType_Freeze() is now implemented by change db96327.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|