GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — gh-140550: Update xxlimited with 3.15 limited API & PEP 697 by encukou · Pull Request #142827 · python/cpython Conversation
|
| /* TODO: This is not quite true yet: there is a race in Xxo_setattro | ||
| * for example. | ||
| */ |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Wouldn't it be better to mark this as Py_MOD_GIL_USED in the meantime, or are you trying to avoid the warning?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
AFAIK, the situation is unchanged since Py_MOD_GIL_NOT_USED was added everywhere in #116882.
I think the intention is/was to make everything thread-safe before some stage of free-threading implementation?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I think we added that everywhere under the assumption that everything was already thread-safe, and anything that should be reported to us as a bug. I'm not sure that applies with our own test suite.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
This is not really a test suite, it supposed to be an example/template showing how to use the Limited API.
The limited API was never compatible with free-threading (yet), so I don't think there's a bug?
There's an earlier PR, #110764, which did this (Py_NOGIL was later renamed to Py_GIL_DISABLED):
+#ifndef Py_NOGIL #define Py_LIMITED_API 0x030d0000 +#endif
... but that seems to defeat the point of this module :/
Maybe we should have not built these modules at all in free-threading builds, and skip the relevant tests?
Doing that now seems like wasted effort though, when mutexes/critical sections are a genuine TODO item for the limited API.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
If it's a template, I'm a little concerned that people will see this and think "oh, this module is thread-safe", when it's really not. The comment helps, but I don't know what percentage of people will ignore it.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Before PyCriticalSection is added to the Stable ABI, we can use PyThread_acquire_lock for per-instance locking. (It's also a nice example of a non-PyObject resource that needs freeing.)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
I'll mark this draft -- this PR makes some shortcomings quite visible, and I guess it's better to address them before suggesting the new API :) |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
| // traverse: Visit all references from an object, including its type | ||
| static int | ||
| Xxo_traverse(PyObject *op, visitproc visit, void *arg) | ||
| Xxo_traverse(PyObject *self, visitproc visit, void *arg) |
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
@colesbury, regarding your post on Discourse:
I think that’s just
PyType_GetBaseByTokenandPyType_GetModuleByToken. It seems pretty rare to need to call them from a tp_traverse handler. If really necessary, I think you can work around that in extensions by calling other APIs.
How would you implement the traverse function here?
Xxo is a subclassable type, and it should work on ABIs with different sizeof(PyObject) (i.e. free-threaded & GIL-ful).
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
By workaround I mean something like:
static Xxo_Type_spec = {
...
.flags = Py_TPFLAGS_IMMUTABLETYPE | ...
};
...
static PyTypeObject *Xxo_Type;
...
Xxo_Type = state->Xxo_Type = (PyTypeObject*)PyType_FromModuleAndSpec(...);
static XxoObject_Data *
Xxo_get_data(PyObject *self)
{
return PyObject_GetTypeData(self, Xxo_Type);
}
We could also avoid all this if we had a PyObject_GetTypeDataByToken().
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
static PyTypeObject *Xxo_Type;
...
Xxo_Type = state->Xxo_Type = (PyTypeObject*)PyType_FromModuleAndSpec(...);
That'll overwrite the pointer in a subinterpreter, or if you otherwise load several copies of the module. It's not something we can recommend.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| } | ||
| if (PyModule_AddType(m, (PyTypeObject*)Str_Type) < 0) { | ||
| if (PyModule_AddType(m, Str_Type) < 0) { | ||
| return -1; |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Don't we have to call Py_DECREF(Str_Type) here (on error)?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the catch!
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| PySlot_FUNC(Py_mod_exec, xx_modexec), | ||
|
|
||
| /* Module state and associated functions */ | ||
| PySlot_SIZE(Py_mod_state_size, (void*)sizeof(xx_state)), |
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I don't think that the (void*) cast is needed here:
| PySlot_SIZE(Py_mod_state_size, (void*)sizeof(xx_state)), | |
| PySlot_SIZE(Py_mod_state_size, sizeof(xx_state)), |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| static PySlot xx_slots[] = { | ||
| /* Basic metadata */ | ||
| PySlot_STATIC_DATA(Py_mod_name, "xxlimited"), | ||
| PySlot_STATIC_DATA(Py_mod_doc, (void*)module_doc), |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Is the (void*) cast useful here?
| PySlot_STATIC_DATA(Py_mod_doc, (void*)module_doc), | |
| PySlot_STATIC_DATA(Py_mod_doc, module_doc), |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.15. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
GH-149785 is a backport of this pull request to the 3.15 branch. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…H-149785) (cherry picked from commit fa81cd9) Co-authored-by: Petr Viktorin <[email protected]>
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.