GeezerVison

Documentation build overview

📚 cpython-previews | 🛠️ Build #35331167 | 📁 Comparing a41538d against main (aed4ad8)

  🔍 Preview build  

Hi @ZeroIntensity thank you for the PR.

Immortal types are fixing the free-threaded slowdown for both PRs numpy/numpy/pull/32853 and numpy/numpy/pull/32552 about as well as the tp_free trick.

Benchmarks:

numpy/numpy/pull/32853 types (flagsobj, _array_converter), free-threaded, million calls/s**

linspace, unique and delete show no difference beyond noise in any build.

numpy/numpy/pull/32552 types (neighborhood iterator, MapIter), free-threaded, million calls/s**

GIL build, one thread, ns per call (lower is better)

import numpy takes the same time in every build (about 58 ms free-threaded, 52 ms GIL).

@ZeroIntensity wasn't there another approach you were thinking about for this?

Per our discussion during the sprint, I think we reached a consensus that NumPy can use static types for things it would otherwise immortalize, right? As far as I can tell, it should be possible to share third-party static types across interpreters; the only requirement for subinterpreters is to use multi-phase initialization, which doesn't disallow static types. (If you find bugs with that, we should fix those!)

@prathamhole14 @mattip does that work for you two?

As far as I know, static types are incompatible with the limited API. If CPython wishes to allow c-extension authors to support subinterpreters, free-threading, and the limited API in a performant way, I think we need to be able to declare heap types immutable and immortal.

The limited API already has performance problems (all the Py_INCREF calls can't be inlined, for example), which is why you should distribute per-version wheels anyway. Could NumPy use heap types when compiling for the limited API, and then use static types otherwise?

The Py_INCREF tax when using the limited API is indeed real and measurable, but fixed per call, with no thread contention. Being able to mark types as immortal solves problems of thread contention that go beyond the Py_INCREF tax, and allow recouping much of the performance impact when using multiple threads. Did the discussion at the sprint touch on the data @prathamhole14 showed above that shows how immortal objects scale with threads in the free threading (default) build? I find it a bit confusing that CPython would prefer a crippled limited API rather than showcasing it as the preferred alternative.

you should distribute per-version wheels anyway

Is this the general consensus?

I think it'd be useful for NumPy to be able to optionally distribute limited API wheels on new platforms where we don't necessarily want to incur the cost of building several version-specific wheels.

Long-term, I think it would be great for NumPy to be able to migrate to using the limited API. But also NumPy is just about the hardest package to do this for because of the NumPy C API. I'm not at all surprised to find there are significant issues that might require several iterations to get right, allowing us to migrate without incurring large trade-offs.

Is this the general consensus?

I'm not sure it's written down, but I think that's been the understanding whenever I've talked to people about this.

But now that I think about it, this wouldn't help for limited API builds anyway, because it won't be in the limited API. (We don't want to tie ourselves to immortality forever, so unfortunately there's a very slim chance that anything like this will make it into the limited API.)

As far as I can tell, it should be possible to share third-party static types across interpreters; the only requirement for subinterpreters is to use multi-phase initialization, which doesn't disallow static types. (If you find bugs with that, we should fix those!)

Although CPython only requires multi-phase init and nothing checks for static types but for the kind of subinterpreter concurrent.interpreters creates (each with its own GIL) static types are not safe on normal GIL builds.

I tried this with a small extension (multi-phase init, declaring Py_MOD_PER_INTERPRETER_GIL_SUPPORTED).
With no threads at all the subclassing a static type in the main interpreter and then in a subinterpreter crashes the process (double free or corruption on 3.14.6, free(): invalid pointer on 3.15.0rc2) every time (the same type as a heap type works totally fine).

I asked AI to investigate: CPython keeps one __dict__ and __mro__ and subclass list for a third-party static type, shared by all interpreters. CPython's own static types get a separate copy per interpreter (gh-94673) but extensions can't use that.

For NumPy this matters because import numpy itself subclasses ndarray.

I asked AI to investigate: CPython keeps one dict and mro and subclass list for a third-party static type, shared by all interpreters. CPython's own static types get a separate copy per interpreter (#94673) but extensions can't use that.

Ok, I think we should fix that. I was under the impression that was what we already did for user-defined static types, but I guess not.