GeezerVison

We should note that it's intended for highly-specific cases where either:

1. The object is not tracked by the GC (so DRC doesn't work).
2. The contention occurs in native code calling Py_INCREF, where DRC does not apply.

We should document that some objects may not support immortalization in the future and that some objects will break when immortalized. (The second part should be in a .. warning note.)

This seems kind of arbitrary, can you add a comment explaining why 3 is used?

Why is the shared refcount being changed?

Don't we need to also check that ob_ref_shared == 0?

Let's expose PyUnstable_SetImmortal as a helper and add tests using some more exotic objects. I'm still not very keen on the idea that "owned by current thread" implies "safe to immortalize".

Here's an outline:

static PyObject *
pyobject_set_immortal(PyObject *self, PyObject *op)
{
    return PyLong_FromLong(PyUnstable_SetImmortal(op));
}

import _testcapi

class TestWhatever(TestCase):
    def test_set_immortal(self):
        for obj, expected_result in [(module_object, 1), (shared_module_object, 0), ...]:
            self.assertEqual(_testcapi.pyobject_enable_deferred_refcount(module), expected_result)

Ok, looking at my version, we need to do a few other things here:

1. The interpreter stack might have some live references to the object, and it currently thinks that the object is mortal. We need to undo that (see this function).
2. I think we need to disable deferred reference counting on the object in case there are some queued references that will break later. I achieved this by exposing this function from gc_free_threading.c and calling _PyObject_MergePerThreadRefcounts.
3. It's probably a good idea to lower the reftotal so -Xshowrefcount doesn't break. (See this loop.)
4. If you want to support calling PyUnstable_SetImmortal on objects that have a non-zero ob_ref_shared as well, then you also need to make a stop-the-world pause (_PyEval_StopTheWorld/_PyEval_StartTheWorld) to ensure that the object is in a consistent state when immortalizing it.

This also furthers my suspicion that accidental immortalization does not work correctly.

Note that my implementation was also designed to eventually deallocate the immortal object, so there are some things in my PyUnstable_Immortalize that we don't need to do in PyUnstable_SetImmortal (namely the _Py_immortal stuff), but the above list makes sense to me even if the object isn't ever deallocated.

• 4: Well, I'll assume stop the world likely not worth the effort. And I expect without that, there could be arbitrary increfs/decrefs in-flight on ob_ref_shared.
  So, unless we assume it's OK if (maybe except the flags space, I'll guess that only gets mutated from the owning thread/during stop-the-world) ob_ref_shared must be considered undefined for immortal objects. Otherwise, I expect it is best to just close this PR.

• 3: Not sure I follow why it matters that immortalizing leaves the total refcount untouched. Immortalizing seems like "arbitrary code" and arbitrary code may modify it?

• 1-2: This is maybe just more complicated then I think, I can't follow that deeply unfortunately. Part of me keeps wondering if there are two competing concepts of "immortal" here, but I just don't know (the best I can do is find this comment in PyStackRef_CheckValid right now:)

  /* Can be immortal if object was made immortal after reference came into existence */
  assert(!_Py_IsStaticImmortal(obj));
  

  (and it isn't even clear to me that the current check would be enough to protect if that wasn't.)

Yeah, I'm not sure what the difference between a "static immortal" and "immortal" is.

For 3, it would just be to improve debugging. Lowering the global refcount would make it so -Xshowrefcount shows 0 leaked references at the end.