GeezerVison

FWIW, this change would be unnecessary no objects were ever shared. It may be worth adding such a note to this comment, so it's clear that this extra cleanup code could be removed at that point.

Once we will be sure that it is really impossible to share any object, we can just remove this code.

So one or more objects "owned" by another interpreter (AKA OTHER) could still hold a reference to one of these untracked objects (AKA UNTRACKED). The bug we're fixing demonstrates that's a possibility.

Could it make it harder to break cycles? Could it lead to memory leaks?

How will the UNTRACKED object impact GC for any OTHER object holding a reference to it?

What happens if one of these UNTRACKED objects is involved in a cycle with one or more OTHER objects? Can we still clean them all up?

What happens if several of these UNTRACKED objects are involved in a cycle, but one or more OTHER objects holds a reference to one of them? Can we still clean them all up?

If an object is not tracked by the GC and is part of a ref cycle, the GC is unable to break the reference cycle, and so yes, memory leaks. Previously, Python was crashing. Well, the situation is less bad :-)

This change doesn't introduce the memory leak. An object cannot be tracked in two GC lists at the same time: _PyObject_GC_TRACK() has an assertion for that. The leak was already there.

If an object is created in interpreter 1, it's tracked by the GC of the interpreter 1. If it's copied to the interpreter 2 and the interpreter 1 is destroyed, the interpreter 2 GC is not going to automatically tracks the object. Moreover, the interpreter 1 cannot guess if another interpreter is using the object or not.

IMO untracking all objects is the least bad solution.

IMO the only way to ensure that no memory is leaked is to prevent sharing objects between interpreters, and rely on existing mechanisms (GC and finalizers) to release memory. So continue to convert static types to heap types, continue to update C extensions to the multi-phase initialization, continue moving globals into module state and per-interpreter structures, etc.

so yes, memory leaks. Previously, Python was crashing. Well, the situation is less bad :-)

🙂

In Python 3.8, when the GC state was shared, it seems like any interpreter could run a GC collection. It couldn't happen in parallel, thanks to the "collecting" flag. The GC is not re-entrant and simply does nothing (exit) if it's already collecting.

I cannot say if Python 3.8 was able to break the reference cycles that Python 3.9 and newer can no longer break: when an object in created in an interpreter and then "migrates" to another interpreter.