GeezerVison

Do we want to call this gc_visit_heaps_lock_held per our discussion on naming these style of functions?

It'd be nice to factor this into a helper like gc_init_refs or gc_maybe_init_refs that is used here and in update_refs

This could similarly be a gc_restore_refs or gc_maybe_restore_refs and used below

Is it worth stopping the world here? It seems like we're fine to race with ob_tid from a correctness stand point, but it seems like a lot of objects are going to end up with merged ref counts in a multithreaded environment!

We need to stop the world for the heap traversal (gc_visit_heaps) to be thread-safe... which I forgot to add here. I'll also add an assert to gc_visit_heaps() that the world is stopped.

This doesn't merge most refcounts. They get restored from the mimalloc data structure. They'll only get merged if the owning thread has already exited.

Ditto to get referrers

It seems like this needs to be an atomic operation? Can this just be a call to _Py_ExplicitMergeRefcount?

The other threads in the interpreter must be paused, so no atomics necessary. I'll add an assert here.

We could probably use _Py_ExplicitMergeRefcount(), but it seems convenient to avoid the atomic operations.

So I think if I understand this correctly what's going to happen with the thread ID and any reference count changes that may happen at this point, but the assumption is that the thread ID is never going to actually clash with an object reference in the work list. And then anything which does have a reference count operation will just become merged when we restore it?

Yeah, the key assumption is that _PyThread_Id() is itself a pointer to a distinct object so never conflicts with an object in the worklists. The implementation of _PyThread_Id() uses the address of the platform's thread control block or equivalent.

On the GC side, we abuse ob_tid for two purposes:

• gc_refs, which can have arbitrary values (might conflict!), but this is only happens while other threads are paused, so nobody sees it.
• worklist pointers, which must not conflict with _PyThread_Id()

Some worklists are only used while other threads are paused (e.g., in _PyGC_GetObjects()). Other worklists are created when threads are paused, but still used while other threads may be running (e.g., unreachable, wrcb_to_call).

If the worklist continues to be used while other threads are running, then there are two other important considerations:

• The worklist holds a strong reference to the object so that it can't be freed while in the worklist.
• The object's reference count fields are merged before adding it to the worklist. This avoids some other thread trying to merge the refcount fields, which would be messy. (It also makes the deallocation happen more quickly when we get around to calling tp_clear).