GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — Thread-safe type operations: type lock, QSBR type-cache reclamation, GC stop-the-world, and interpreter optimizations by youknowone · Pull Request #7416 · RustPython/RustPython

Thread-safe type operations: type lock, QSBR type-cache reclamation, GC stop-the-world, and interpreter optimizations #7416

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

youknowone merged 92 commits into RustPython:main from youknowone:typelock

Jul 8, 2026

Uh oh!

There was an error while loading. Please reload this page.

Merged

Thread-safe type operations: type lock, QSBR type-cache reclamation, GC stop-the-world, and interpreter optimizations #7416

Show file tree

Hide file tree

Changes from 1 commit

Commits

Show all changes

92 commits

Select commit Hold shift + click to select a range

09f9961

type lock

youknowone Mar 9, 2026

2a9b982

Drop old PyObjectRef outside type lock to prevent deadlock

youknowone Mar 17, 2026

4b71002

Align type lock behavior with CPython

youknowone Apr 13, 2026

9196dd4

Add PUBLISHED flag bit to RefCount state word

youknowone Jul 5, 2026

c32cb0e

Add QSBR module for deferred memory reclamation

youknowone Jul 5, 2026

9bfa070

Defer memory free of cache-published objects via QSBR

youknowone Jul 5, 2026

59a3127

Wire QSBR checkpoints into thread lifecycle and eval breaker

youknowone Jul 5, 2026

3f93b63

Publish type-cache values to QSBR and bypass cache without a VM

youknowone Jul 5, 2026

06871c4

Use try-incref reads and QSBR-backed swaps in specialization cache

youknowone Jul 5, 2026

6eecfe8

Reset QSBR thread registry after fork in the child

youknowone Jul 5, 2026

020edd3

Add threaded stress test for type cache mutation races

youknowone Jul 5, 2026

d565da5

Restrict qsbr internals to pub(crate) and fix clippy lints

youknowone Jul 5, 2026

cbd20c0

Gate per-instruction QSBR check behind a global pending flag

youknowone Jul 5, 2026

eb633b2

Merge signal and QSBR eval-breaker flags into one atomic word

youknowone Jul 5, 2026

9baa344

Skip freelist reuse for published objects in default_dealloc

youknowone Jul 5, 2026

97dbfcc

Use itoa for PyInt::to_str_radix_10 i64 fast path

youknowone Jul 5, 2026

e4ddae6

Make object attribute dunders wrapper descriptors

youknowone Jul 5, 2026

1ac47f3

Resolve TpSetattro pair on attribute deletion, not just addition

youknowone Jul 5, 2026

8f32450

Generate Opcode::cache_entries/deopt as table lookups

youknowone Jul 5, 2026

efc62e3

Make Opcode/Instruction numeric conversions O(1)

youknowone Jul 5, 2026

7e4c765

Pop CallIsinstance arguments directly instead of collecting into Vecs

youknowone Jul 5, 2026

d9796ea

Inherit tp_new instead of always installing new_wrapper

youknowone Jul 5, 2026

dd72583

Skip locals dict allocation for the init-cleanup shim frame

youknowone Jul 5, 2026

da6633d

Guard CALL_ALLOC_AND_ENTER_INIT with cached function version

youknowone Jul 5, 2026

6fe1fed

Apply rustfmt and ruff-format fixes to files from earlier commits

youknowone Jul 5, 2026

d26a553

Add frame object freelist

youknowone Jul 5, 2026

4c3bb64

Drop frame children directly in tp_clear instead of extracting

youknowone Jul 5, 2026

c197615

Untrack objects before tp_clear and guard cross-thread f_locals

youknowone Jul 5, 2026

cff6798

Skip localsplus heap copy for uniquely referenced dying frames

youknowone Jul 5, 2026

1888e7d

Stage exact-args call arguments in stack slot buffers

youknowone Jul 5, 2026

7517364

Reduce allocations in generic call paths

youknowone Jul 6, 2026

71197eb

Skip redundant exc_info restore on frame exit

youknowone Jul 6, 2026

bc2927c

Apply rustfmt to gc_state.rs

youknowone Jul 6, 2026

de52aea

Avoid empty FuncArgs clone in PyType::call

youknowone Jul 6, 2026

e10c44d

Push freelist husks only after tp_clear and child drops

youknowone Jul 6, 2026

ebc56c9

Resolve __set__ and __delete__ together for the descr_set slot

youknowone Jul 6, 2026

e902ad1

Update type base on __bases__ assignment

youknowone Jul 6, 2026

bc2aba2

Fix rollback order, dead weakrefs, and lock-held drops in set_bases

youknowone Jul 6, 2026

b8d11f2

Reify number sub-slot wrapper once in update_one_slot

youknowone Jul 6, 2026

a7fc01a

Restrict type_cache_after_fork visibility and guard type-cache reads

youknowone Jul 6, 2026

4d7126a

Guard BinaryOpSubscrGetitem with the cached type version

youknowone Jul 6, 2026

18e8474

Call __abstractmethods__ __len__ once in object.__new__

youknowone Jul 6, 2026

6133908

Use i64 fast paths for specialized int add/sub/mul

youknowone Jul 6, 2026

1499b31

Use i64 fast paths for exact int floordiv and remainder

youknowone Jul 6, 2026

70409fa

Skip trashcan and untrack for non-GC-tracked objects in dealloc

youknowone Jul 6, 2026

1b8683b

Merge trashcan depth and defer queue into one thread-local struct

youknowone Jul 6, 2026

f85c30d

Check object layout compatibility on __bases__ and __class__ assignment

youknowone Jul 6, 2026

f7dda03

Stop the world around GC pointer-reading phases

youknowone Jul 6, 2026

bca1ad9

Add threaded GC vs executing-frame stress snippet

youknowone Jul 6, 2026

2769945

Serialize fork and GC stop-the-world requesters

youknowone Jul 6, 2026

a33d16f

Add fork under concurrent GC stop-the-world snippet

youknowone Jul 6, 2026

1b0fa99

Detach while acquiring the import lock

youknowone Jul 6, 2026

92a4d00

Add concurrent-import vs GC deadlock snippet

youknowone Jul 6, 2026

d1e5e97

Create call frames untracked and track generators explicitly

youknowone Jul 6, 2026

ee94840

Track escaped call frames lazily at datastack release

youknowone Jul 6, 2026

2982b66

Guard the tracked-frame localsplus invariant

youknowone Jul 6, 2026

1fa2d5f

Reword frame/dealloc invariant comments and drop needless borrow

youknowone Jul 6, 2026

bd4b371

Move type SetAttr slot rewrite inside the type lock and tighten cache…

youknowone Jul 6, 2026

2ae9fb8

Capture the type version before reading mutable slots in specializers

youknowone Jul 6, 2026

68e8b91

Use _get_method_dict for the test_type check in the test runner

youknowone Jul 6, 2026

ec8b128

Fix lint hooks: cspell dictionary entries, spelling fix, formatting

youknowone Jul 6, 2026

9d44aaf

Rebuild all slots for type and descendants on __bases__ reassignment

youknowone Jul 6, 2026

ab0cddb

Derive the thread-local frame stack from the frame chain

youknowone Jul 6, 2026

ae106ab

Inherit sub-slot fallback into the field being resolved

youknowone Jul 6, 2026

8f4f7a5

Gate type_cache_after_fork to its fork caller and use Self in downcast

youknowone Jul 6, 2026

5f43c79

Unmark test_attr and test_method_call_error in test_monitoring

youknowone Jul 6, 2026

a3f9243

Run tp_new specialization __init__ without a trampoline frame

youknowone Jul 6, 2026

df70284

Replace per-call thread-frame mutex with an atomic top-of-stack (unix)

youknowone Jul 6, 2026

f525b38

Skip exc_info save/restore for callees that never touch the slot

youknowone Jul 6, 2026

98351a5

Return a write-through FrameLocalsProxy from frame.f_locals

youknowone Jul 6, 2026

a9f8b61

Retain the caller frame so f_back resolves after it returns

youknowone Jul 6, 2026

f05926a

Fix debug-build native stack overflow on deep recursion

youknowone Jul 6, 2026

4a92aad

Allocate exception instance __dict__ lazily

youknowone Jul 6, 2026

072bc2a

Allocate exception __dict__ lazily in vm.new_exception

youknowone Jul 6, 2026

c172c74

Validate FrameLocalsProxy.update() arguments

youknowone Jul 6, 2026

2652f4d

Address review nits in frame and faulthandler

youknowone Jul 6, 2026

0499c0a

Use scopeguard for start_the_world in faulthandler dump_all_threads

youknowone Jul 6, 2026

811f230

Gitignore docs/superpowers

youknowone Jul 7, 2026

7982da8

Rename type_bases_slot_rebuild.py to builtin_type_bases.py

youknowone Jul 7, 2026

683f164

Apply rustfmt, ruff, and cspell lint fixes

youknowone Jul 7, 2026

ac2ce2b

Gate unix-only QSBR methods to their call sites

youknowone Jul 7, 2026

f843473

Visit the function closure tuple as a GC edge

youknowone Jul 7, 2026

7ab5f76

Apply formatting hook fix to builtin_type_bases.py

youknowone Jul 7, 2026

6626177

Unmark asyncgen finalization-by-gc tests in test_base_events

youknowone Jul 7, 2026

77e6d31

Unmark test_sni_callback_refcycle in test_ssl

youknowone Jul 7, 2026

d272003

Widen GC stop-the-world gates from unix-only to all threading builds

youknowone Jul 7, 2026

e590e3b

Detach current thread around blocking _winapi/_overlapped waits

youknowone Jul 7, 2026

cdd8908

Format WaitForMultipleObjects allow_threads closure

youknowone Jul 7, 2026

b6e8cbf

Treat concurrent sni_callback removal as no-op in invoke_sni_callback

youknowone Jul 7, 2026

5e2d095

Assert capi refcount on a fresh mortal list instead of the int type o…

youknowone Jul 7, 2026

4f8116d

Traverse and GC-track PyOSError instances

youknowone Jul 8, 2026

db737e1

Unmark test_blockingioerror in test_io

youknowone Jul 8, 2026

File filter

Filter by extension .json (1) .py (1) .rs (7) All 3 file types selected

Viewed files

Clear filters

Conversations

Failed to load comments.

Uh oh!

There was an error while loading. Please reload this page.

Jump to

Jump to file

Failed to load files.

Uh oh!

There was an error while loading. Please reload this page.

Diff view

Unified

Split

Hide whitespace

Diff view

Unified

Split

Hide whitespace

Previous commit

Next commit

Apply rustfmt, ruff, and cspell lint fixes

Reformat with rustfmt, fix import spacing in builtin_type_bases.py with
ruff, and add "pointee" to the cspell word list.

Assisted-by: Claude

Loading branch information

Uh oh!

There was an error while loading. Please reload this page.

youknowone committed Jul 7, 2026

commit 683f16490d269cff6367e857f03704bb368781e9

There are no files selected for viewing

1 change: 1 addition & 0 deletions 1 .cspell.json

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -79,6 +79,7 @@
		"mcache",
		"oparg",
		"opargs",
		"pointee",
		"pyc",
		"reborrow",
		"reborrows",
Expand Down

28 changes: 14 additions & 14 deletions 28 crates/vm/src/builtins/frame_locals_proxy.rs

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -90,7 +90,12 @@ impl FrameLocalsProxy {
		self.frame.framelocalsproxy_getitem(key, vm)
		}

		fn __setitem__(&self, key: PyObjectRef, value: PyObjectRef, vm: &VirtualMachine) -> PyResult<()> {
		fn __setitem__(
		&self,
		key: PyObjectRef,
		value: PyObjectRef,
		vm: &VirtualMachine,
		) -> PyResult<()> {
		self.frame.framelocalsproxy_setitem(key, value, vm)
		}

Expand Down Expand Up		@@ -145,12 +150,7 @@ impl FrameLocalsProxy {
		}

		#[pymethod]
		fn setdefault(
		&self,
		key: PyObjectRef,
		default: OptionalArg,
		vm: &VirtualMachine,
		) -> PyResult {
		fn setdefault(&self, key: PyObjectRef, default: OptionalArg, vm: &VirtualMachine) -> PyResult {
		self.frame
		.framelocalsproxy_setdefault(key, default.unwrap_or_none(vm), vm)
		}
Expand All		@@ -163,9 +163,7 @@ impl FrameLocalsProxy {
		#[pymethod]
		fn update(&self, args: FuncArgs, vm: &VirtualMachine) -> PyResult<()> {
		if !args.kwargs.is_empty() {
		return Err(
		vm.new_type_error("FrameLocalsProxy.update() takes no keyword arguments")
		);
		return Err(vm.new_type_error("FrameLocalsProxy.update() takes no keyword arguments"));
		}
		if args.args.len() != 1 {
		return Err(vm.new_type_error(format!(
Expand All		@@ -183,9 +181,9 @@ impl FrameLocalsProxy {
		} else if let Some(proxy) = other.downcast_ref::<Self>() {
		proxy.snapshot(vm)?.into_iter().collect()
		} else {
		return Err(vm.new_type_error(
		"update() argument must be dict or another FrameLocalsProxy",
		));
		return Err(
		vm.new_type_error("update() argument must be dict or another FrameLocalsProxy")
		);
		};
		for (key, value) in items {
		self.frame.framelocalsproxy_setitem(key, value, vm)?;
Expand Down Expand Up		@@ -229,7 +227,9 @@ impl FrameLocalsProxy {
		impl AsMapping for FrameLocalsProxy {
		fn as_mapping() -> &'static PyMappingMethods {
		static AS_MAPPING: LazyLock<PyMappingMethods> = LazyLock::new(\|\| PyMappingMethods {
		length: atomic_func!(\|mapping, vm\| FrameLocalsProxy::mapping_downcast(mapping).__len__(vm)),
		length: atomic_func!(
		\|mapping, vm\| FrameLocalsProxy::mapping_downcast(mapping).__len__(vm)
		),
		subscript: atomic_func!(\|mapping, needle, vm\| {
		FrameLocalsProxy::mapping_downcast(mapping).__getitem__(needle.to_owned(), vm)
		}),
Expand Down

8 changes: 2 additions & 6 deletions 8 crates/vm/src/coroutine.rs

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -183,9 +183,7 @@ impl Coro {
		} else {
		None
		};
		let (result, entered_frame) = self.run_with_context(jen, vm, \|f\| {
		f.resume(value, vm)
		});
		let (result, entered_frame) = self.run_with_context(jen, vm, \|f\| f.resume(value, vm));
		self.finalize_send_result(result, entered_frame, jen, vm)
		}

Expand All		@@ -211,9 +209,7 @@ impl Coro {
		} else {
		None
		};
		let (result, entered_frame) = self.run_with_context(jen, vm, \|f\| {
		f.resume(value, vm)
		});
		let (result, entered_frame) = self.run_with_context(jen, vm, \|f\| f.resume(value, vm));
		self.finalize_send_result(result, entered_frame, jen, vm)
		}

Expand Down

4 changes: 3 additions & 1 deletion 4 crates/vm/src/exceptions.rs

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -2011,7 +2011,9 @@ pub(super) mod types {
		}
		}
		let payload = Self::py_new(&cls, args, vm)?;
		payload.into_ref_with_type_lazy_dict(vm, cls).map(Into::into)
		payload
		.into_ref_with_type_lazy_dict(vm, cls)
		.map(Into::into)
		}
		}

Expand Down

18 changes: 10 additions & 8 deletions 18 crates/vm/src/frame.rs

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -1491,10 +1491,11 @@ impl Frame {
		vm: &VirtualMachine,
		) -> PyResult<()> {
		self.check_locals_access(vm)?;
		if self.framelocalsproxy_getkeyindex(&key, false, vm)?.is_some() {
		return Err(
		vm.new_value_error("cannot remove local variables from FrameLocalsProxy")
		);
		if self
		.framelocalsproxy_getkeyindex(&key, false, vm)?
		.is_some()
		{
		return Err(vm.new_value_error("cannot remove local variables from FrameLocalsProxy"));
		}
		let extra = self.f_extra_locals.lock().clone();
		if let Some(extra) = extra
Expand All		@@ -1513,10 +1514,11 @@ impl Frame {
		vm: &VirtualMachine,
		) -> PyResult {
		self.check_locals_access(vm)?;
		if self.framelocalsproxy_getkeyindex(&key, false, vm)?.is_some() {
		return Err(
		vm.new_value_error("cannot remove local variables from FrameLocalsProxy")
		);
		if self
		.framelocalsproxy_getkeyindex(&key, false, vm)?
		.is_some()
		{
		return Err(vm.new_value_error("cannot remove local variables from FrameLocalsProxy"));
		}
		let extra = self.f_extra_locals.lock().clone();
		if let Some(extra) = extra
Expand Down

9 changes: 4 additions & 5 deletions 9 crates/vm/src/gc_state.rs

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -641,15 +641,14 @@ impl GcState {
		reason = "assertion over every registered thread slot"
		)]
		for slot in registry.values() {
		let mut cur = slot
		.top_frame
		.load(core::sync::atomic::Ordering::Relaxed)
		let mut cur = slot.top_frame.load(core::sync::atomic::Ordering::Relaxed)
		as *const crate::frame::Frame;
		while !cur.is_null() {
		// SAFETY: frames on a thread's active call stack are
		// alive, and the world is stopped so none can be popped.
		let obj = unsafe { &*crate::Py::<crate::frame::Frame>::from_payload_ptr(cur) }
		.as_object();
		let obj =
		unsafe { &*crate::Py::<crate::frame::Frame>::from_payload_ptr(cur) }
		.as_object();
		let ptr = GcPtr(NonNull::from(obj));
		debug_assert!(
		!unreachable_set.contains(&ptr),
Expand Down

7 changes: 3 additions & 4 deletions 7 crates/vm/src/types/zoo.rs

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -3,10 +3,9 @@ use crate::{
		builtins::{
		asyncgenerator, bool_, builtin_func, bytearray, bytes, capsule, classmethod, code, complex,
		coroutine, descriptor, dict, enumerate, filter, float, frame, frame_locals_proxy, function,
		generator,
		genericalias, getset, int, interpolation, iter, list, map, mappingproxy, memory, module,
		namespace, object, property, pystr, range, set, singletons, slice, staticmethod, super_,
		template, traceback, tuple,
		generator, genericalias, getset, int, interpolation, iter, list, map, mappingproxy, memory,
		module, namespace, object, property, pystr, range, set, singletons, slice, staticmethod,
		super_, template, traceback, tuple,
		type_::{self, PyType},
		union_, weakproxy, weakref, zip,
		},
Expand Down

4 changes: 2 additions & 2 deletions 4 crates/vm/src/vm/mod.rs

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -44,10 +44,10 @@ use crate::{
		warn::WarningsState,
		};
		use alloc::{borrow::Cow, collections::BTreeMap};
		#[cfg(all(unix, feature = "threading"))]
		use core::sync::atomic::AtomicI64;
		#[cfg(all(not(unix), feature = "threading"))]
		use core::ptr::NonNull;
		#[cfg(all(unix, feature = "threading"))]
		use core::sync::atomic::AtomicI64;
		use core::{
		cell::{Cell, OnceCell, RefCell},
		sync::atomic::{AtomicBool, AtomicU64, Ordering},
Expand Down

1 change: 0 additions & 1 deletion 1 extra_tests/snippets/builtin_type_bases.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
		@@ -1,6 +1,5 @@
		from testutils import assert_raises


		# Reassigning __bases__ must rebuild slot dispatchers for the type and all its
		# descendants: a slot whose method left the new MRO must be reset, not left stale.

Expand Down

Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.