GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — C API type slots by youknowone · Pull Request #8435 · RustPython/RustPython

Conversation

@youknowone

@youknowone youknowone commented Aug 3, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

@bschoenmaeckers could you check if this is a working design? tried not to increase runtime cost for non-capi path

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Copilot AI review requested due to automatic review settings August 3, 2026 00:07
@coderabbitai

coderabbitai Bot commented Aug 3, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Explain Change →

Caution

Review failed

The pull request is closed.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

  • X
  • Mastodon
  • Reddit
  • LinkedIn
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Copilot started reviewing on behalf of youknowone August 3, 2026 00:07
View session
Copilot AI reviewed Aug 3, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds initial support for C extension–provided type slots (starting with tp_new) by storing C ABI function pointers in a per-heap-type table and routing RustPython’s slot dispatch through a shared trampoline. It also exposes a minimal C-API surface to install and query these slots, plus tests validating inheritance and safety behavior.

Changes:

  • Introduces a CSlots table (owned by heap types) and a c_new_trampoline to dispatch tp_new supplied from C.
  • Threads c_slots inheritance/identity through PyTypeSlots and uses it in the “is not safe” __new__ check.
  • Adds PyType_GetSlot / Py_tp_new support in crates/capi with tests, and reuses shared C-call utilities in methodobject.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
crates/vm/src/types/slot.rs Adds c_slots tracking on PyTypeSlots, plus new_identity() and inheritance/override logic integration.
crates/vm/src/types/mod.rs Exposes the new c_slots module and re-exports selected C-slot types/trampolines.
crates/vm/src/types/c_slots.rs New module implementing C slot tables, trampolines, and shared argument/return conversion utilities.
crates/vm/src/builtins/type.rs Stores owned C-slot tables on heap types and adjusts tp_new identity comparisons for safety checks.
crates/capi/src/typeobject.rs New C-API glue for installing/querying tp_new via CSlots, with unit tests.
crates/capi/src/methodobject.rs Reuses the shared (args, kwds) split + return conversion helpers for C function calls.
crates/capi/src/lib.rs Registers the new typeobject module.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread crates/vm/src/types/slot.rs Outdated
Comment on lines +911 to +914
let owns_table = self.slots.c_slots.load().is_some()
&& self.slots.c_slots.load()
!= self.base.deref().and_then(|base| base.slots.c_slots.load());
if owns_table {
Comment thread crates/vm/src/builtins/type.rs
Comment on lines +296 to 299
/// The C slot table this type owns, once an extension has filled one in.
/// Types that inherit from it point at this same table.
pub c_slots: PyRwLock<Option<OwnedCSlots>>,
pub specialization_cache: TypeSpecializationCache,
Comment thread crates/capi/src/typeobject.rs Outdated
Comment on lines +26 to +29
/// Install a C `newfunc` as the type's tp_new.
///
/// `ty` must be a heap type that does not already define `__new__` itself.
pub fn set_tp_new(vm: &VirtualMachine, ty: &Py<PyType>, tp_new: newfunc) -> PyResult<()> {
@youknowone youknowone mentioned this pull request Aug 9, 2026
Allow c functions in tp_new type slot #8346
Closed
bschoenmaeckers
bschoenmaeckers suggested changes Aug 19, 2026
View reviewed changes
Comment thread crates/capi/src/typeobject.rs Outdated
Comment on lines +50 to +62
pub unsafe extern "C" fn PyType_GetSlot(ty: *const PyTypeObject, slot: c_int) -> *mut c_void {
let ty = unsafe { &*ty };
let Some(c_slots) = ty.slots.c_slots() else {
return ptr::null_mut();
};
match CSlotId::from_raw(slot) {
Some(CSlotId::TpNew) => c_slots
.new
.load()
.map_or(ptr::null_mut(), |f| f as *mut c_void),
None => ptr::null_mut(),
}
}

@bschoenmaeckers bschoenmaeckers Aug 19, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not work. As PyType_GetSlot may be called on all types, so this will fail for builtin types. This is also required for extending classes because it requires fetching tp_new of the base type, which is almost always builtins.object.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bschoenmaeckers

bschoenmaeckers commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

I like the approach but this unfortunately does not work. When constructing heaptypes using the c-api, a type requires the tp_new of the base type. This is most of the time just object which we can fill in a c-slot trivially. To support extending all RustPython defined types we need to define the C tp_new slot for all types. Which might defeats the purpose separate slot definitions. Is there a way to make both C and Rust slots interchangeable?

I tested your approach in this commit bc481ed. Which fails this assert in pyo3: https://github.com/PyO3/pyo3/blob/4df5ac6296fa9c9a375d874082a8ad90e43f9b2d/src/internal/pyclass_init.rs#L50

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@youknowone

youknowone commented Aug 19, 2026

Copy link
Copy Markdown
Member Author

Thank you for the review!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bschoenmaeckers

bschoenmaeckers commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

CPython semantics are “call base tp_new with subtype as argument”, not “call subtype’s current tp_new”. So we need a extern "C" tp_new for all types. We might be able to fill in the c slot automatically for all types even when defined in rust?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@youknowone
youknowone force-pushed the main branch from d1c4925 to 3191eca Compare September 16, 2026 17:13
@youknowone
youknowone force-pushed the c-api-type-slots branch from aaa9f3c to 2c3d5a8 Compare September 16, 2026 17:17
@bschoenmaeckers

bschoenmaeckers commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

After some experimenting with this, it makes me want FnStatic. This allow us to create a generic C trampoline for all rust defined type slots.

fn get_c_tp_new<F>(
    _: F,
) -> extern "C" fn(*mut PyTypeObject, *mut PyObject, *mut PyObject) -> *mut PyObject
where
    F: FnStatic(PyTypeRef, FuncArgs, &VirtualMachine) -> PyResult,
{
    extern "C" fn trampoline<F>(
        subtype: *mut PyTypeObject,
        args: *mut PyObject,
        kwargs: *mut PyObject,
    ) -> *mut PyObject
    where
        F: FnStatic(PyTypeRef, FuncArgs, &VirtualMachine) -> PyResult,
    {
        unsafe { F::call_static(&*subtype.to_owned(), tuple_to_args(args), dict_to_kwargs(kwargs)) }
    }

    trampoline::<F>
}

 let c_tp_new = ty.slots.new.load().map(|rust_tp_new| get_c_tp_new(rust_tp_new)

I'm not sure how we can do this in todays ✨ rust.

  • https://lang-team.rust-lang.org/design_notes/fn_type_trait_impls.html#fnstatic-trait
  • Tracking Issue for FnStatic and FnPtr::addr/from_ptr/as_ptr and Code rust-lang/rust#148768
youknowone reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bschoenmaeckers

bschoenmaeckers commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

In todays rust we may have to use a static dispatch table. What do you think of this solution?

fn get_c_tp_new<F: Any + Send + Sync>(
    rust_tp_new: F,
) -> extern "C" fn(*mut PyTypeObject, *mut PyObject, *mut PyObject) -> *mut PyObject
where
    F: Fn(PyTypeRef, FuncArgs, &VirtualMachine) -> PyResult,
{
    extern "C" fn trampoline<F: Any + Send + Sync>(
        subtype: *mut PyTypeObject,
        args: *mut PyObject,
        kwargs: *mut PyObject,
    ) -> *mut PyObject
    where
        F: Fn(PyTypeRef, FuncArgs, &VirtualMachine) -> PyResult,
    {
        let entry = SLOT_CACHE
            .get(&TypeId::of::<F>())
            .expect("could not find tp_new trampoline");
        let func = entry.downcast_ref::<F>().expect("trampoline type mismatch");
        with_vm(|vm| unsafe {
            let args = FuncArgs::new(
                tuple_to_args((&*args).try_downcast_ref(vm)?),
                kwargs
                    .as_ref()
                    .map(|kwargs| dict_to_kwargs(vm, kwargs.try_downcast_ref::<PyDict>(vm)?))
                    .transpose()?
                    .unwrap_or_default(),
            );
            func((&*subtype).to_owned(), args, vm)
        })
    }

    static SLOT_CACHE: LazyLock<DashMap<TypeId, Box<dyn Any + Send + Sync>>> =
        LazyLock::new(DashMap::new);
    SLOT_CACHE
        .entry(TypeId::of::<F>())
        .or_insert_with(|| Box::new(rust_tp_new));
    trampoline::<F>
}
youknowone reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@youknowone
youknowone force-pushed the c-api-type-slots branch from 2c3d5a8 to 965e4e0 Compare September 22, 2026 21:49
@youknowone

youknowone commented Sep 22, 2026

Copy link
Copy Markdown
Member Author

@bschoenmaeckers Thank you so much! And I am sorry for late response. I am going to try your suggestion without table

bschoenmaeckers reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@codspeed

codspeed Bot commented Sep 22, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 62 untouched benchmarks
⏩ 4 skipped benchmarks1


Comparing youknowone:c-api-type-slots (2fc0aca) with main (7f314dd)

Open in CodSpeed

Footnotes

  1. 4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@youknowone
youknowone force-pushed the c-api-type-slots branch from d9f1e3a to 6ab3f65 Compare September 23, 2026 03:07
bschoenmaeckers
bschoenmaeckers suggested changes Sep 23, 2026
View reviewed changes
Comment thread crates/capi/src/typeobject.rs Outdated

@bschoenmaeckers bschoenmaeckers Sep 23, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you move this to the object/pytype.rs file?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

youknowone reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@youknowone youknowone Sep 24, 2026

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved crates/capi/src/typeobject.rs into crates/capi/src/object/pytype.rs and dropped the extra typeobject module. Slot APIs are re-exported through object as before.

— commented by Grok 4.6

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bschoenmaeckers reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@bschoenmaeckers

bschoenmaeckers commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

Please limit this to PyType_GetSlot only. I already have code for PySlot & PyType_Slot handling.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bschoenmaeckers
bschoenmaeckers suggested changes Sep 25, 2026
View reviewed changes
Comment thread crates/vm/src/object/c_body.rs Outdated
const ALIGN: usize = 16;

/// Instance payload whose body is a C-owned byte buffer of `basicsize` bytes.
pub struct PyCBody {

@bschoenmaeckers bschoenmaeckers Sep 25, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this will work. The last time I checked it is only possible for a type to have 1 Payload type with data in it. So for a C class extending a PyList, it needs both PyCBody & PyList holding data.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@youknowone
youknowone force-pushed the c-api-type-slots branch 2 times, most recently from 2fc0aca to 4c0b716 Compare September 30, 2026 13:00
@bschoenmaeckers bschoenmaeckers mentioned this pull request Oct 1, 2026
Add first draft of PyType_FromSlots #8918
Merged
@youknowdot youknowdot mentioned this pull request Oct 10, 2026
Export the stable ABI PyCFunction_Call shim #9055
Merged
@bschoenmaeckers

bschoenmaeckers commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

What is the status of this?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bschoenmaeckers

bschoenmaeckers commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

@youknowone Do you have time to finish this? Or should I take it over?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bschoenmaeckers bschoenmaeckers mentioned this pull request Oct 27, 2026
Add support for tp_new slot in the c-api #9407
Open
@youknowone
youknowone force-pushed the c-api-type-slots branch from 4c0b716 to 5c4325b Compare November 5, 2026 14:26
youknowone and others added 2 commits November 6, 2026 00:32
@youknowone
capi: pass NULL kwargs to keywordless METH_KEYWORDS calls
d7170ae
A METH_VARARGS | METH_KEYWORDS function received an empty dict when the
call had no keywords, where the calling convention passes NULL. A callee
that rejects keywords by testing the pointer saw a non-NULL kwargs.
The METH_FASTCALL | METH_KEYWORDS path already passed a NULL kwnames.

Name the four function pointer types the PyMethodPointer union holds
instead of spelling out each signature inline.

Assisted-by: Claude
@bschoenmaeckers @youknowone
Allow c functions in tp_new type slot
5e5918f
youknowone added 7 commits November 6, 2026 00:32
@youknowone
Reach a C tp_new through a per-type C slot table
252ae8a
The tp_new slot holds a Rust fn pointer, which a C `newfunc` cannot be.
Put the C function in a `CSlots` table owned by the heap type it belongs
to, and store `c_new_trampoline` in `new`; the trampoline reads the table
off the type it is called with, so a subclass that inherited both reaches
the same function and passes itself as `subtype`.

`PyTypeSlots` holds one 8-byte pointer to the table rather than a field
per C-provided slot, so further slots are a field in `CSlots` and a
trampoline beside this one. The pointer is inherited with `new` by
`set_new` and by `update_one_slot`, and dropped when a Python-level
`__new__` replaces the slot.

Compare what tp_new dispatches to, not the slot, in the "is not safe"
check: every C type shares one trampoline, so comparing `new` alone let
`CBase.__new__(CSub)` run CSub's tp_new.

Move the C call marshalling from capi to `types::c_slots` so the
trampoline and the METH_* call paths share it.

Assisted-by: Claude
@youknowone
Expose a Rust tp_new to C through PyType_GetSlot
f680832
PyType_GetSlot reported Py_tp_new only for a function installed from C.
A Rust slot such as int's or object's read as NULL, so an extension that
reads its base type's tp_new to call it with its own subtype had nothing
to call.

Pair each Rust tp_new with a statically instantiated extern "C"
function. `c_new_for::<S>` marshals `(subtype, args, kwds)`, calls
`S::call`, and returns the object or NULL with the exception pending.
`S` is `ViaConstructor<T>` for a `#[pyclass(with(Constructor))]` payload
and `PythonNew` for `new_wrapper`; the target is fixed by the type
parameter, so a subtype passed in runs the base's body.

The derive macro emits a `HasStaticCSlots::TABLE` const per Constructor
class holding the (Rust fn, C fn) pair, and stores a reference to it in
the new `PyTypeSlots::static_c_slots` field on the defining type only.
`PyType::c_tp_new` resolves a C-installed slot from `CSlots`, and a Rust
slot by matching the slot's address against the tables along the MRO,
then against the `new_wrapper` pair.

An empty heap subclass of object now reports object's tp_new rather
than NULL; the round-trip test is updated for that.

Assisted-by: Claude
Assisted-by: Grok
@youknowone
capi: test an extension tp_new that calls its base's tp_new with subtype
b124139
Reproduce what pyo3's PyNativeTypeInitializer does: a heap type whose C
tp_new fetches the base type's tp_new through PyType_GetSlot and calls
it with the subtype it was given. Cover object and int as bases, a
Python-level subclass of the extension type, the slot each type reports,
and a ValueError raised by the base crossing both trampolines.

Replace a map/unwrap_or_else pair in a test helper that clippy flags.

Assisted-by: Claude
Assisted-by: Grok
@youknowone
address review: move type slots into object/pytype.rs
13f4404
Assisted-by: Grok:4.6
@youknowone
c_slots: hand a tp_new error to C through the raised exception slot
c6aa446
pyresult_to_ret_ptr stored the error with set_exception, which is the
exc_info slot PUSH_EXC_INFO and POP_EXCEPT maintain. The C side reads the
pending error with raised_exception and take_raised_exception, so a Rust
tp_new reached through c_new_for returned NULL with nothing raised.
Use set_raised_exception, and the same in the test helper that reports a
missing base tp_new.

Assisted-by: Claude
@youknowone
capi: install Py_tp_new from a type spec
5e0c80e
PyType_FromSlots left the Py_tp_new entry as a TODO. Hand it to
set_tp_new once the type exists, so the type instantiates through the
C function, gets the checked __new__ wrapper, and reports the function
from PyType_GetSlot.

A spec test covers those three paths. The pyo3 tests zero_sized_class
and subclass_with_property were ignored; they pass now that
PyType_GetSlot answers Py_tp_new for object and list, so the ignore
markers are removed. rust_class and rust_class_with_member stay ignored
because the spec asks for a non-zero basicsize.

Assisted-by: Claude
@youknowone
c_slots: refuse set_tp_new on a type that defines __new__
e5d33e4
set_tp_new now returns TypeError when the type's own dict already has
__new__, matching its documented precondition. HeapTypeExt::c_slots is
narrowed to pub(crate), and update_one_slot loads the type's C slot
table once instead of twice.

Add a capi test for the refusal and for the second installation, and a
pyo3 test that a #[new] taking arguments runs from Python code, raises
its error, and refuses a Python subclass.

Assisted-by: Claude
@youknowone
youknowone requested a review from bschoenmaeckers November 5, 2026 15:49
@youknowone
youknowone force-pushed the c-api-type-slots branch from 5c4325b to e5d33e4 Compare November 5, 2026 15:51
@youknowone
youknowone marked this pull request as ready for review November 6, 2026 03:31
@youknowone
youknowone merged commit 1dbe198 into RustPython:main Nov 6, 2026
19 of 20 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 6, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-11-06T03:31:48.512226Z
We couldn’t complete this review. Comment @codex review to try again.
e5d33e4 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@youknowone
youknowone deleted the c-api-type-slots branch November 6, 2026 03:31
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 6, 2026

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@bschoenmaeckers bschoenmaeckers Awaiting requested review from bschoenmaeckers

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@youknowone @bschoenmaeckers
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.