Feature or enhancement

Proposal:

I was reading the code around CALL_LIST_APPEND and noticed a possible optimization for list subclasses that inherit list.append without overriding it.

I am wondering whether inherited list.append calls could be supported deliberately, with matching specialization time and runtime checks.

For example:

class MyList(list):
    pass

def append_one(values):
    values.append(1)

For an exact list, this call can specialize to CALL_LIST_APPEND. For MyList, it currently specializes to CALL_METHOD_DESCRIPTOR_O, even though attribute lookup still resolves to the built-in list.append.

The specialization is made in specialize_method_descriptor().

It checks that the receiver is an exact list:

if ((PyObject *)descr == list_append && oparg == 1) {
    assert(self_or_null != NULL);
    if (PyList_CheckExact(self_or_null)) {
        specialize(instr, CALL_LIST_APPEND);
        return 0;
    }
}

and CALL_LIST_APPEND macro is defined as follows:

macro(CALL_LIST_APPEND) =
    unused/1 +
    unused/2 +
    _GUARD_CALLABLE_LIST_APPEND +
    _GUARD_NOS_NOT_NULL +
    _GUARD_NOS_LIST +
    _CALL_LIST_APPEND +
    POP_TOP +
    POP_TOP;

As far as I understand, this should exclude subclasses that shadow append.

But, _CALL_LIST_APPEND ultimately uses _PyList_AppendTakeRef, which accepts list subclasses through PyList_Check, just like the generic list.append path. So the underlying append operation already appears to support subclasses.

The remaining problem is ensuring that the specialization and runtime guards admit them safely.

Possible approach

The safer structure seems to be leaving the existing instruction unchanged and adding a separate specialization for list subclasses such as CALL_LIST_APPEND_SUBTYPE

The specializer could select the existing instruction for exact lists and the new instruction for subclasses:

if (PyList_CheckExact(self_or_null)) {
    specialize(instr, CALL_LIST_APPEND);
}
else if (PyList_Check(self_or_null)) {
    specialize(instr, CALL_LIST_APPEND_SUBTYPE);
}

Both instructions could then share _CALL_LIST_APPEND while keeping their receiver assumptions separate:

macro(CALL_LIST_APPEND) =
    unused/1 +
    unused/2 +
    _GUARD_CALLABLE_LIST_APPEND +
    _GUARD_NOS_NOT_NULL +
    _GUARD_NOS_LIST +
    _CALL_LIST_APPEND +
    POP_TOP +
    POP_TOP;

macro(CALL_LIST_APPEND_SUBTYPE) =
    unused/1 +
    unused/2 +
    _GUARD_CALLABLE_LIST_APPEND +
    _GUARD_NOS_NOT_NULL +
    _GUARD_NOS_LIST_SUBTYPE +
    _CALL_LIST_APPEND +
    POP_TOP +
    POP_TOP;

op(_GUARD_NOS_LIST_SUBTYPE, (self -- self)) {
    EXIT_IF(!PyList_Check(self));
}

This keeps the meaning of CALL_LIST_APPEND unchanged, while isolating the new subtype behavior.

Alternative approach

A smaller alternative would be to let the existing CALL_LIST_APPEND instruction accept list subclasses:

- if (PyList_CheckExact(self_or_null)) {
+ if (PyList_Check(self_or_null)) {
      specialize(instr, CALL_LIST_APPEND);
  }

and replace its exact list guard:

 macro(CALL_LIST_APPEND) =
     unused/1 +
     unused/2 +
     _GUARD_CALLABLE_LIST_APPEND +
     _GUARD_NOS_NOT_NULL +
-    _GUARD_NOS_LIST +
+    _GUARD_NOS_LIST_SUBTYPE +
     _CALL_LIST_APPEND +
     POP_TOP +
     POP_TOP;

It is a smaller code change, but it broadens the meaning of an existing instruction. I have not verified all assumptions outside, so I am not sure that changing the existing instruction is the right approach.

Is there any semantic or implementation reason why a list subclass inheriting the built-in list.append should not reuse the _CALL_LIST_APPEND fast path?

If the optimization is worthwhile, would it be preferable to extend the existing instruction, or to keep the new behavior isolated in a separate specialization?

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

#141367

Linked PRs

• gh-155456 Specialize list.append for list subclasses #155474