Bug report

Bug description:

Since gh-125631 (GH-125752, backported to 3.13 in GH-126528), _pickle.Pickler has a custom tp_getattro (Pickler_getattr) so that persistent_id can be set on instances. A side
effect is that PyObject_GetOptionalAttr() no longer takes its fast path for Pickler: that path is only used when tp_getattro is PyObject_GenericGetAttr, and it reports a missing
attribute without creating an exception object.

dump() looks up the optional reducer_override method on every call:

https://github.com/python/cpython/blob/a5a4659548e/Modules/_pickle.c#L4862-L4867

A plain Pickler has no such attribute, so every pickle.dumps() call goes through Pickler_getattr() → PyObject_GenericGetAttr(), which creates an AttributeError with a formatted
message and sets its name and obj, and then PyObject_GetOptionalAttr() clears it. In 3.12 and 3.13.0 Pickler used the generic tp_getattro, and this lookup did not create an
exception.

The cost is visible with plain getattr():

import io
import pickle
import timeit

p = pickle.Pickler(io.BytesIO())


class C:
    pass


c = C()
cases = [
    ('getattr(pickler, "reducer_override", None)', lambda: getattr(p, 'reducer_override', None)),
cases = [
    ('getattr(pickler, "reducer_override", None)', lambda: getattr(p, 'reducer_override', None)),
    ('getattr(pickler, "fast", None)', lambda: getattr(p, 'fast', None)),
    ('getattr(c, "reducer_override", None)', lambda: getattr(c, 'reducer_override', None)),
    ('pickle.dumps(None)', lambda: pickle.dumps(None)),
]
for label, f in cases:
    t = timeit.Timer(f)
    n, _ = t.autorange()
    print(f'{min(t.repeat(7, n)) / n * 1e9:8.1f} ns  {label}')

On main (a5a4659, ./configure --enable-experimental-jit=yes, clang 21, no PGO/LTO, Linux x86-64):

   333.2 ns  getattr(pickler, "reducer_override", None)
    46.3 ns  getattr(pickler, "fast", None)
    39.3 ns  getattr(c, "reducer_override", None)
   524.8 ns  pickle.dumps(None)

A missing attribute costs about 40 ns on a regular object and about 330 ns on a Pickler, which is most of the cost of pickle.dumps(None).

Pickler_getattr() only special-cases persistent_id, so dump() could use _PyObject_GenericGetAttrWithDict(self, name, NULL, 1) when Py_TYPE(self)->tp_getattro == Pickler_getattr:

+static PyObject *Pickler_getattr(PyObject *self, PyObject *name);
+
 static int
 dump(PickleState *state, PicklerObject *self, PyObject *obj)
 {
@@
     /* Cache the reducer_override method, if it exists. */
-    if (PyObject_GetOptionalAttr((PyObject *)self, &_Py_ID(reducer_override),
-                             &tmp) < 0) {
+    if (Py_TYPE(self)->tp_getattro == Pickler_getattr) {
+        tmp = _PyObject_GenericGetAttrWithDict((PyObject *)self,
+                                               &_Py_ID(reducer_override),
+                                               NULL, 1);
+        if (tmp == NULL && PyErr_Occurred()) {
+            goto error;
+        }
+    }
+    else if (PyObject_GetOptionalAttr((PyObject *)self,
+                                      &_Py_ID(reducer_override), &tmp) < 0) {
         goto error;
     }

With this change (only the _pickle extension rebuilt from main, same interpreter, best of 3 runs of timeit with 7 repeats, pinned to one core):

test_pickle, test_pickletools and test_copyreg pass with the change. Behaviour of subclasses is unchanged: a reducer_override method on a subclass, an instance attribute, a
property that raises AttributeError (suppressed) or another exception (propagated), and subclasses that define __getattr__ (they keep the old path).

The same pattern exists in Pickler.__init__() for the optional dispatch_table lookup:
https://github.com/python/cpython/blob/a5a4659548e/Modules/_pickle.c#L5143-L5148

getattr(pickler, "dispatch_table", None) takes about 390 ns, and pickle.Pickler(buf) about 550 ns. This affects code that creates Pickler objects directly, for example
multiprocessing.reduction.ForkingPickler.dumps().

This is similar to gh-157840, where hasattr() stopped creating an AttributeError for member descriptors without a value.

I found this while profiling logging.handlers.DatagramHandler, which calls pickle.dumps() for every record: about 26% of the time spent in pickle.dumps() was this lookup.

CPython versions tested on:

CPython main branch, 3.16, 3.15, 3.14, 3.13

Operating systems tested on:

Linux

Linked PRs

• gh-158126: Avoid creating AttributeError for optional Pickler attribute lookups #158148
• [3.15] gh-158126: Avoid creating AttributeError for optional Pickler attribute lookups (GH-158148) #160251
• [3.14] gh-158126: Avoid creating AttributeError for optional Pickler attribute lookups (GH-158148) #160252