GeezerVison is showing github.com — pickletools optimize performance degraded with factor 3-4 in python 3.14 · Issue #140175 · python/cpython
pickletools optimize performance degraded with factor 3-4 in python 3.14 #140175DescriptionBug report
Bug description:
The pickletools.optimize code suddenly performs bad starting with python 3.14, with or without JIT.
The degradation is a factor 3 to 4.
It is easy to test.
First use this script to make a sizeable dict and pickle it and write it to file.
import pickle
x = {}
for i in range(1, 1000000):
x[i] = f"ii{i:>07}"
with open("pickled", "wb") as fh:
pickle.dump(x, fh, protocol=4)Then write a script, let's call it optpickle.py to read the pickled dict from file and optimize it and write the result to another file.
# optpickle.py
import pickletools
with open("pickled", "rb") as fh:
p = fh.read()
s = pickletools.optimize(p)
with open("pickledopt", "wb") as fh:
fh.write(s)Now execute this in three separate terminal windows, one geared to python 3.13, one to python 3.14.0 without JIT, and one to python 3.14.0 with JIT.
My results on a mac with M1 processor were
| version |
time elapsed |
cpu time |
| 3.13 |
1.67 |
1.773 |
| 3.14.0 no JIT |
6.52 |
6.664 |
| 3.14.0 JIT |
6.52 |
6.642 |
That is a dramatic drop in performance!
The filesizes are 16.9 MB for the unoptimized file and 15.9 for the optimized one.
I decided to drop the optimization in an app where I used it heavily
CPython versions tested on:
3.14
Operating systems tested on:
macOS ActivityStanFromIreland commented on Oct 15, 2025 Are your results an average, or just one off tests? tim-one commented on Oct 15, 2025 Just curious about how things might change if you added:
Another recent report (sorry, can't find it now) pointed out slowdowns when creating lots of little tuples, due to changes in the cyclic garbage collector (which isn't really needed here). optimize() does create lots of little tuples. sergey-miryanov commented on Oct 15, 2025 tim-one commented on Oct 15, 2025 FYI, on my box (Win10 Pro with the released 3.14.0 from pyhon.org), the time for your second script drops from about 12 seconds to about 4 seconds when disabling gc, and enclosing the body of your script in a loop to run both ways repeatedly (to factor out possible "warmup" distortions). tim-one commented on Oct 16, 2025 @dirkroorda, thank you for the very thorough and helpful follow-ups! You've done more than your share here 😄
pickletools.optimize() rarely saves much. What it's aiming at: pickling is a 1-pass process, writing out "opcodes" as it goes along. When an object is pickled, there's no way to know whether that object may appear again as a sub-object in the entire object graph being pickled. So, "just in case", a PUT opcode is pumped out after a single object's pickle has been produced. If the same object is encountered again, a GET opcode is emitted referencing the index of that object saved away by PUT.
It's often the case that a given PUT is never referenced again. All optimize() aims at is removing the PUT opcodes that are never used. But a PUT opcode occupies few bytes, and there can't be more of them than there are individual sub-objects pickled. Bytes produced for the payload (actual data bytes) typically account for the vast bulk of a pickle stream. optimize() can't do anything to shrink those.
So optimize() is rarely worth the bother. I wrote optimize(), so that's not just sour grapes 😉. Nevertheless, it can be useful in some niche cases. I used to be a "gc expert" too, but that's changed a lot since my days on it, and I no longer have actual insight into how it currently works. So I can't do more here than confirm that this really has almost nothing to do with optimize(), apart from that it creates object graphs of kinds that provoke 3.14's version of gc into bad performance.
I have no doubt that the current gc experts have already thought of several ways that could be repaired. sergey-miryanov commented on Oct 27, 2025 @dirkroorda Since #139951 has been fixed and merged, could you please confirm whether it works for you? Reacted by Mikhail Efimov and Tim Peters efimov-mikhail commented on Oct 27, 2025 tim-one commented on Oct 27, 2025 On current main branch, whether or not gc is enabled makes scant difference now to the speed of the OP's test case. So closing,, as fixed by #140262. efimov-mikhail commented on Oct 29, 2025
I'll have to figure out how I can get a python where this is fixed. Maybe I'll wait till Python is a tad closer to π
You can make Python (3.14 or 3.15) from the source code (see https://devguide.python.org/#quick-reference).
Or you can wait for Python 3.14.1 to be released, of course.
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|
