GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — gh-159137: `gc`: Count GC survivors in `untrack_tuples()` instead of walking the young generation again by maurycy · Pull Request #159149 · python/cpython Conversation
|
| static void | |
| untrack_tuples(PyGC_Head *head) | |
| { | |
| PyGC_Head *gc = GC_NEXT(head); | |
| while (gc != head) { | |
| PyObject *op = FROM_GC(gc); | |
| PyGC_Head *next = GC_NEXT(gc); | |
| if (PyTuple_CheckExact(op)) { | |
| _PyTuple_MaybeUntrack(op); | |
| } | |
| gc = next; | |
| } | |
| } |
and then gc_list_size(young) walks the same list again only to update long_lived_pending or long_lived_total:
Lines 1517 to 1535 in 9028df3
| untrack_tuples(young); | |
| /* Move reachable objects to next generation. */ | |
| if (young != old) { | |
| if (generation == NUM_GENERATIONS - 2) { | |
| gcstate->long_lived_pending += gc_list_size(young); | |
| } | |
| gc_list_merge(young, old); | |
| } | |
| else { | |
| // In Python <= 3.13, we called untrack_dicts(young) here to untrack | |
| // atomic-only dicts (see issue #14775). Python 3.14 removed the lazy | |
| // dict tracking machinery entirely (GH-127010) -- dicts are always | |
| // tracked from creation and never untracked by GC. That way, we don't | |
| // have to restore MAINTAIN_TRACKING across every PyDict_SetItem call | |
| // site; the cost is slightly more work for full collections on dicts | |
| // with only atomic values. | |
| gcstate->long_lived_pending = 0; | |
| gcstate->long_lived_total = gc_list_size(young); | |
| } |
So the idea is to untrack_tuples() to return the number of objects that stay tracked, and use it instead of gc_list_size(young).
Full collections on a heap that stays alive for the whole run. macOS 27 (M4 Air), release build w/o PGO. Median of 6 processes per binary, interleaved A/B/B/A:
| heap | main (9028df3) |
9028df3 + patch |
|
|---|---|---|---|
| 2M objects, dict + list + tuple per entry | 77.0 ms | 67.4 ms | -12% |
| 2M dicts with atomic values | 66.9 ms | 59.0 ms | -12% |
| 5M objects, dict + list + tuple per entry | 201.7 ms | 169.6 ms | -16% |
| 2M tuples of ints | 11.6 ms | 11.6 ms | 0% |
Unfortunately, I don't have access to Linux baremetal at the moment.
Sphinx 8.2.3 building the CPython HTML docs with one worker and fresh doctrees (sphinx-build -b html -j 1 -E), same machine and builds, 6 runs per binary interleaved A/B/B/A. GC time is the sum of duration from gc.get_stats()
main (9028df3) |
9028df3 + patch |
||
|---|---|---|---|
| CPU time | 46.00 s | 44.37 s | -3.5% |
| GC time | 10.07 s | 8.63 s | -14% |
| GC time, full collections only | 8.66 s | 7.26 s | -16% |
| CPU time outside GC | 35.93 s | 35.74 s | -0.5% |
import gc, sys, time N = int(sys.argv[1]); kind = sys.argv[2] gc.disable() if kind == "mixed": keep = [({"a": i, "b": None}, [i, i], (i, i + 1)) for i in range(N // 3)] elif kind == "tuples": keep = [(i, i + 1, i + 2) for i in range(N)] elif kind == "dicts": keep = [{"a": i, "b": "x"} for i in range(N)] for _ in range(2): gc.collect() ts = [] for _ in range(7): t0 = time.perf_counter_ns(); gc.collect(); ts.append(time.perf_counter_ns() - t0) ts.sort(); print(ts[3] / 1e6)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
This is subset of another PR I m finishing now :) |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
See #159154 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.