GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||
GeezerVison is showing github.com — Fix memory leaks during interpreter and thread teardown by VladimirKras · Pull Request #8966 · RustPython/RustPython Conversation
|
| Location | Observation on unpatched main |
|---|---|
PyFunction |
The separately owned builtins reference is omitted from traversal. A function/builtins dictionary cycle survives collection, including when globals and builtins alias. |
PyMemberDescriptor |
The declaring type is strongly owned without participating in GC traversal. Slotted classes survive collection; importlib's slotted KeyedRef class can retain its module graph. |
PyWeak |
The owned callback is omitted from traversal. A weakref/proxy, callback and closure cycle survives collection even when the referent remains externally alive. |
FreeList<T>::drop |
Thread teardown deliberately skips payload destructors and frees only raw object allocations. Dictionary buffers and other payload storage remain allocated across repeated thread/interpreter creation and shutdown. |
The GC snippets isolate ownership edges that can keep object graphs alive after interpreter teardown. The slice snippet isolates a reference retained by a cached payload; allocation scans separately demonstrate payload buffers surviving OS-thread exit. These focused snippets use explicit collection to reproduce the ownership defects without a full embedding harness. Each fails on unpatched main (7536730ed6afb948ffa335fc113a0fcd9af0cdbf) and passes under CPython 3.14 and the candidate patch:
extra_tests/snippets/stdlib_gc_function_builtins.pyextra_tests/snippets/stdlib_gc_member_descriptor.pyextra_tests/snippets/stdlib_gc_weakref_callback.pyextra_tests/snippets/stdlib_gc_slice_payload.pyFor example, run cargo run -- extra_tests/snippets/stdlib_gc_slice_payload.py. Deleting a slice should release its stop argument; the weakref assertion demonstrates that the cached payload retains it instead.
The GC changes report the omitted strong-reference edges. Weakref callbacks are visited under the existing stripe lock while referents remain weak.
For freelists, the patch destroys payloads before publishing cached entries, while thread-local deallocation state is available. Reuse writes a fresh payload, and thread teardown releases only the remaining header/allocation using its complete layout. Custom freelists such as the MemoryError pool preserve their initialized-payload contract.
This is intended to respect the reason for skipping destructors during TLS teardown. It also changes when payload destruction occurs and adds a payload-state contract to freelist reuse. Maintainers may prefer a different ownership or cleanup design; the reproducers and measurements should be useful independently of that choice.
Across repeated interpreter creation/shutdown cycles, RSS sampled after the interpreter was destroyed and its thread joined originally grew about 1.72 MiB per cycle. With all candidate fixes, a 1,000-cycle run has a tail RSS slope of about 0.0044 MiB/cycle. RSS includes reachable allocations and allocator retention, so this is not a claim of zero memory growth.
Before the freelist change, macOS leaks --atExit --noContent reported 372,640 leaked bytes after ten bare interpreter creation/shutdown cycles and 410,144 bytes after ten creation/shutdown cycles in an embedding harness. With the complete patch, scans report zero leaked blocks/bytes across 100 bare interpreter creation/shutdown cycles, 100 JSON-workload interpreter creation/shutdown cycles in the embedding harness, and additional object and native-call workloads. These are workload-specific results using an optimized RustPython dependency in a dev-profile embedding binary.
test_gc and test_weakref: 194 tests run, 28 skipped; both modules pass.AI assistance: Codex (GPT-6) investigated the ownership graph, wrote the candidate changes and regression tests, and ran the checks. This draft is prepared for the contributor's review; no human review or human-use verification is claimed yet.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Traverse function builtins, member descriptor declaring types, and weakref callbacks so unreachable cycles can be collected. Keep referents weak and read callbacks under their existing stripe lock. Destroy thread-local freelist payloads before caching, while TLS is still available. Reinitialize the payload on reuse and release cached headers with the complete allocation layout on thread exit. Preserve initialized payloads for custom freelists such as the MemoryError pool. Add regressions for live references, cycle collection, weakref callbacks, and releasing a deleted slice's stop argument. Assisted-by: Codex:gpt-6
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews: auto_review: drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Merging this PR will improve performance by 70.49%
Performance Changes
Tip Curious why performance improved? Comment Comparing Footnotes
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.