Immortalizing Runtime Heap After Startup
This is an optimization on top of PR19474.
The improvement here uses the assumption that everything that is alive after the runtime startup will (most likely) be alive during the entire lifecycle of the runtime. This then immortalizes the existing heap after the runtime has finished bootstrapping and right before the loading and execution of bytecode.
Benchmark Results
Overall: 0% slower compared to the main branch
pyperformance results
2to3: Mean +- std dev: [cpython_master] 432 ms +- 15 ms -> [immortal_instances_opt2] 440 ms +- 12 ms: 1.02x slower
chaos: Mean +- std dev: [cpython_master] 126 ms +- 4 ms -> [immortal_instances_opt2] 119 ms +- 3 ms: 1.06x faster
float: Mean +- std dev: [cpython_master] 128 ms +- 4 ms -> [immortal_instances_opt2] 133 ms +- 6 ms: 1.04x slower
go: Mean +- std dev: [cpython_master] 244 ms +- 10 ms -> [immortal_instances_opt2] 232 ms +- 8 ms: 1.05x faster
hexiom: Mean +- std dev: [cpython_master] 11.5 ms +- 0.6 ms -> [immortal_instances_opt2] 11.3 ms +- 0.3 ms: 1.02x faster
json_dumps: Mean +- std dev: [cpython_master] 19.2 ms +- 0.7 ms -> [immortal_instances_opt2] 19.7 ms +- 0.8 ms: 1.02x slower
logging_format: Mean +- std dev: [cpython_master] 10.4 us +- 0.3 us -> [immortal_instances_opt2] 10.8 us +- 0.3 us: 1.04x slower
logging_silent: Mean +- std dev: [cpython_master] 201 ns +- 8 ns -> [immortal_instances_opt2] 205 ns +- 7 ns: 1.02x slower
logging_simple: Mean +- std dev: [cpython_master] 9.77 us +- 0.32 us -> [immortal_instances_opt2] 9.46 us +- 0.41 us: 1.03x faster
meteor_contest: Mean +- std dev: [cpython_master] 164 ms +- 5 ms -> [immortal_instances_opt2] 161 ms +- 5 ms: 1.02x faster
nbody: Mean +- std dev: [cpython_master] 163 ms +- 6 ms -> [immortal_instances_opt2] 159 ms +- 6 ms: 1.03x faster
nqueens: Mean +- std dev: [cpython_master] 159 ms +- 5 ms -> [immortal_instances_opt2] 152 ms +- 6 ms: 1.05x faster
pathlib: Mean +- std dev: [cpython_master] 28.5 ms +- 0.7 ms -> [immortal_instances_opt2] 28.2 ms +- 0.9 ms: 1.01x faster
pickle: Mean +- std dev: [cpython_master] 16.0 us +- 0.5 us -> [immortal_instances_opt2] 15.7 us +- 0.8 us: 1.02x faster
pickle_dict: Mean +- std dev: [cpython_master] 37.3 us +- 0.6 us -> [immortal_instances_opt2] 35.2 us +- 1.4 us: 1.06x faster
pickle_list: Mean +- std dev: [cpython_master] 5.77 us +- 0.24 us -> [immortal_instances_opt2] 5.53 us +- 0.22 us: 1.04x faster
pidigits: Mean +- std dev: [cpython_master] 284 ms +- 15 ms -> [immortal_instances_opt2] 276 ms +- 7 ms: 1.03x faster
python_startup: Mean +- std dev: [cpython_master] 12.6 ms +- 0.4 ms -> [immortal_instances_opt2] 11.9 ms +- 0.5 ms: 1.05x faster
python_startup_no_site: Mean +- std dev: [cpython_master] 8.89 ms +- 0.39 ms -> [immortal_instances_opt2] 8.21 ms +- 0.34 ms: 1.08x faster
raytrace: Mean +- std dev: [cpython_master] 529 ms +- 16 ms -> [immortal_instances_opt2] 542 ms +- 15 ms: 1.03x slower
regex_compile: Mean +- std dev: [cpython_master] 233 ms +- 6 ms -> [immortal_instances_opt2] 239 ms +- 6 ms: 1.03x slower
regex_dna: Mean +- std dev: [cpython_master] 239 ms +- 6 ms -> [immortal_instances_opt2] 257 ms +- 6 ms: 1.08x slower
regex_effbot: Mean +- std dev: [cpython_master] 4.53 ms +- 0.12 ms -> [immortal_instances_opt2] 4.69 ms +- 0.17 ms: 1.04x slower
regex_v8: Mean +- std dev: [cpython_master] 33.2 ms +- 0.8 ms -> [immortal_instances_opt2] 34.4 ms +- 1.1 ms: 1.04x slower
richards: Mean +- std dev: [cpython_master] 82.8 ms +- 3.7 ms -> [immortal_instances_opt2] 85.1 ms +- 3.5 ms: 1.03x slower
scimark_fft: Mean +- std dev: [cpython_master] 571 ms +- 12 ms -> [immortal_instances_opt2] 614 ms +- 19 ms: 1.08x slower
scimark_lu: Mean +- std dev: [cpython_master] 195 ms +- 6 ms -> [immortal_instances_opt2] 207 ms +- 6 ms: 1.06x slower
scimark_monte_carlo: Mean +- std dev: [cpython_master] 116 ms +- 5 ms -> [immortal_instances_opt2] 119 ms +- 5 ms: 1.02x slower
scimark_sor: Mean +- std dev: [cpython_master] 211 ms +- 6 ms -> [immortal_instances_opt2] 222 ms +- 8 ms: 1.05x slower
scimark_sparse_mat_mult: Mean +- std dev: [cpython_master] 8.28 ms +- 0.40 ms -> [immortal_instances_opt2] 8.74 ms +- 0.36 ms: 1.06x slower
sympy_expand: Mean +- std dev: [cpython_master] 878 ms +- 34 ms -> [immortal_instances_opt2] 853 ms +- 16 ms: 1.03x faster
sympy_integrate: Mean +- std dev: [cpython_master] 35.2 ms +- 1.0 ms -> [immortal_instances_opt2] 35.7 ms +- 1.2 ms: 1.01x slower
sympy_sum: Mean +- std dev: [cpython_master] 291 ms +- 13 ms -> [immortal_instances_opt2] 296 ms +- 6 ms: 1.02x slower
sympy_str: Mean +- std dev: [cpython_master] 514 ms +- 11 ms -> [immortal_instances_opt2] 523 ms +- 14 ms: 1.02x slower
unpack_sequence: Mean +- std dev: [cpython_master] 77.9 ns +- 2.3 ns -> [immortal_instances_opt2] 70.0 ns +- 2.0 ns: 1.11x faster
unpickle: Mean +- std dev: [cpython_master] 21.5 us +- 0.7 us -> [immortal_instances_opt2] 22.1 us +- 1.1 us: 1.03x slower
unpickle_pure_python: Mean +- std dev: [cpython_master] 463 us +- 17 us -> [immortal_instances_opt2] 447 us +- 7 us: 1.04x faster
xml_etree_process: Mean +- std dev: [cpython_master] 102 ms +- 2 ms -> [immortal_instances_opt2] 98.2 ms +- 2.4 ms: 1.04x faster
Benchmark hidden because not significant (13): deltablue, django_template, fannkuch, html5lib, json_loads, pickle_pure_python, pyflate, spectral_norm, telco, unpickle_list, xml_etree_parse, xml_etree_iterparse, xml_etree_generate
Geometric mean: 1.00x slower
Implementation Details
To achieve this, in pymain_main we call the newly introduced internal API: _PyGC_ImmortalizeHeap. This uses the internal gc.freeze API to take all the existing objects in the heap and move them to the permanent generation. Then, it iterates the permanent generation marking all the containers as immortal, including the first traversal layer.
Pseudo-Topological Ordered Finalization
Due to this change, there will now be slight differences in the module finalization order due to the immortality of various instances. More specifically, at runtime shutdown, the modules that exist during __del__ will now change. Note that the behavior of __del__ during runtime shutdown is an implementation detail and clearly labeled undefined behavior by the official documentation.
That being said, to reduce the variance introduced here by the runtime heap immortalization, we introduce a pseudo-topological order finalization of modules and its globals. Unfortunately, at runtime shutdown, we can’t fully guarantee the finalization dependency graph. Therefore, it does a best attempt at doing a topological finalization by executing the following heuristic:
- Start clearing from the newest to oldest User defined modules
- Finalize globals starting only with '_', excluding modules
- Finalize globals, excluding modules and
__builtins__
- Finalize the remaining objects excluding
__builtins__
- Start clearing from the newest to oldest Standard Library modules
- Finalize globals starting only with '_', excluding modules
- Finalize globals, excluding modules and
__builtins__
- Finalize the remaining objects excluding
__builtins__
This ordering guarantees that for any user defined module, the entire standard library will still be available at runtime shutdown for any custom __del__ function. Not only that but it also keep the current flexibility of using underscores to have more control over the destruction of these objects during the shutdown of the runtime. The current heuristics can still be extended to provide a more well defined destruction ordered (i.e perhaps splitting stdlib between C modules and Lib modules).
Test Output Checks
A large portion of the tests that check __del__ behavior rely on using the subprocess module to execute and parse the result of a given python program after it exists. This change caused the behavior of subprocess stdout PIPE to change due. That is, if we don’t PIPE stdout, the output is correctly printed to the terminal. However, when using stdout PIPE the output is suppressed but not picked up by the subprocess module. To get around this, rather than using stdout to test the output of __del__, we use stderr which still behaves correctly even during the topological ordered destruction.
Permanent Generation Finalization
Currently the permanent generation is not cleaned up at runtime shutdown. This preserves that behavior but it could be extended to do a best effort destruction of these instances.
https://bugs.python.org/issue40255