GeezerVison is showing github.com — Improve Interpreter Isolation · Issue #100227 · python/cpython
Improve Interpreter Isolation #100227DescriptionIn CPython, we store (persistent) runtime data in several locations. (See https://github.com/ericsnowcurrently/multi-core-python/wiki/3-CPython's-Runtime#cpythons-run-time-data.)
The state of each interpreter (PyInterpreterState) encapsulates the data which is unique to that interpreter and isolated from other interpreters. That isolation is important for the proper operation of multiple interpreters in a process, and will be critical for a per-interpreter GIL, AKA PEP 684 (assuming it gets approved). There are still a number of gaps in isolation which we will address here.
The deficiencies may be categorized by the following:
- shared runtime state (
_PyRuntimeState) conceptually better suited to each interpreter
- shared runtime state currently protected by the GIL
- shared global variables that shouldn't be shared
- shared global variables currently protected by the GIL
Linked PRs
Activityericsnowcurrently commented on Dec 14, 2022 A related analysis of _PyRuntimeState and the remaining global variables is found at https://github.com/ericsnowcurrently/multi-core-python/wiki/3-CPython's-Runtime#isolating-interpreter-state.
In summary, the following state needs attention:
- isolate to
PyInterpreterState
-
_PyRuntimeState.obmalloc (issue)
-
_PyRuntimeState.signals.default_handler (move to module state)
-
_PyRuntimeState.signals.ignore_handler (move to module state)
-
_PyRuntimeState.imports.lock
-
_PyRuntimeState.imports.find_and_load
-
_PyRuntimeState.dtoa
-
_PyRuntimeState.dict_state.global_version
-
_PyRuntimeState.dict_state.next_keys_version
-
_PyRuntimeState.func_state.next_version
-
_PyRuntimeState.types.next_version_tag (issue) (special-case for global/immortal types)
-
_PyRuntimeState.cached_objects.str_replace_inf
-
_PyRuntimeState.cached_objects.interned
- currently protected by the GIL (need granular locks for per-interpreter GIL)
-
_PyRuntimeState.exitfuncs (race in Py_AtExit())
-
_PyRuntimeState.nexitfuncs (race in Py_AtExit())
-
_PyRuntimeState.allocators (PyMem_SetAllocator() with "wrappers"; also when using mem/obj allocators)
-
_PyRuntimeState.audit_hook_head (race in PySys_AddAuditHook())
-
_PyRuntimeState.ceval.perf
- Objects/longobject.c - (long_from_non_binary_base)
log_base_BASE (lazy)
- Objects/longobject.c - (long_from_non_binary_base)
convwidth_base (lazy)
- Objects/longobject.c - (long_from_non_binary_base)
convmultmax_base (lazy)
- both (move some state and keep some state; copy)
- Objects/object.c -
_Py_RefTotal (issue)
Moves to PyInterpreterState with the GIL (not part of this issue):
_PyRuntimeState.gilstate.tstate_current
Also note:
- the GIL is currently fine as it is (part of
_PyRuntimeState); moving it to PyInterpreterState is a separate matter
- we'll handle faulthandler and tracemalloc state separately
old list
The following state needs further investigation and maybe further attention:
- maybe isolate to
PyInterpreterState
-
_PyRuntimeState.allocators.obj_arena (effectively coupled to _PyRuntime.obmalloc)
-
_PyRuntimeState.obmalloc.dump_debug_stats
- ...
-
_PyRuntimeState.signals.wakeup
-
_PyRuntimeState.signals.is_tripped
-
_PyRuntimeState.signals.unhandled_keyboard_interrupt
- ...
-
_PyRuntimeState.tracemalloc.traced_memory
-
_PyRuntimeState.tracemalloc.peak_traced_memory
-
_PyRuntimeState.tracemalloc.traces
-
_PyRuntimeState.tracemalloc.domains
- ...
-
_PyRuntimeState.parser.memo_statistics
- maybe protected by the GIL
-
_PyRuntimeState.fileutils.force_ascii
-
Modules/posixmodule.c - (os_dup2_impl) dup3_works (lazy)
-
Modules/faulthandler.c - (faulthandler_dump_traceback) reentrant
-
Objects/unicodeobject.c - bloom_linebreak (lazy)
-
Python/pylifecycle.c - (_Py_FatalErrorFormat) reentrant
-
Python/pylifecycle.c - (fatal_error) reentrant
- currently protected by the GIL but maybe isolate to
PyInterpreterState
-
_PyRuntimeState.imports.last_module_index
-
_PyRuntimeState.imports.extensions
-
_PyRuntimeState.imports.pkgcontext
- ...
-
_PyRuntimeState.tracemalloc.filenames
-
_PyRuntimeState.tracemalloc.traceback
-
_PyRuntimeState.tracemalloc.tracebacks
- maybe either
-
_PyRuntimeState.open_code_hook
-
_PyRuntimeState.open_code_userdata
- ...
-
_PyRuntimeState.faulthandler.fatal_error
-
_PyRuntimeState.faulthandler.thread
-
_PyRuntimeState.faulthandler.user_signals
-
_PyRuntimeState.faulthandler.stack
-
_PyRuntimeState.faulthandler.old_stack
- ...
-
_PyRuntimeState.tracemalloc.config
-
_PyRuntimeState.tracemalloc.allocators
-
_PyRuntimeState.tracemalloc.tables_lock
- ...
-
Parser/action_helpers.c - (_PyPegen_dummy_name) cache
- safe as-is but may be unnecessary (or need other solution)
-
_PyRuntimeState.gilstate.check_enabled
-
_PyRuntimeState.gilstate.autoInterpreterState
corona10 commented on Dec 14, 2022
Objects/object.c - _Py_RefTotal
Hmm how about introducing a new debug API: _Py_Get_RefTotal()?
And then move _Py_RefTotal into per interpreter attribute(_ref_total) from the global.
_Py_Get_RefTotal() will return the aggregated value of each interpreter _ref_total value..
Current _Py_RefTotal will only return the main interpreter value and will be deprecated.
Ahh.. But it looks like stable API, I am not sure it's proper approach.. encukou commented on Dec 14, 2022 It's OK to deprecate it. It just can't be removed :)
If PEP 684 is accepted, _Py_RefTotal should be used for old-style interpreters, and interpreters with own_gil could get their own counter. markshannon commented on Dec 14, 2022 Before proceeding with these changes, I think we need some up-front design.
(TBH I think we need more up-front design for a lot of things in CPython)
Memory allocation
We need some sort of global memory allocator. Is a per-interpreter ob_malloc, backed by a global allocator, presumably malloc the correct design? It might be, but we seem to be making an architectural decision by accident here.
An alternative design for allocation, that I would like to try, is per-interpreter size-based freelists backed by a global allocator.
This allows backwards compatibility with pluggable allocators and is fast. For more details faster-cpython/ideas#132
Tracemalloc
Tracemalloc data structures should be per-interpreter. Access to global values (like process-wide total memory traced) will need to be done through a function interface.
Efficiency
As more and more of the VM becomes per-interpreter, the performance impact of getting the interpreter with a C-API that was not designed with it in mind becomes larger.
@vstinner added a version of many C API functions that take the thread-state, but these aren't joined up, so there is a lot of PyThreadState_Get() calls scattered throughout the code.
We need to decide whether to pass the thread state or interpreter state.
IMO, we should follow the lead of HPy and pass the interpreter state.
We will need new API functions in many places, but many of those will be simple mechanical transformations.
Backwards compatibility
_Py_RefTotal is only defined if Py_REF_DEBUG is defined.
All updates to _Py_RefTotal can be made atomic (which will have terrible performance with multiple parallel interpreters) to preserve the API.
Ownership (runtime or per-interpreter) and constants
Making something const means it can be global (and shouldn't be in the runtime state, which is mutable).
A few of the values listed above are constants. E.g. log_base_BASE
Some values are not const but are constant once initialized. These can be shared without locks. These values should be grouped separately from values that are shared and mutable and will need locks.
Nothing should be in the "maybe both" category above. Reacted by Eric Snow and Donghee Na ericsnowcurrently commented on Dec 14, 2022 FYI, I have a plan for _Py_RefTotal that should preserve stable ABI compatibility without breaking interpreter isolation (and without relying on granular locks). Basically:
- deprecate
_Py_Reftotal but keep exporting the symbol (for stable ABI compatibility)
- add
_PyRuntimeState.object_state.last_legacy_reftotal
- add
_PyRuntimeState.object_state.reftotal
- update everywhere to use the
reftotal field (for simplicity, add #define _Py_RefTotal _PyRuntime.object_state.reftotal)
We would update _Py_GetRefTotal() to something like this:
Py_ssize_t
_Py_GetRefTotal(void)
{
// _Py_GetLegacyRefTotal() returns the value of the actual `_Py_RefTotal` global.
Py_ssize_t legacy = _Py_GetLegacyRefTotal();
_PyRuntimeState.object_state.reftotal += legacy - _PyRuntimeState.object_state.last_legacy_reftotal;
_PyRuntimeState.object_state.last_legacy_reftotal = legacy;
return _PyRuntimeState.object_state.reftotal;
}All this assumes that folks are not using _Py_RefTotal directly.
For a per-interpreter GIL, I see two options:
- add a granular lock around modifying
_PyRuntimeState.object_state.reftotal, which would hurt the performance of incref/decref, but only on debug builds
- switch to
PyInterpreterState.object_state.reftotal and aggregate the values from all interpreters in _Py_GetRefTotal()
ericsnowcurrently commented on Dec 14, 2022
FYI, I have a plan for _Py_RefTotal that should preserve stable ABI compatibility without breaking interpreter isolation
FWIW, the 3.10 changes to Py_INCREF() and Py_DECREF() help here because _Py_RefTotal is no longer used by stable ABI extensions (other than ones compiled against 3.9 and earlier). ericsnowcurrently commented on Dec 14, 2022
Before proceeding with these changes, I think we need some up-front design.
Yes, for any changes that don't make sense without a per-interpreter GIL, we would not merge them before PEP 684 is accepted.
All the points you have brought up are worth discussing. Note, though, that I consider the current per-interpreter GIL work to be a minimal approach to getting there, with at little disruption as possible while tackling a variety of improvements that make sense anyway, including improving the isolation of the existing interpreters. Much of what you've brought up are important points, but ones that will require much more discussion, take substantially more effort, and introduce more disruption. I don't think we should hold up the current project for a resolution of those broader matters.
That said, in the context of the above checklists, I agree that we must be very conservative in how we bake in any new design/architecture. I just consider isolated interpreters to be an old design (with a less than ideal API and an incomplete implementation) that we are carrying through. Improving the design is a separate project, which I wholly support! added a commit that references this issue on Apr 25, 2023gh-100227: Add a Granular Lock for _PyRuntime.imports.extensions.dict (… added 5 commits that reference this issue on Jun 8, 2023gh-100227: Lock Around Modification of the Global Allocators State (g…
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|