GeezerVison is showing github.com — Trace recording JIT · Issue #139109 · python/cpython
Trace recording JIT #139109DescriptionThis issue tracks progress of changing the JIT from one that projects traces to one that records them.
Follow-ups:
- Embed ENTER_EXECUTOR's executor in the bytecode and on trace graphs?
- Do not increase chain depth when side-exiting a branch, as that is just normal control-flow.
- Optimize through CALL_ALLOC_AND_ENTER_INIT
- Specialize: CALL_FUNCTION_EX,
__init__ of slots, SEND for yield from
- Loop peel.
Linked PRs
Activitymarkshannon commented on Sep 23, 2025 Just to flesh this out a bit:
Motivation
Currently we project traces from the initial hotspot, but there are a number of places in the projection where we would benefit from having live values as well as the statistical information gathered by the interpreter.
We can combine the information from live values with the statistical information to give us a trace that should, in theory, be better than from either pure trace recording or from projection.
Design
Confidence
As a trace gets longer, it is less likely that a future execution will remain on trace. Our estimate of that future likelihood is our "confidence" in the trace at that point. All traces start with confidence at 100%, but that confidence drops with each guard in the trace.
The trace recording interpreter
The trace recording interpreter operates exactly as the normal tier 1 interpreter, while recording the instructions executed.
While trace recording we should maintain a confidence level, but this confidence level will not be as accurate as the optimizer can provide. Since the optimizer might be more confident, and we cannot optimize beyond the end of the recorded trace, the confidence limit for recording should be lower than for the final trace.
Possible implementations
We can:
- Record the trace at the bytecode level, then lower to micro-ops as a separate pass
- Lower as we go. This is more complicated, but may be faster as it saves recording the bytecode.
I think we should start with 1. We can move to 2. later without impacting the rest of the JIT.
To avoid doubling the size of the interpreter, each recording opcode implementation should call a common recording function then jump to the normal implementation of that opcode.
However, some specialized instructions could "deopt", executing the non-specialized instruction. This would result in a misleading trace, so we might want to somehow force re-specialization in that case. It is not obvious how to do that without duplicating all specialized instructions. We could force re-specialization of all instructions, that might be slow. markshannon commented on Oct 29, 2025 @Fidget-Spinner
We can keep the switch interpreter working with the tracing JIT using the same technique as we did for sys.settrace tracing in 3.11.
- Change Tools/cases_generator/analyzer.py to change ENTER_EXECUTOR to 254, and lower the number of all the instrumented instructions by one, leaving 255 as reserved. Call opcode 255
INTERNAL_TRACING.
- In PyEval_EvalDefault add a local variable
tracing which is set to 0 if tracing is off and 255 if tracing is on.
- The
INTERNAL_TRACING instruction should jump to record_previous_inst which should dispatch using the plain opcode without tracing. It will be unused for computed gotos and tailcalling.
- Dispatch is a bit more complicated. Instead of dispatching on
opcode, dispatch on dispatch_opcode. Before the goto dispatch, the various DISPATCH macros need to set dispatch_opcode:
- Non-tracing dispatch:
dispatch_opcode = opcode
- Normal dispatch:
dispatch_opcode = opcode | tracing
- Tracing dispatch:
dispatch_opcode = 255
Removing the 3.11 scheme was worth about 1% speedup, so adding it back will slow the interpreter at least 1% as this is more complex than the 3.11 scheme. However, if we make the JIT faster, we should get that back and more. markshannon commented on Nov 11, 2025 Once #140310 is merged, there are a few more tasks to do:
- Restore switch based interpreter
- Record values, so we can get rid of the ad-hoc caches for functions, classes and code objects.
- Document how it works
- Improve efficiency and thread safety by storing the buffer on interpreter and swapping it to the thread when needed.
- Restore some sort of confidence estimate to tracing or optimization or both, to avoid overly long traces
- Restore the optimization for stack size checks and its tests
- Remove some code duplication by setting the previous instruction to
NOP when entering tracing.
Fidget-Spinner commented on Nov 11, 2025 The only test we need to restore is the stack size optimization checks. That's all the tests that are disabled for now. Fidget-Spinner commented on Nov 13, 2025 Another action item: prime number table for the backoff counters. This will allow for a trace of a "better" iteration. Note that pypy also uses a "strange" number for warmup of loops. markshannon commented on Nov 13, 2025 Fidget-Spinner commented on Dec 10, 2025 Yeah lets do follow ups separately Reacted by Savannah Ostrowski markshannon commented on Dec 16, 2025 I'd rather keep this open, otherwise it gives a false impression of being finished when there's some clean up to do.
Or, make new issues for the list above and then close this one. Fidget-Spinner commented on Dec 16, 2025
Restore the optimization for stack size checks and its tests
I think we should abandon this and just clean up the code implementing this optimisation. It's generally very finnicky and I don't think it nets us much perf wins. added a commit that references this issue on Dec 22, 2025gh-139109: Add terminator to JIT code when halting due to invalid dep… cocolato commented on Feb 2, 2026
I think we should abandon this and just clean up the code implementing this optimisation. It's generally very finnicky and I don't think it nets us much perf wins.
Hi, @Fidget-Spinner
Refer: #144388. Should we convert _CHECK_STACK_SPACE to _CHECK_STACK_SPACE_OPERAND one by one? If we need this, i'll implement it. added a commit that references this issue on Feb 2, 2026gh-139109: Replace `_CHECK_STACK_SPACE` with `_CHECK_STACK_SPACE_OPER…
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|