GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-139109: A new tracing JIT compiler frontend for CPython by Fidget-Spinner · Pull Request #140310 · python/cpython

Conversation

@Fidget-Spinner

@Fidget-Spinner Fidget-Spinner commented Oct 18, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This PR changes the current JIT model from trace projection to trace recording. Benchmarking: better pyperformance (about 1.7% overall) geomean versus current https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251108-3.15.0a1%2B-7e2bc1d-JIT/bm-20251108-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-7e2bc1d-vs-base.svg, 100% faster Richards on the most improved benchmark versus the current JIT. Slowdown of about 10-15% on the worst benchmark versus the current JIT. Note: the fastest version isn't the one merged, as it relies on fixing bugs in the specializing interpreter, which is left to another PR. The speedup in the merged version is about 1.1%. https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251112-3.15.0a1%2B-f8a764a-JIT/bm-20251112-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-f8a764a-vs-base.svg

Stats: 50% more uops executed, 30% more traces entered the last time we ran them. It also suggests our trace lengths for a real trace recording JIT are too short, as a lot of trace too long aborts https://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20251023-3.15.0a1%2B-eb73378-CLANG%2CJIT/bm-20251023-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-eb73378-pystats-vs-base.md .

This new JIT frontend is already able to record/execute significantly more instructions than the previous JIT frontend. In this PR, we are now able to record through custom dunders, simple object creation, generators, etc. None of these were done by the old JIT frontend. Some custom dunders uops were discovered to be broken as part of this work gh-140277

Some tests for the optimizers are disabled for now because their invariants have changed. The main one is the stack space check, as it's no longer valid to deal with underflow.

The full test suite passes on my system.

Not close enough to a holiday, so no poem this time!

Pros:

  • Ignoring the generated tracer code as it's automatically created, this is only additional 1k lines of code. The maintenance burden is handled by the DSL and code generator.
  • optimizer.c is now significantly simpler, as we don't have to do strange things to recover the bytecode from a trace.
  • The new JIT frontend is able to handle a lot more control-flow than the old one.
  • Tracing is very low overhead. We use the tail calling interpreter/computed goto interpreter to switch between tracing mode and non-tracing mode. I call this mechanism dual dispatch, as we have two dispatch tables dispatching to each other. Specialization is still enabled while tracing.
  • Better handling of polymorphism. We leverage the specializing interpreter for this.

Cons:

  • (For now) requires tail calling interpreter or computed gotos. This means no Windows JIT for now :(. Not to fret, tail calling is coming soon to Windows though gh-139922: Tail calling for MSVC (VS 2026) #139962

Design:

  • After each instruction, the record_previous_inst function/label is executed. This does as the name suggests.
  • The tracing interpreter lowers bytecode to uops directly so that it can obtain "fresh" values at the point of lowering.
  • The tracing version behaves nearly identical to the normal interpreter, in fact it even has specialization! This allows it to run without much of a slowdown when tracing. The actual cost of tracing is only a function call and writes to memory.
  • The tracing interpreter uses the specializing interpreter's deopt to naturally form the side exit chains. This allows it to side exit chain effectively, without repeating much code. We force a re-specializing when tracing a deopt.
  • The tracing interpreter can even handle goto errors/exceptions, but I chose to disable them for now as it's not tested.
  • Because we do not share interpreter dispatch, there is should be no significant slowdown to the original specializing interpreter on tailcall and computed got with JIT disabled. With JIT enabled, there might be a slowdown in the form of the JIT trying to trace.
  • Things that could have dynamic instruction pointer effects are guarded on. The guard deopts to a new instruction --- _DYNAMIC_EXIT.

Future ideas:

  • _DYNAMIC_EXIT will cause a re-trace when hot enough starting from the target instruction.
  • As such, traces "grow" like a web/graph rather than a tree. It starts from a loop, then grows outwards naturally consuming more code around it as the hotspot gets hotter.
  • Issue: Trace recording JIT #139109

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

AA-Turner, Zheaoli, corona10, brettcannon, nmstoker, and tekknolagi reacted with hooray emoji stonebig reacted with heart emoji emmatyping, dolfinus, Sacul0457, chris-eibl, Wulian233, savannahostrowski, efimov-mikhail, rafaeljcd, Zheaoli, sergey-miryanov, and 11 more reacted with rocket emoji
All reactions
  • 🎉 6 reactions
  • ❤️ 1 reaction
  • 🚀 21 reactions
Fidget-Spinner added 30 commits September 18, 2025 17:00
@Fidget-Spinner
Basic trace recording
795ef49
@Fidget-Spinner
WIP generators
40bf6c1
@Fidget-Spinner
refactor to translate on the go
13188a9
@Fidget-Spinner
working python startup
e63de39
@Fidget-Spinner
fix a bug with specializzation
fba9d2d
@Fidget-Spinner
Fully working bm_generators
7192671
@Fidget-Spinner
fix jit build
07542dd
@Fidget-Spinner
Fix exception tracing
021fc44
@Fidget-Spinner
Fix jump tracing
f886c43
@Fidget-Spinner
Fix handling of ENTER_EXECUTOR
3066963
@Fidget-Spinner
Fix ENTER_EXECUTOR bug
20b283b
@Fidget-Spinner
Fix over-tracing bug
36554a5
@Fidget-Spinner
fix JIT + debug builds
92bba64
@Fidget-Spinner
Fix double-initialization
2e3ddc1
@Fidget-Spinner
fix exception bug
aada168
@Fidget-Spinner
Fix dispatch_inlined
7b5c655
@Fidget-Spinner
Fix handling of EXTENDED_ARG
3e9f782
@Fidget-Spinner
Fix chain depth bug
9a66605
@Fidget-Spinner
remove printf
108ab7f
@Fidget-Spinner
fix problem with jumping labels
fac8c74
@Fidget-Spinner
Point to previous executor when side-exiting
fd3bb48
@Fidget-Spinner
Fix progress needed and warmup
96b7bb2
@Fidget-Spinner
Fix unsupported opcode bug, turn off optimizer again
396818b
@Fidget-Spinner
fix branch tracing
57f417e
@Fidget-Spinner
fix branch prediction for real
2c603cc
@Fidget-Spinner
fix non-sstandard C
02f1fb4
@Fidget-Spinner
Track from JUMP_BACKWARD rather than FOR_ITER
dc414a3
@Fidget-Spinner
Fix bug where code/func get freed halfway
0ffc2dd
@Fidget-Spinner
add back replaced, move jit tracing env var to
2032b9c
@Fidget-Spinner
Handle recursive tracing and CALL_ALLOC_AND_ENTER_INIT
299a068
@Fidget-Spinner
fix JIT builds
f8a764a
@Fidget-Spinner

Fidget-Spinner commented Nov 12, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

The results after trusting the interpreter with specialization is slower. https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251112-3.15.0a1%2B-f8a764a-JIT/bm-20251112-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-f8a764a-vs-base.svg

This likely indicates bugs in the specializing interpreter. For example, deltablue is suffering from the CALL_LIST_APPEND bug.

I say we just merge the "slower" version though. We should fix the bugs in the interpreter separately.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon

markshannon commented Nov 12, 2025

Copy link
Copy Markdown
Member

I'm not worried about the deltablue slowdown. It subclasses a list for no reason, which I don't think we care about.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon

markshannon commented Nov 12, 2025

Copy link
Copy Markdown
Member

Is the iOS failure anything to do with this PR or is it failing elsewhere?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

markshannon
markshannon requested changes Nov 12, 2025
View reviewed changes
Comment thread Python/bytecodes.c
GOTO_TIER_ONE(target);
}

tier2 op(_GUARD_IP__PUSH_FRAME, (ip/4 --)) {

@markshannon markshannon Nov 12, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the plan was to generate these exits, otherwise they get out of sync with the source instruction.

What is OFFSET_OF(x)? The offset of what?

If generating the whole uop is too cumbersome because of uop ids and such, then generate the body, leaving the declaration looking like this:

tier2 op(_GUARD_IP__PUSH_FRAME, (ip/4 --)) {
    IP_GUARD(_PUSH_FRAME);
}

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@Fidget-Spinner Fidget-Spinner Nov 12, 2025

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is automatically generating it from the offset of the IP.

I've changed the name to IP_OFFSET_OF to make it clearer.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Python/bytecodes.c
DISPATCH();
}

label(record_previous_inst) {

@markshannon markshannon Nov 12, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOTE: To support the switch/case interpreter this will need to be an instruction.
No need to do so in this PR.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Python/ceval.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Python/ceval.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Fidget-Spinner added 2 commits November 12, 2025 18:42
@Fidget-Spinner
Address review
b72ab8b
@Fidget-Spinner
Merge remote-tracking branch 'upstream/main' into tracing_jit
8993e6c
@mhsmith

mhsmith commented Nov 12, 2025

Copy link
Copy Markdown
Member

Is the iOS failure anything to do with this PR or is it failing elsewhere?

It's been failing intermittently for a few weeks: #141358 (comment).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

markshannon
markshannon requested changes Nov 13, 2025
View reviewed changes

@markshannon markshannon left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final set of requests.

You seem to be using code_curr_size in various places for checking things other than just the uop count, such as whether tracing is active.
Can you add comments and/or names for the constants for all comparisons involving code_curr_size?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Python/optimizer_analysis.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Python/optimizer.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Python/optimizer.c
}
if (jump_target != current_jump_target || current_exit_op != exit_op) {
make_exit(&buffer[next_spare], exit_op, jump_target);
bool is_control_flow = (opcode == _GUARD_IS_FALSE_POP || opcode == _GUARD_IS_TRUE_POP || is_for_iter_test[opcode]);

@markshannon markshannon Nov 13, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the only use of is_for_iter_test, in which case replace it with is_control_flow and simplify this test.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@Fidget-Spinner Fidget-Spinner Nov 13, 2025

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it's not. We also use it to decide to offset the target to pointer after the END_FOR too.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Python/optimizer.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Python/optimizer.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner
add stats, document
fa45ae8
markshannon
markshannon reviewed Nov 13, 2025
View reviewed changes
Comment thread Python/optimizer.c Outdated
/* Special case the first instruction,
* so that we can guarantee forward progress */
if (progress_needed && _tstate->jit_tracer_state.prev_state.code_curr_size <= 3) {
if (progress_needed && _tstate->jit_tracer_state.prev_state.code_curr_size <= CODE_SIZE_NO_PROGRESS) {

@markshannon markshannon Nov 13, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is different. You've changed the rhs from 3 to 5.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@Fidget-Spinner Fidget-Spinner Nov 13, 2025

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it should be CODE_SIZE_NO_PROGRESS. I realised the problem is that it's < not <= though!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
markshannon
markshannon reviewed Nov 13, 2025
View reviewed changes
Comment thread Python/optimizer.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Fidget-Spinner added 2 commits November 13, 2025 16:10
@Fidget-Spinner
Fix off-by-one
308ccc0
@Fidget-Spinner
address review
33cf287
@markshannon
markshannon self-requested a review November 13, 2025 16:30
markshannon
markshannon approved these changes Nov 13, 2025
View reviewed changes

@markshannon markshannon left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a big step forward for the JIT.
@Fidget-Spinner thanks again for doing this.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

sergey-miryanov, hugovk, tekknolagi, stonebig, Sacul0457, and AIdjis reacted with hooray emoji Fidget-Spinner, efimov-mikhail, sergey-miryanov, ZeroIntensity, AlexWaygood, chris-eibl, Sacul0457, AIdjis, and Banicoder77 reacted with heart emoji
All reactions
  • 🎉 6 reactions
  • ❤️ 9 reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting changes labels Nov 13, 2025
@Fidget-Spinner
Fix a bug in dependencies detected by CI
3163d9b
@Fidget-Spinner

Fidget-Spinner commented Nov 13, 2025

Copy link
Copy Markdown
Member Author

CI caught a bug in the dependency tracking. I fixed it in the latest commit.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner
add code object only after we add to trace
658fd98
@Fidget-Spinner
Fidget-Spinner merged commit 4fa80ce into python:main Nov 13, 2025
72 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting merge label Nov 13, 2025
@Fidget-Spinner
Fidget-Spinner deleted the tracing_jit branch November 13, 2025 18:08
@efimov-mikhail

efimov-mikhail commented Nov 13, 2025

Copy link
Copy Markdown
Member

This is a big step forward for the JIT. @Fidget-Spinner thanks again for doing this.

Congratulations, @Fidget-Spinner !

Fidget-Spinner, ZeroIntensity, rafalp, and Sacul0457 reacted with heart emoji savannahostrowski, Fidget-Spinner, ZeroIntensity, chris-eibl, rafalp, and Sacul0457 reacted with rocket emoji
All reactions
  • ❤️ 4 reactions
  • 🚀 6 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request Nov 13, 2025
@CuriousLearner
Merge branch 'main' into fix-issue21914
b9f473c
* main: (463 commits)
  pythongh-140601: Add ResourceWarning to iterparse when not closed (pythonGH-140603)
  pythongh-137969: Fix double evaluation of `ForwardRef`s which rely on globals (python#140974)
  pythongh-139109: A new tracing JIT compiler frontend for CPython (pythonGH-140310)
  pythongh-141004: Document `PyErr_RangedSyntaxLocationObject` (python#141521)
  pythongh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (pythonGH-140884)
  pythongh-139653: Add PyUnstable_ThreadState_SetStackProtection() (python#139668)
  pythongh-141004: Document `PyCode_Optimize` (pythonGH-141378)
  pythongh-141004: Document C APIs for dictionary keys, values, and items (pythonGH-141009)
  pythongh-137959: Fix `TIER1_TO_TIER2` macro name in JIT InternalDocs (pythonGH-141496)
  pythongh-139871: Add `bytearray.take_bytes([n])` to efficiently extract `bytes` (pythonGH-140128)
  pythongh-140601: Refactor ElementTree.iterparse() tests (pythonGH-141499)
  pythongh-135801: Add the module parameter to compile() etc (pythonGH-139652)
  pythongh-140260: fix data race in `_struct` module initialization with subinterpreters (python#140909)
  pythongh-137109: refactor warning about threads when forking (python#141438)
  pythongh-141004: Document `PyRun_InteractiveOneObject` (pythonGH-141405)
  pythongh-124111: Fix TCL 9 thread detection (pythonGH-128103)
  pythongh-141442: Add escaping to iOS testbed arguments (python#141443)
  pythongh-140936: Fix JIT assertion crash at finalization if some generator is alive (pythonGH-140969)
  Add details about JIT build infrastructure and updating dependencies to `Tools/jit` (python#141167)
  pythongh-141412: Use reliable target URL for urllib example (pythonGH-141428)
  ...
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request Nov 14, 2025
@CuriousLearner
Merge branch 'main' of github.com:python/cpython into fix-Issue18387
c46ecc5
* 'main' of github.com:python/cpython: (464 commits)
  pythongh-140601: Add ResourceWarning to iterparse when not closed (pythonGH-140603)
  pythongh-137969: Fix double evaluation of `ForwardRef`s which rely on globals (python#140974)
  pythongh-139109: A new tracing JIT compiler frontend for CPython (pythonGH-140310)
  pythongh-141004: Document `PyErr_RangedSyntaxLocationObject` (python#141521)
  pythongh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (pythonGH-140884)
  pythongh-139653: Add PyUnstable_ThreadState_SetStackProtection() (python#139668)
  pythongh-141004: Document `PyCode_Optimize` (pythonGH-141378)
  pythongh-141004: Document C APIs for dictionary keys, values, and items (pythonGH-141009)
  pythongh-137959: Fix `TIER1_TO_TIER2` macro name in JIT InternalDocs (pythonGH-141496)
  pythongh-139871: Add `bytearray.take_bytes([n])` to efficiently extract `bytes` (pythonGH-140128)
  pythongh-140601: Refactor ElementTree.iterparse() tests (pythonGH-141499)
  pythongh-135801: Add the module parameter to compile() etc (pythonGH-139652)
  pythongh-140260: fix data race in `_struct` module initialization with subinterpreters (python#140909)
  pythongh-137109: refactor warning about threads when forking (python#141438)
  pythongh-141004: Document `PyRun_InteractiveOneObject` (pythonGH-141405)
  pythongh-124111: Fix TCL 9 thread detection (pythonGH-128103)
  pythongh-141442: Add escaping to iOS testbed arguments (python#141443)
  pythongh-140936: Fix JIT assertion crash at finalization if some generator is alive (pythonGH-140969)
  Add details about JIT build infrastructure and updating dependencies to `Tools/jit` (python#141167)
  pythongh-141412: Use reliable target URL for urllib example (pythonGH-141428)
  ...
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request Nov 14, 2025
@CuriousLearner
Merge branch 'main' of github.com:python/cpython into fix-issue24255
c240289
* 'main' of github.com:python/cpython: (464 commits)
  pythongh-140601: Add ResourceWarning to iterparse when not closed (pythonGH-140603)
  pythongh-137969: Fix double evaluation of `ForwardRef`s which rely on globals (python#140974)
  pythongh-139109: A new tracing JIT compiler frontend for CPython (pythonGH-140310)
  pythongh-141004: Document `PyErr_RangedSyntaxLocationObject` (python#141521)
  pythongh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (pythonGH-140884)
  pythongh-139653: Add PyUnstable_ThreadState_SetStackProtection() (python#139668)
  pythongh-141004: Document `PyCode_Optimize` (pythonGH-141378)
  pythongh-141004: Document C APIs for dictionary keys, values, and items (pythonGH-141009)
  pythongh-137959: Fix `TIER1_TO_TIER2` macro name in JIT InternalDocs (pythonGH-141496)
  pythongh-139871: Add `bytearray.take_bytes([n])` to efficiently extract `bytes` (pythonGH-140128)
  pythongh-140601: Refactor ElementTree.iterparse() tests (pythonGH-141499)
  pythongh-135801: Add the module parameter to compile() etc (pythonGH-139652)
  pythongh-140260: fix data race in `_struct` module initialization with subinterpreters (python#140909)
  pythongh-137109: refactor warning about threads when forking (python#141438)
  pythongh-141004: Document `PyRun_InteractiveOneObject` (pythonGH-141405)
  pythongh-124111: Fix TCL 9 thread detection (pythonGH-128103)
  pythongh-141442: Add escaping to iOS testbed arguments (python#141443)
  pythongh-140936: Fix JIT assertion crash at finalization if some generator is alive (pythonGH-140969)
  Add details about JIT build infrastructure and updating dependencies to `Tools/jit` (python#141167)
  pythongh-141412: Use reliable target URL for urllib example (pythonGH-141428)
  ...
StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025
@Fidget-Spinner @StanFromIreland
pythongh-139109: A new tracing JIT compiler frontend for CPython (pyt…
6ddc353
…honGH-140310)

This PR changes the current JIT model from trace projection to trace recording. Benchmarking: better pyperformance (about 1.7% overall) geomean versus current https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251108-3.15.0a1%2B-7e2bc1d-JIT/bm-20251108-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-7e2bc1d-vs-base.svg, 100% faster Richards on the most improved benchmark versus the current JIT. Slowdown of about 10-15% on the worst benchmark versus the current JIT. **Note: the fastest version isn't the one merged, as it relies on fixing bugs in the specializing interpreter, which is left to another PR**. The speedup in the merged version is about 1.1%. https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251112-3.15.0a1%2B-f8a764a-JIT/bm-20251112-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-f8a764a-vs-base.svg

Stats: 50% more uops executed, 30% more traces entered the last time we ran them. It also suggests our trace lengths for a real trace recording JIT are too short, as a lot of trace too long aborts https://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20251023-3.15.0a1%2B-eb73378-CLANG%2CJIT/bm-20251023-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-eb73378-pystats-vs-base.md .

This new JIT frontend is already able to record/execute significantly more instructions than the previous JIT frontend. In this PR, we are now able to record through custom dunders, simple object creation, generators, etc. None of these were done by the old JIT frontend. Some custom dunders uops were discovered to be broken as part of this work pythongh-140277

The optimizer stack space check is disabled, as it's no longer valid to deal with underflow.

Pros:
* Ignoring the generated tracer code as it's automatically created, this is only additional 1k lines of code. The maintenance burden is handled by the DSL and code generator.
* `optimizer.c` is now significantly simpler, as we don't have to do strange things to recover the bytecode from a trace.
* The new JIT frontend is able to handle a lot more control-flow than the old one.
* Tracing is very low overhead. We use the tail calling interpreter/computed goto interpreter to switch between tracing mode and non-tracing mode. I call this mechanism dual dispatch, as we have two dispatch tables dispatching to each other. Specialization is still enabled while tracing.
* Better handling of polymorphism. We leverage the specializing interpreter for this.

Cons:
* (For now) requires tail calling interpreter or computed gotos. This means no Windows JIT for now :(. Not to fret, tail calling is coming soon to Windows though python#139962

Design:
* After each instruction, the `record_previous_inst` function/label is executed. This does as the name suggests.
* The tracing interpreter lowers bytecode to uops directly so that it can obtain "fresh" values at the point of lowering.
* The tracing version behaves nearly identical to the normal interpreter, in fact it even has specialization! This allows it to run without much of a slowdown when tracing. The actual cost of tracing is only a function call and writes to memory.
* The tracing interpreter uses the specializing interpreter's deopt to naturally form the side exit chains. This allows it to side exit chain effectively, without repeating much code. We force a re-specializing when tracing a deopt.
* The tracing interpreter can even handle goto errors/exceptions, but I chose to disable them for now as it's not tested.
* Because we do not share interpreter dispatch, there is should be no significant slowdown to the original specializing interpreter on tailcall and computed got with JIT disabled. With JIT enabled, there might be a slowdown in the form of the JIT trying to trace.
* Things that could have dynamic instruction pointer effects are guarded on. The guard deopts to a new instruction --- `_DYNAMIC_EXIT`.
@chris-eibl chris-eibl mentioned this pull request Feb 14, 2026
remove unused _PyFunction_LookupByVersion #144814
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sergey-miryanov sergey-miryanov sergey-miryanov left review comments
@efimov-mikhail efimov-mikhail efimov-mikhail left review comments
@savannahostrowski savannahostrowski savannahostrowski left review comments
@kumaraditya303 kumaraditya303 kumaraditya303 left review comments
@chris-eibl chris-eibl chris-eibl left review comments
@markshannon markshannon markshannon approved these changes
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner
@ZeroIntensity ZeroIntensity Awaiting requested review from ZeroIntensity ZeroIntensity is a code owner
@tomasr8 tomasr8 Awaiting requested review from tomasr8 tomasr8 is a code owner
@brandtbucher brandtbucher Awaiting requested review from brandtbucher brandtbucher is a code owner
@diegorusso diegorusso Awaiting requested review from diegorusso diegorusso is a code owner
@erlend-aasland erlend-aasland Awaiting requested review from erlend-aasland
@AA-Turner AA-Turner Awaiting requested review from AA-Turner AA-Turner is a code owner
@emmatyping emmatyping Awaiting requested review from emmatyping
@ezio-melotti ezio-melotti Awaiting requested review from ezio-melotti ezio-melotti is a code owner
@hugovk hugovk Awaiting requested review from hugovk hugovk is a code owner
@Eclips4 Eclips4 Awaiting requested review from Eclips4 Eclips4 is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

10 participants

@Fidget-Spinner @markshannon @Eclips4 @mhsmith @efimov-mikhail @sergey-miryanov @hugovk @savannahostrowski @kumaraditya303 @chris-eibl
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.