GeezerVison is showing github.com — Make the JIT thread-safe · Issue #133171 · python/cpython
Make the JIT thread-safe #133171DescriptionCurrently, --enable-experimental-jit and --disable-gil can be configured together. However, the behavior in this configuration is probably surprising, and not what the user intends: the JIT is built, but never actually used.
I've been seeing a lot of people configuring these "bleeding edge" experimental builds now, and we even have one in CI (we've been using it to make sure the JIT build step isn't broken). Until the free-threaded build supports the JIT, we should start by erroring during the configure step for these builds in 3.14. I'll open a PR to do this soon.
For 3.15, the JIT should be updated to work on the free-threaded build. This shouldn't be done in a way that pessimizes the JIT in the default build... it will probably start with a bunch of #ifdef Py_GIL_DISABLED/#ifndef Py_GIL_DISABLED blocks, in order to at least get something working.
This isn't something that I am planning on working on personally. This isn't a good beginner issue either. If there is somebody with free-threading experience interested in learning more about the JIT and willing to take on this project, then that would be great. I can't claim deep free-threading knowledge, but I'm willing to walk you through how the JIT works, help identify what the potential thread-safety issues may be, and review/debug any PRs.
In my understanding, here's a very high-level triage of the current situation:
- (phase1)
JUMP_BACKWARD: We need to re-enable specialization for this instruction. This shouldn't be too hard, since we already have idioms for specialization on the free-threaded build.
- (phase1)
bytecodes.c: There are several instructions that are tier-two (JIT) only. These need to be audited for thread-safety.
- (phase1)
_PyOptimizer_Optimize: Finding space for and inserting executors into the code object needs to be done in a thread-safe way, which probably will involve locking the code object such that only one part of it can be compiled at a time.
- (phase1)
_PyExecutorObject: These are objects that manage JIT code. They are joined into a giant doubly-linked list, and are unlinked as they are invalidated or no longer needed. This list needs to be made thread-safe.
- (phase1)
translate_bytecode_to_trace: This shouldn't be too bad, since each thread has its own copy of the bytecode and inline caches (so we don't need to worry about concurrent mutations during tracing). Function inlining and for loop headers may need some attention.
- (phase1)
_PyJIT_Compile: This is probably fine, actually, which is a relief. The entire machine-code backend is stateless and self-contained. As long as mmap/mprotect/munmap/etc. are threadsafe, then we're okay. The generated code is immutable, and can be shared between threads without issue.
- (phase2)
_Py_uop_analyze_and_optimize is going to be where most of our time and ongoing effort is spent. Thankfully it's all optional (in theory), so we may just be able to turn it all off and selectively turn things back on. It consists of three passes:
remove_globals: This has the potential to be annoying, but I think it may not be too bad (as long as watchers are already thread-safe?).optimize_uops: This is going to be the long tail, I think. This one is different, since we don't only care if the optimize_uops itself is thread-safe (easy), but also if our optimizations will be thread-safe at runtime (hard). We'll probably just end up #ifdef'ing out many of the bodies in optimizer_bytecodes.c and figuring out how to turn them back on (if at all).remove_unneeded_uops: This already looks fine to me.
If stuff is missing, please add it to the list.
It may even be reasonable to stop the world for JIT compilation initially (just to get something working), then later introduce finer-grained locking as needed.
Linked PRs
Reacted by Djemmal Islam and Bogdan Romanyuk Reacted by Sam Gross, Donghee Na, nitishxp, Kirill Podoprigora, Diego Russo, Oskar Skog, Виталий Дмитриев, Tomas R., Mikhail Efimov, andreaimprovised and 1 more Activityadded a commit that references this issue on May 2, 2025GH-133171: Prevent combinations of --disable-gil and --enable-experim… corona10 commented on Sep 18, 2025 I discussed this with @brandtbucher.
Our first milestone for JIT × free-threading will be running JIT without any optimizations.
To achieve this, we need to complete the items marked as Phase 1 in the original issue description.
Once the phase 1 is done, we can CI for JITxFree-threading without any optimization.
cc @Fidget-Spinner corona10 commented on Sep 18, 2025 Zheaoli commented on Sep 18, 2025 A little bit curious, is this PR open to people claiming different targets? corona10 commented on Sep 18, 2025
A little bit curious, is this PR open to people claiming different targets?
Can you elaborate what is diffrent target? Zheaoli commented on Sep 18, 2025
Can you elaborate what is diffrent target?
My bad, here's misunderstanding here. I mean is issue open to people who want to claiming the target we list above.(not different target) corona10 commented on Sep 18, 2025
My bad, here's misunderstanding here. I mean is issue open to people who want to claiming the target we list above.(not different target)
If you want to tackle a specific issue from the list, feel free to work on it.
But first, PR #137800 should be merged. See the explanation here: link to comment. Reacted by Nadeshiko Manju corona10 commented on Sep 18, 2025 Some design decisions with Brandt: we decided not to create executors per thread, since we don’t want duplicated traces across threads. A simple lock around the logic when inserting executors should be fine. If this causes performance issues, we can reconsider moving it to per-thread, but let’s see how it goes for now. corona10 commented on Oct 4, 2025 The next target is to investigate why this line fails when uop_optimize is executed.
Assertion failed: (PyObject_IS_GC(op)), function op_from_block, file gc_free_threading.c, line 367.
/bin/sh: line 1: 217 Abort trap: 6 ./python.exe -E -S -m sysconfig --generate-posix-vars
efimov-mikhail commented on Oct 4, 2025 FWIW, I've found some PyStackRef problem with sysconfig, which I'm still investigating.
There is minimal example:
-> % cat issue_lib.py
X = "TEXT"
-> % cat issue.py
import issue_lib
def f():
s = "TEXT"
return s
f()
-> % ./python issue.py
If issue_lib.py is not compiled yet, there is no problem.
But with pyc file compiled there is a PyStackRef borrow issue: some PyStackRef is closed before borrowed PyStackRef was closed.
Edited: It seems there's no real problem in PyStackRef borrowing, some issue occurs with immortal string. Reacted by Donghee Na and Sergey Miryanov Fidget-Spinner commented on Nov 7, 2025 I have a different idea for _Py_uop_analyze_and_optimize:
We could what (I believe) ZJIT in Ruby does. Ruby also has the concept of free-threading, where multiple ractors may execute at the same time.
Phase 1:
For the first and simplest phase, we could add a watcher/callback to thread creation. When multiple threads are created, we simply invalidate the executor as _CHECK_VALIDITY fails.
This means single threaded code gets all the benefits of JIT in the future, while multi-threaded code will lose all the benefits. However, both can coexist at the same time in the same build. We will finally get JIT + FT in a limited form.
Phase 2:
If we detect multiple threads are running, we turn off the non-thread safe optimizations, and redo the trace with those optimizations off. If we detect only a single thread, we follow phase 1 and run till we get invalidated. This allows single threaded code to run faster, while multi-threaded code will run a little slower. Note that all optimizations I've proposed for the JIT in the past 6 months and in the future are FT safe (in theory. whether they are implemented in practice is different). Reacted by Donghee Na, Nadeshiko Manju, Mikhail Efimov and Chris Eibl Fidget-Spinner commented on Nov 9, 2025
The next target is to investigate why this line fails when uop_optimize is executed.
Assertion failed: (PyObject_IS_GC(op)), function op_from_block, file gc_free_threading.c, line 367.
/bin/sh: line 1: 217 Abort trap: 6 ./python.exe -E -S -m sysconfig --generate-posix-vars
I cannot repro on main anymore. I think it fixed itself? Fidget-Spinner commented on Nov 15, 2025 I created an alternative design here #141594. The PR turns on the entire JIT including optimizer on FT. hroncok commented on Nov 25, 2025
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|