Compiler frontend infrastructure with tiered JIT compilation and native code generation

Zyntax is a compiler frontend infrastructure written in Rust. A language frontend hands it a typed AST; Zyntax lowers that to an SSA IR, optimises it once, starts running it in an interpreter, and compiles each function to native code as it proves hot.

• Tiered execution: HIR interpreter, Cranelift and LLVM, promoted per function, with on-stack replacement into running loops
• One optimisation pipeline: each body is optimised once in HIR and every tier compiles that same body
• SIMD in the IR: vector operations are HIR instructions both CPU tiers lower natively
• Effects, fibers and async as first-class HIR operations
• Runtime plugins (ZRTL): native libraries linked into the executables
• Grammar-defined languages: a Zyn grammar maps syntax to typed AST, so a new language needs no Rust

cargo build --release --features llvm-backend --bin zypy --bin zylua --bin zynml --bin zyntax

ZYPY_LLVM=1 ./target/release/zypy run program.py
ZYLUA_LLVM=1 ./target/release/zylua program.lua
./target/release/zynml run program.zynml
./target/release/zyntax repl --grammar examples/zpeg_test/calc.zyn

ZYPY_LLVM and ZYLUA_LLVM let hot code tier up to LLVM; without them the interpreter and Cranelift run it. The LLVM build needs LLVM 21 (see Contributing); leave out --features llvm-backend for a build without it.

New to Zyn grammars: The Zyn Book.

A frontend produces a typed AST, either from a Zyn grammar's semantic actions or from a hand-written frontend. The typed AST is lowered to HIR, an SSA form over basic blocks, and HIR is optimised once. The tiered runtime interprets it first and promotes functions to Cranelift, then to LLVM, as their counters cross the thresholds.

See docs/ARCHITECTURE.md for the layers and docs/tiered-compilation.md for the tier ladder.

• The Zyn Book: grammars, the CLI, the typed AST, packaging, embedding, plugins
• Architecture: layers, IR, backends
• Zyn grammar spec: syntax and semantic actions
• Embedding SDK: running Zyntax inside a Rust program
• Async, effects and fibers
• Bytecode format

The LLVM tier needs LLVM 21, which llvm-sys finds through LLVM_SYS_211_PREFIX:

# macOS
brew install llvm@21
export LLVM_SYS_211_PREFIX=$(brew --prefix llvm@21)

# Debian / Ubuntu
wget -qO llvm.sh https://apt.llvm.org/llvm.sh && sudo bash llvm.sh 21 all
export LLVM_SYS_211_PREFIX=/usr/lib/llvm-21

cargo build
cargo test -p zyntax_compiler
cargo test -p zyntax_lua --test conformance
cargo test -p zyntax_python --test conformance

Test the crates you change; the whole workspace takes a long time.

Issues are tracked with git-bug and live in the repository under refs/bugs/*. Run git-bug pull to fetch them, then git-bug bug --status open to list open work; filter with --label bug, --label perf or an area such as --label area:lua, and read one with git-bug bug show <id>.

Apache License 2.0. See LICENSE.

The Cranelift project, LLVM, the Lua and Python projects, and the Rust community.

Issues · Discussions