IREElink

IREE (Intermediate Representation Execution Environment¹) is an MLIR-based end-to-end compiler and runtime that lowers Machine Learning (ML) models to a unified IR that scales up to meet the needs of the datacenter and down to satisfy the constraints and special considerations of mobile and edge deployments.

Key featureslink

Support matrixlink

IREE supports importing from a variety of ML frameworks:

• JAX
• ONNX
• PyTorch
• TensorFlow and TensorFlow Lite

The IREE compiler tools run on Linux, Windows, and macOS and can generate efficient code for a variety of runtime platforms:

• Linux
• Windows
• macOS
• Android
• iOS
• Bare metal
• WebAssembly (experimental)

and architectures:

• ARM
• x86
• RISC-V

Support for hardware accelerators and APIs is also included:

• Vulkan
• ROCm/HIP
• CUDA
• Metal (for Apple silicon devices)
• AMD AIE (experimental)
• WebGPU (experimental)

Project architecturelink

IREE adopts a holistic approach towards ML model compilation: the IR produced contains both the scheduling logic, required to communicate data dependencies to low-level parallel pipelined hardware/API like Vulkan, and the execution logic, encoding dense computation on the hardware in the form of hardware/API-specific binaries like SPIR-V.

Workflow overviewlink

Using IREE involves the following general steps:

1. Import your model

   Develop your program using one of the supported frameworks, then import into IREE

2. Select your deployment configuration

   Identify your target platform, accelerator(s), and other constraints

3. Compile your model

   Compile through IREE, picking settings based on your deployment configuration

4. Run your model

   Use IREE's runtime components to execute your compiled model

Importing models from ML frameworkslink

IREE supports importing models from a growing list of ML frameworks and model formats:

• JAX
• ONNX
• PyTorch
• TensorFlow and TensorFlow Lite

Selecting deployment configurationslink

IREE provides a flexible set of tools for various deployment scenarios. Fully featured environments can use IREE for dynamic model deployments taking advantage of multi-threaded hardware, while embedded systems can bypass IREE's runtime entirely or interface with custom accelerators.

• What platforms are you targeting? Desktop? Mobile? An embedded system?
• What hardware should the bulk of your model run on? CPU? GPU?
• How fixed is your model itself? Can the weights be changed? Do you want to support loading different model architectures dynamically?

IREE supports the full set of these configurations using the same underlying technology.

Compiling modelslink

Model compilation is performed ahead-of-time on a host machine for any combination of targets. The compilation process converts from layers and operators used by high level frameworks down into optimized native code and associated scheduling logic.

For example, compiling for GPU execution using Vulkan generates SPIR-V kernels and Vulkan API calls. For CPU execution, native code with static or dynamic linkage and the associated function calls are generated.

Running modelslink

IREE offers a low level C API, as well as several sets of API bindings for compiling and running programs using various languages.

Communitylink

IREE is a sandbox-stage project of LF AI & Data Foundation made possible thanks to a growing community of developers. The project is governed by a Technical Steering Committee — see GOVERNANCE.md.

Find out about IREE community meetings and see how IREE is used:

Community

Project newslink

• 2025-04-02: AMD submitted an IREE-based SDXL implementation to the MLPerf benchmark suite
• 2024-05-23: IREE joins the LF AI & Data Foundation as a sandbox-stage project

Communication channelslink

• GitHub issues: Feature requests, bugs, and other work tracking
• IREE Discord server: Daily development discussions with maintainers, contributors, and collaborators
• (New) iree-announce email list: Announcements
• (New) iree-technical-discussion email list: General and low-priority discussion
• (Legacy) iree-discuss email list: Announcements, general and low-priority discussion

Project operationslink

Developer documentationlink

Interested in contributing to IREE? Check out our developer documentation:

Developers

Roadmaplink

IREE uses GitHub Issues for most work planning. Some subprojects use both GitHub Projects and GitHub Milestones to track progress.