Deeploy is an ONNX-to-C compiler that generates low-level optimized C Code for multi-cluster, heterogeneous SoCs. Its goal is to enable configurable deployment flows from a bottom-up compiler perspective, modeling target hardware in a fine-grained and modular manner.
Deeploy is developed as part of the PULP project, a joint effort between ETH Zurich and the University of Bologna.
You can find the documentation at the following links:
A DeepWiki generated documentation is availabe here.
Download the repository and its submodules:
git clone https://github.com/pulp-platform/Deeploy.git && cd Deeploy
git submodule update --init --recursive
Installing Deeploy is as simple as running:
pip install -e .
However, to run the code generated by Deeploy on a certain target, you need the toolchains and the simulators associated with this platform.
We provide a Docker container where Deeploy works Out-of-the-Box (i.e. with all the dependencies pre-installed). To pull the docker image, run:
docker pull ghcr.io/pulp-platform/deeploy:main
Then you can create and start the container in interactive mode with:
docker run -it --name deeploy_main -v $(pwd):/app/Deeploy ghcr.io/pulp-platform/deeploy:main
Install Deeploy inside the container in editable mode:
cd Deeploy
pip install -e .
Congratulations, you installed Deeploy and its dependencies! Now, to test your installation let's run one simple test on each platform with the following commands:
cd DeeployTest
python deeployRunner_generic.py -t ./Tests/Kernels/Integer/Add/Regular
python deeployRunner_cortexm.py -t ./Tests/Kernels/Integer/Add/Regular
python deeployRunner_mempool.py -t ./Tests/Kernels/Integer/Add/Regular
python deeployRunner_snitch.py -t ./Tests/Kernels/Integer/Add/Regular
python deeployRunner_tiled_snitch.py -t ./Tests/Kernels/Integer/Add/Regular
python deeployRunner_siracusa.py -t ./Tests/Kernels/Integer/Add/Regular
python deeployRunner_tiled_siracusa.py -t ./Tests/Kernels/Integer/Add/Regular
python deeployRunner_tiled_siracusa_w_neureka.py -t ./Tests/Kernels/Integer/Add/Regular
python deeployRunner_softhier.py -t ./Tests/Kernels/Integer/Add/Regular --toolchain=GCC
python deeployRunner_chimera.py -t ./Tests/Kernels/Integer/Add/Regular
To restart and connect to the container, run:
docker start -i deeploy_main
cd Deeploy
You can find the ONNX file in DeeployTest/Tests/Kernels/Integer/Add/Regular, to visualize it, you can use Netron. You can also find the generated code for the platform X in TEST_X in DeeployTest and you should notice that the generated code for the Add test is very simple. However, this gets more complex when you add tiling. Let's generate the code for a single layer but using tiling this time:
python deeployRunner_tiled_siracusa.py -t Tests/Kernels/Integer/MatMul/Regular --cores=8 --l1=16000
Now you can open the generated code in DeeployTest/TEST_SIRACUSA/Tests/testMatMul/Network.c and see how we executed a tiled layer.
| Platform | Hardware | Simulator | CI Status |
|---|---|---|---|
| Generic CPU | Your laptop CPU :) | Host |  |
| CortexM Processors | Documentation | QEMU |  |
| MemPool + ITA | Mempool paper, ITA paper | Banshee |  |
| Siracusa | Siracusa paper | GVSoC |    |
| Snitch Cluster | Snitch paper | GVSoC |   |
| SoftHier | Repo | GVSoC |  |
| Chimera | Repo | GVSoC |  |
| GAP9 | Repo | GVSoC |   |
If you use Deeploy in your work or research, you can cite us with:
ESWEEK 2024: Deeploy: Enabling Energy-Efficient Deployment of Small Language Models On Heterogeneous Microcontrollers
@article{schererDeeployEnablingEnergyEfficient2024,
title = {Deeploy: {{Enabling Energy-Efficient Deployment}} of {{Small Language Models}} on {{Heterogeneous Microcontrollers}}},
shorttitle = {Deeploy},
author = {Scherer, Moritz and Macan, Luka and Jung, Victor J. B. and Wiese, Philip and Bompani, Luca and Burrello, Alessio and Conti, Francesco and Benini, Luca},
year = {2024},
month = nov,
journal = {IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems},
volume = {43},
number = {11},
pages = {4009--4020},
issn = {1937-4151},
doi = {10.1109/TCAD.2024.3443718},
}
The preprint is available on arXiv @ arXiv:2408.04413.
IEEE Design & Test: Toward Attention-based TinyML: A Heterogeneous Accelerated Architecture and Automated Deployment Flow
@article{WieseTowardAttentionBasedTinyML2025,
author={Wiese, Philip and İslamoğlu, Gamze and Scherer, Moritz and Macan, Luka and Jung, Victor J.B. and Burrello, Alessio and Conti, Francesco and Benini, Luca},
journal={IEEE Design & Test},
title={Toward Attention-based TinyML: A Heterogeneous Accelerated Architecture and Automated Deployment Flow},
year={2025},
pages={1-1},
keywords={Tiny machine learning;Transformers;Memory management;Hardware acceleration;Bandwidth;Registers;Software;Engines;Energy efficiency;Computational modeling;Neural Networks;TinyML;Deployment;Transformers;Accelerators},
doi={10.1109/MDAT.2025.3527371}}
The preprint is available on arXiv @ arXiv:2408.02473.
All licenses used in this repository are listed under the LICENSES folder. Unless specified otherwise in the respective file headers, all code checked into this repository is made available under a permissive license.
- Most software sources and tool scripts are licensed under the Apache 2.0 license.
- Some files in the
scriptsdirectory are licensed under the MIT license. - Markdown, JSON, text files, pictures, and files in the
DeeployTest/Testsdirectory are licensed under the Creative Commons Attribution 4.0 International license (CC BY 4.0).
To extract license information for all files, you can use the reuse tool and by running reuse spdx in the root directory of this repository.