A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient deployment.

awesome awesome-list binarization awesome-lists vector-quantization binary-neural-networks efficient-inference multimodal low-precision diffusion-models model-quantization kv-cache quantization-aware-training post-training-quantization data-free-quantization large-language-models quantizations kv-cache-quantization lattice-quantization large-language-models-llms

• Updated Oct 26, 2026

A curated list for Efficient Large Language Models

compression language-model knowledge-distillation model-quantization pruning-algorithms llm llm-compression efficient-llm

• Updated Jun 17, 2025
• Python

模型压缩的小白入门教程，PDF下载地址 https://github.com/datawhalechina/awesome-compression/releases

compression quantization svd knowledge-distillation model-pruning prune model-compression neural-architecture-search kd model-quantization low-rank-matrix-decomposition

• Updated Nov 25, 2025

Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).

bloom falcon moe gemma mistral mixture-of-experts model-quantization multi-gpu-inference m2m100 llamacpp llm-inference internlm llama2 qwen baichuan2 mixtral phi-2 deepseek minicpm

• Updated Mar 15, 2024
• C++

This repository contains notebooks that show the usage of TensorFlow Lite for quantizing deep neural networks.

inference pruning tensorflow-lite tensorflow-2 on-device-ml tf-hub model-quantization model-optimization quantization-aware-training post-training-quantization tf-lite-model

• Updated Jan 23, 2023
• Jupyter Notebook

[WINNER! 🏆] Psychopathology FER Assistant. Because mental health matters. My project submission for #TFWorld TF 2.0 Challenge at Devpost.

python raspberry-pi flask tensorflow dash firebase-realtime-database kaggle-dataset google-colab tflite kaggle-api assistant-app model-quantization dash-bootstrap-components

• Updated Mar 25, 2023
• Jupyter Notebook

[NeurIPS 2023 Spotlight] This project is the official implementation of our accepted NeurIPS 2023 (spotlight) paper QuantSR: Accurate Low-bit Quantization for Efficient Image Super-Resolution.

super-resolution quantized-neural-networks model-quantization

• Updated Sep 11, 2026
• Python

[ICML 2023] This project is the official implementation of our accepted ICML 2023 paper BiBench: Benchmarking and Analyzing Network Binarization.

benchmark binarization model-compression binary-neural-networks binarized-neural-networks model-quantization icml-2023

• Updated Sep 11, 2026
• Python

离线可用的本地类型化决策：4 核 CPU 单题 15.6ms。Local & offline Jev / System One inference on CPU — ONNX + INT8, no torch at runtime. 支持 laya / kev / PlayJev

offline-first decision-model quantization int8 laya onnx edge-ai kev on-device-ai onnxruntime model-quantization cpu-inference local-inference llm-inference offline-inference calibrated-probabilities jev typesafe-ai system-one typed-decisions

• Updated Nov 1, 2026
• Python

High-accuracy Russian ANPR system built with YOLOv8 for detection and an optimized, custom-trained PyTorch CRNN for OCR.

python opencv machine-learning ocr computer-vision deep-learning pytorch yolo object-detection anpr alpr fine-tuning crnn licence-plates-recognition model-quantization ultralytics yolov8

• Updated Oct 22, 2025
• Jupyter Notebook

The official implementation of the ICML 2023 paper OFQ-ViT

icml model-compression model-compression-papers model-quantization vision-transformer vision-transformers icml2023 quantization-awar

• Updated Oct 3, 2023
• Python

[ICLR 2026] This is the official PyTorch implementation of "QVGen: Pushing the Limit of Quantized Video Generative Models".

wan iclr qat video-generation diffusion-models videogen model-quantization quantization-aware-training generative-ai text-to-video-generation cogvideox wan21 iclr2026

• Updated Feb 11, 2026
• Python

Chat to LLaMa 2 that also provides responses with reference documents over vector database. Locally available model using GPTQ 4bit quantization.

chatbot cuda transformers question-answering gpt quantization rye model-quantization chatai streamlit-chat chatgpt langchain llama2 llama-2

• Updated Nov 25, 2023
• Python

A list of papers, docs, codes about diffusion quantization.This repo collects various quantization methods for the Diffusion Models. Welcome to PR the works (papers, repositories) missed by the repo.

awesome model-compression model-acceleration diffusion-models model-quantization

• Updated Feb 2, 2026

A tutorial of model quantization using TensorFlow

machine-learning tensorflow tensorflow-lite tflite model-quantization inference-efficiency quantization-aware-training

• Updated Aug 2, 2021
• Python

H.E.R.A. (Healing Evaluation and Recognition Architecture) — an on-device edge-AI system for offline wound assessment using YOLOv8 segmentation, HSV tissue analysis, and PUSH-based decision logic.

android opencv computer-vision mobile-app image-segmentation semantic-segmentation coreml tflite edge-ai medical-ai on-device-ai digital-health model-quantization healthtech tinyml healthcare-ai yolov8 wound-assessment hsv-color-space wound-care

• Updated Nov 4, 2026
• Dart

AI Engineering: Annotated NBs to dive into Self-Attention, In-Context Learning, RAG, Knowledge-Graphs, Fine-Tuning, Model Optimization, and many more.

embeddings transformer knowledge-graph chunking bert fine-tuning self-attention model-quantization weights-and-biases huggingface-transformers ai-engineering in-context-learning large-language-models generative-ai langchain retrieval-augmented-generation llama3-1

• Updated Apr 2, 2025
• Jupyter Notebook

The Ark Project: Selecting the perfect AI model to reboot civilization from a 64GB USB drive. Comprehensive analysis of open-source LLMs under extreme constraints, with final recommendation: Meta Llama 3.1 70B Instruct (Q6_K GGUF). Includes interactive tools, detailed comparisons, and complete implementation guide for offline deployment.

model-quantization cpu-inference llm llama-cpp qwen gguf open-source-ai mixtral offline-ai meta-llama survival-technology civilization-reboot

• Updated Aug 7, 2025
• HTML

🧠 A comprehensive toolkit for benchmarking, optimizing, and deploying local Large Language Models. Includes performance testing tools, optimized configurations for CPU/GPU/hybrid setups, and detailed guides to maximize LLM performance on your hardware.

cuda gpu-acceleration model-management inference-optimization model-quantization cpu-inference llama-cpp local-llm llm-deployment llm-benchmarking ollama-optimization hybrid-inference wsl-ai-setup context-window-scaling

• Updated Mar 27, 2025
• Shell

Official ICML 2026 Spotlight implementation for structural MoE compression, including attribution-guided channel scoring, coverage-maximized pruning, compact checkpoint construction, and fine-tuning support.

moe pruning sparsification model-quantization llm structural-pruning llm-compression

• Updated May 23, 2026
• Python