-
PH.D. student at Sun Yat-sen university
-
AI Infra, MLSys, Simulaters, GPU architecture
- [2026/01/13] [LMSYS blog] EPD Disaggregation: Elastic Encoder Scaling for Vision-Language Models in SGLang
- [2025/06/27] [arXiv] [Code] gLLM is accepted by SC'25. Congratulations!
- [2025/05/28] [arXiv] [Code] EFIM is accepted by Euro-Par'25
- [2025/04/27] [arXiv] [Code] We have released gLLM, an efficient pipeline parallelism inference engine for LLM.
32 merged PRs
- [Model] Add Clef-Flash support
- [Bugfix][KVConnector][NIXL] Fix TP handshake test after per-region FA desc flags
- [Bugfix][Model] Fix Qwen3-Omni interleaved M-RoPE boundary positions
- [Bugfix][Attention] Restrict FlashInfer sparse MLA FULL graphs to decode
- [Bugfix][EPD] Skip sampling for encoder-only async steps
Show 27 more PRs
- [Bugfix] Disable prefix caching for encoder-only before model config hooks
- [Bugfix][EC Connector] Size transfers by measured encoder output widths
- [EPD] Support metadata-only audio inputs
- [Perf][EPD] Batch image requests per encoder
- [Performance][EPD] Reduce Python proxy serialization overhead
- [Bugfix][EPD] Preserve explicit multimodal UUIDs with caches disabled
- [EC] Automatically enable embedding inputs on EC/KV consumers
- [Frontend][EPD] Use JSON arrays for multimodal metadata
- [Bugfix][EC Connector] Fail the request, not the engine, when a remote encoding cannot arrive
- [Feature][EPD] Support dynamic EPD (EC-connector) proxy
- [Config][EC] Normalize producer-only encoder config
- [Refactor][Model Runner V2][Multimodal] Move the encoder-only path out of the shared runner
- [Bugfix][V1][Multimodal] Ignore stale same-step encoder cache evictions
- [Perf] Avoid repeated multimodal prompt update scans
- [Bugfix][EPD][Model Runner V2] Skip gather mm embeddings for encoder only instance
- [EPD] Remove duplicate image preprocessing in EPD and enable preprocess on GPU
- [Bugfix][EC Connector] Don't stop an encoder-instance request before its images are encoded
- [Model] Support Unlimited OCR
- [Bugfix][Multimodal] Fix Qwen3-Omni use_audio_in_video with mixed image/video inputs
- Fix the order of _free_encoder_inputs
- Remove unused EVS functions in qwen3_vl.py
- Support online use_audio_in_video
- [Bugfix] Fix benchmark_moe.py
- [Benchmark] Refactor sample_requests in benchmark_throughput
- [Bugfix] fix automatic prefix args and add log info
- [Minor Fix] Fix comments in benchmark_serving
- [Minor Fix] Remove unused code in benchmark_prefix_caching.py
1 merged PR
6 merged PRs
- Support PP for zmq_to_scheduler
- Add EPD disaggregation doc
- feat: support EPD disaggregation
- [VLM] Support PP for Qwen2.5-VL
- Add PP support for dots_vlm
Show 1 more PR
2 merged PRs
1 merged PR
2 merged PRs