English | 繁體中文

Native C++ tools, GPU & edge AI, and performance engineering.
I turn real-world problems into practical tools, and share the experiments, methods, and trade-offs behind them.

Technical writing on Medium · Projects & contributions · Research portfolio

How does a model reach different silicon, and what does that actually cost? My work connects model conversion, native runtimes, GPU memory, compiler-generated kernels, and reproducible measurement.

• Multi-backend edge inference: a C++20 framework with a stable execution-provider interface and separate model-family logic. I evaluate detection and RIFE frame interpolation across ten providers, spanning Windows, Jetson, and Android, and investigate IREE/RVV compiler paths.
• GPU-resident video: keep frames on the GPU across decode, interpolation, and encode. I study buffer interoperability, the CPU transfer envelope, and the effects on latency and energy—not just the neural-network kernel.
• LLM performance engineering: use a shared HTTP harness to compare vllm.cpp, llama.cpp, and vLLM; separate prefill from decode, control caching and concurrency, and investigate paged KV memory, dispatch overhead, GEMM/GEMV, and Vulkan/Triton CPU kernels.

Research portfolio & methodology · Published edge-inference deep dive

I contribute to vllm.cpp, a community C++ inference engine. My port/vulkan development branch brings together work on Vulkan compute kernels and subgroup portability, an optional Triton CPU provider, native CPU optimization, and Windows/CUDA build support.

• Vulkan: capability-gated NVIDIA cooperative-matrix paths, attention kernels that use the device's subgroup width, and decode GEMV tuning.
• CPU: an opt-in Triton CPU provider with native fallback, plus AVX-512 attention and matmul optimizations. The Triton CPU kernel loader currently serves operations on Linux.
• Windows & CUDA: MSVC portability repairs, consumer Blackwell Triton AOT assets, and fork-specific release work.

Explore my branch · Contribution record & selected commits

This is a contribution fork of mudler/vllm.cpp. The contribution record links my implementation commits and separates branch work from verified upstream acceptance.

I write in English and Traditional Chinese about native software, GPU pipelines, local AI, and the measurements behind engineering decisions.

• Edge inference across CPU, GPU, NPU, and RISC-V — execution-provider design, correctness checks, and GPU-resident video pipelines. Research write-up; a public standalone engine repository is not listed here.
• GPU thermal protection for sustained AI workloads — the investigation behind GpuThermalGuard.
• A modern C++ hooking runtime — the design behind Splice.
• Explore more articles on Medium →

GWT agent integrations · Midp2EXE · Windows SDK 7.1 / Visual Studio

A Windows utility for removing ASUS software, services, drivers, and leftover registry entries, with a dry-run preview.

Current repository & usage · Original gist / migration notice

The tool moved from the gist to the repository. Start with the current repository; the gist preserves the original entry point and directs existing users to the new home.

My repositories include tools I maintain, forks with substantial implementation work, and forks used for research. Start with projects & contributions to explore my work. A repository's fork label describes its lineage; the linked contributions show what I added.