Open Source GPU Performance Engineer — Making every FLOP count.
Field Application Engineer @ Taiwan AILabs
wayne.is-a.dev · PR wall · LinkedIn · X @itswaynechiu
GPU & kernels
Inference serving
Languages & ML
Systems
- Notes — TensorRT-LLM + Triton serving
- Tensor-parallel & the NVLink wall
- RAG groundedness: 0% vs 50%
- Notes — CUDA Tensor Core GEMM (WMMA)
I work at the intersection of GPU systems and ML — writing CUDA kernels, benchmarking inference stacks, and shipping fixes upstream to the LLM-serving ecosystem. My niche: early consumer-Blackwell (SM120 / SM121) enablement, validated on real Blackwell workstation hardware — the kernels and dispatch paths most repos can't yet test because almost no one has the hardware.
Across the LLM-inference stack — 17 merged/landed · 60 in review — see the live PR wall → prs.wayne.is-a.dev (auto-updating, with RSS).
Merged / landed in main
| Project | # | Representative merged work |
|---|---|---|
| FlashInfer | 7 | SM120/121 multi-CTA radix top-k stream hang · SM120 NVFP4 attention qk-correction layout / row-sum / lse · MXFP8-aware MoE gemm profiler |
| vLLM | 3 | Tokenizer survives pickling · streaming message_start sets type/role · clear error for structured outputs on diffusion decoders |
| LMDeploy | 3 | InternVL LoRA loading TypeError · double-counted max_q_seqlen in decode delta · builtin chat-template ImportError |
| PyTorch | 1 | nccl.broadcast was dropping its root argument — landed in main (c3c33fd) |
| Dynamo | 1 | KV-router cancels in-flight recovery on worker removal |
| compressed-tensors | 1 | Skip device-map entries with no local module in dispatch |
| torchao | 1 | PT2E/X86 plain-linear annotation fallback for reused nn.Linear |
In review — grouped by project (counts as of 2026-07-18):
| Project | # | Representative work |
|---|---|---|
| SGLang | 17 | SM120/SM121 dispatch for int8_scaled_mm & fp8_blockwise_scaled_grouped_mm · SM120 shared-mem-safe attention block size |
| vLLM | 15 | NVFP4 MoE per-expert scale validation · FP8 MoE+LoRA routed to Marlin · async-KV-load scheduling fix |
| FlashInfer | 13 | NVFP4 global-scale threading through the unified MoE API · cuDNN full-sequence Q batch stride in batch prefill |
| Dynamo | 5 | Blackwell workstation GPU SKU support · KV-router hardening follow-ups |
| NVIDIA TensorRT-LLM | 4 | CuteDSL MoE ghost-token & global-index fixes · DeepSeek-V2-Lite e_score_correction_bias guard |
| NVIDIA CUTLASS | 4 | SM120 grouped NVFP4 block-scaled GEMM in cutlass_library · CuTeDSL sub-byte make_ptr / is_major / uint-lowering fixes |
| torchao | 1 | Reused-module fallback extended to nn.Conv2d (follow-up to merged #4480) |
| LMCache | 1 | Clear reqs_status on async-lookup timeout to prevent recall KeyError |
Most SM120 work above is reproduced and validated on real Blackwell workstation hardware.
| Project | What it is | Key number |
|---|---|---|
| tensor-core-from-scratch | 10 progressive CUDA matmul kernels from naive to tensor cores on Blackwell. | 100 TFLOPS (34% of cuBLAS HGEMM) |
| inference-kernel-cookbook | LLM inference techniques from scratch — Flash Attention, KV Cache, Paged Attention. | 81× speedup, 1000× memory reduction |
| trtllm-triton-serving | TensorRT-LLM vs vLLM head-to-head on H100 — 12 studies reproducing NVIDIA's 27.7k tok/s. | 100.3% of published benchmark |
| nccl-collectives-bench | NCCL collective benchmarks on 8×H100 NVSwitch — bandwidth, latency, NVLS, TP-decode ceiling. | 365 GB/s (77% NVLink budget) |
| llm-security-lab | LLM security from first principles — prompt extraction, injection, model stealing. Attack + defense. | 19 leaks → 2 with defense |
| Project | What it is |
|---|---|
| blackwell-tensorcore-kernels | Hand-written CUDA Tensor Core GEMM kernels on Blackwell (sm_120) and Hopper (sm_90). |
| federated-learning-lab | From-scratch FL: FedAvg / FedProx / SCAFFOLD / FedLoRA, DP-SGD & secure aggregation. 33/33 tests. |
| nim-agent-blueprint | Agentic RAG on NVIDIA NIM — hallucination evaluation on adversarial SQuAD 2.0 (N=200). |
| physgate | Validate LLM-generated robot plans in GPU physics simulation (Isaac Lab + ROS2 + MCP). |
- SelGrad — selective-gradient defense for privacy-preserving ML. Under review, IEEE TDSC.
- Federated learning & differential privacy — robustness and personalization under Non-IID data.
Also known as Wayne Chiu / 邱偉誠 · wayne.is-a.dev · LinkedIn · My PRs · X · Email