Baidu Intelligent Cloud Tech Hub

Oct 14, 2026 · Artificial Intelligence

Baidu's KVCache Overhaul: 36% TTFT Cut via Hit-Rate & Multi-Chip Tuning

Baidu engineers detail KVCache optimizations for LLM inference: a three-layer cache-aware scheduler boosting hit rates, prefetch/interrupt mechanisms, Rolling Hash offload, Radix Tree C++ rewrite, UVA copies, MLA-specific read/write cuts, plus Kunlun XPU L2 layout tuning, DMA batching, bidirectional async, huge-page pinning, NUMA affinity, PCIe relaxed ordering, Ghost List for selective write-through, and TP-group shared buffers yielding 36% TTFT reduction.