Can DeepSeek’s DualPath Break GPU Bottlenecks and Ignite an Agentic AI Surge?

DeepSeek announced a fresh research paper (arXiv:2602.21548) that introduces the DualPath inference framework, developed together with teams from Peking University and Tsinghua University, to tackle the severe I/O bottleneck caused by large KV‑Cache loads from external storage during multi‑round agentic inference.

Problem Statement

In agentic scenarios, each token’s “thought trace” is stored in a KV‑Cache that grows with context length. Keeping this cache in GPU HBM is necessary for fast access, but HBM supply cannot keep up with demand, leading to GPU idle time while waiting for data over limited PCIe bandwidth. Measurements show average interaction rounds of 157, average context length of 32.7 k tokens, and a KV‑Cache hit rate of 98.7 %.

DualPath Architecture

DualPath adds a second loading path – “Storage‑to‑Decode” – that separates the KV‑Cache loading from the compute pipeline. It creates two independent pipelines:

Access Path : aggressively moves KV‑Cache blocks from SSD/DRAM to GPU memory.

Compute Path : immediately starts computation on the already‑loaded blocks.

This design implements the classic systems concept of decoupling compute and memory access. By streaming cache chunks (Chunk‑based Streaming), the system can pre‑fetch the next block while the current one is being processed, analogous to buffering the first few seconds of a video while the rest continues to download.

Implementation Details

DualPath stores KV‑Cache chunks in DRAM on the decode GPU server, then transfers them to the prefill GPU via GDRDMA, avoiding the PCIe bottleneck of loading the entire cache at once. The framework defines three stages of the pipeline (Access, Compute, and a shared DRAM buffer) and uses layer‑wise prefill to further reduce latency.

Performance Evaluation

Benchmarks on DeepSeek‑V3.2 (660 B and 27 B) and Qwen 2.5‑32 B show:

Offline (offline‑inference) throughput improvement up to 1.87× .

Online (per‑token latency) improvement up to 1.96× , i.e., roughly a 200 % speedup.

When compared with a basic pipeline, DualPath approaches the theoretical Oracle limit, indicating that KV‑Cache I/O overhead is essentially eliminated.

Performance gains are larger for bigger batch sizes and longer maximum added length (MAL), with the best‑case acceleration reaching 2.46×.

Figure‑based results (omitted here) illustrate that DualPath consistently outperforms the baseline across different P:D (prefill:decode) ratios, with the optimal range from 1:7 to 7:2. Ablation studies attribute the overall 45 % reduction in job‑completion time (JCT) to three components: layer‑wise prefill (45 %), DualPath loading (39 %), and the scheduler (16 %).

Scalability

On a 1,152‑GPU cluster configured as 48 P:96 D, the system supports 48 k concurrent agents, scaling linearly from a 2 P:4 D test with 2 k agents. Similar linear scaling is observed for a 44 P:88 D configuration.

Implications

The study demonstrates that, as LLM parameter counts cease to be the primary limiter, memory bandwidth becomes the new Achilles’ heel. DualPath’s decoupling of compute and storage shows that pure software‑level optimizations can deliver near‑doubling of performance without additional hardware, effectively shifting the AI industry focus from raw compute to efficient I/O handling.

References: https://arxiv.org/pdf/2602.21548