How Baidu’s New MTP Inference Code Doubles DeepSeek‑V3.2 Throughput

Open‑source communities drive AI innovation by pooling global developer expertise.

Recently, Baidu Baige and the SGLang community announced the open‑source release of a production‑tested, high‑performance MTP inference codebase.

This code not only delivers outstanding performance but has also proven exceptional stability and reliability in large‑scale Baidu services.

Benchmarks from the SGLang community show that the code provides more than a two‑fold increase in decoding throughput for the latest DeepSeek‑V3.2 model, enabling direct deployment of production‑grade optimized solutions.

The core contribution is a DSA‑tailored MTP implementation designed specifically for DeepSeek‑V3.2. The DSA architecture renders previous DeepSeek MTP code incompatible, yet it introduces new challenges and opportunities for performance breakthroughs.

MTP reduces the total number of generation steps by predicting multiple future tokens in a single forward pass and validating them together.

Traditional autoregressive decoding generates one token at a time, waiting for each token before proceeding, which is stable but slow.

MTP batch generation predicts several tokens simultaneously and validates them in a unified step, akin to upgrading from typing each character to intelligent predictive input, dramatically cutting generation rounds.

Baidu Intelligent Cloud’s core work implements this efficient MTP solution for the new DSA architecture, allowing SGLang developers to obtain a stable, high‑performance inference capability without repeated low‑level experimentation.

Looking ahead, Baidu Baige’s AI computing platform team will continue to open‑source more production‑level core code to the SGLang community, accelerating large‑model technology innovation and accessibility.