How Xiaomi’s MiMo‑V2.5‑Pro UltraSpeed Achieves 1000 TPS on a 1‑Trillion‑Parameter Model

The MiMo‑V2.5‑Pro UltraSpeed mode pushes the generation speed of a 1‑trillion‑parameter (1T) model beyond 1000 tokens / s, a milestone that reshapes the practical limits of AI applications.

Why Speed Matters

When inference is fast enough, AI moves from a passive “wait‑for‑answer” tool to an active extension of human thought, enabling real‑time response, instant iteration, and seamless collaboration. The authors argue that 1000 TPS transforms AI agents, coding assistants, and time‑critical scenarios such as high‑frequency trading, fraud interception, and medical decision‑making.

Extreme Model‑System Co‑Design

Achieving this speed is not the result of a single trick but of a deep co‑design between the MiMo model and the TileRT inference system. While many competitors pursue custom ASICs (e.g., Cerebras wafer‑scale chips or Groq SRAM‑only designs), the MiMo team chose to stay on general‑purpose GPUs and optimize both model and system together.

FP4 Quantization for MoE Experts

The model applies FP4 (MXFP4[1]) quantization only to the Mixture‑of‑Experts (MoE) experts, which hold the majority of parameters and tolerate lower precision. Other components retain their original precision. This selective FP4 QAT reduces model size and memory bandwidth without degrading overall capability.

DFlash Block‑Masked Speculative Decoding

Traditional speculative decoding relies on a small draft model that predicts one token at a time, limiting throughput. DFlash replaces the draft with a block‑masked parallel predictor[2], allowing a whole mask region to be filled in a single forward pass. The draft model uses Sliding Window Attention (SWA) aligned with MiMo‑V2’s architecture, making the compute cost constant with respect to context length.

Mask signal sampling is performed on‑GPU, generating tens of thousands of training signals per step and avoiding cross‑device communication.

Mask block size is limited to 8 to keep verification overhead low while maximizing acceptance length.

In coding‑heavy workloads the average acceptance length reaches 6.30 (peak 7.14), meaning 6–7 of the 8 draft tokens are accepted per verification round, dramatically boosting throughput.

TileRT Ultra‑Low‑Latency Kernel

TileRT introduces a persistent engine kernel that eliminates per‑operator launch overhead, keeping the entire compute pipeline resident on the GPU. Warp specialization further splits communication, data movement, and tensor computation into dedicated warp groups, turning the GPU into a continuously flowing heterogeneous execution system.

These innovations compress the lifecycle of individual operators to the microsecond scale, removing the “execution gap” that traditionally throttles high‑throughput inference.

Results and Availability

The combined FP4‑quantized MoE model and TileRT system deliver >1000 tokens / s on a standard 8‑GPU node. The UltraSpeed API is offered at three times the price of the regular MiMo‑V2.5‑Pro API but promises roughly ten‑fold higher output speed. Access is limited to a trial period (June 9 – June 23 2026) via an application form.

Open‑Source Release and Outlook

The FP4‑quantized checkpoint and DFlash‑enhanced model have been open‑sourced on HuggingFace (https://huggingface.co/XiaomiMiMo/MiMo‑V2.5‑Pro‑FP4‑DFlash). Future work will extend ultra‑fast inference to the full MiMo‑V2.5 series.

References:

MXFP4 specification: https://www.opencompute.org/documents/ocp‑microscaling‑formats‑mx‑v1‑0‑spec‑final‑pdf

DFlash paper: https://arxiv.org/abs/2602.06036