What 1000 tokens/s Really Means: Inside Xiaomi MiMo’s UltraSpeed Breakthrough

MiMo‑V2.5‑Pro‑UltraSpeed inference speed

On June 8, Xiaomi released the MiMo‑V2.5‑Pro‑UltraSpeed mode, achieving 1000 tokens per second (TPS) on a trillion‑parameter model, the fastest reported speed for a general‑purpose GPU‑based inference.

Token throughput definition

A token corresponds to roughly one Chinese character or 0.75 English words; 1000 TPS ≈ 750 English words or > 1000 Chinese characters per second.

Comparison with existing models

Typical large models: 50–130 TPS (human‑level reading speed).

Top domestic models: ~400 TPS (fast speech).

UltraSpeed: 1000 TPS, ≈200 × human reading speed, can generate an A4 page in a blink.

Why extreme speed matters

Latency‑critical scenarios—emergency transport, deadline‑driven tasks, AI agents requiring millisecond responses—cannot tolerate the seconds‑level delays of conventional models. In multi‑agent pipelines, each second of latency compounds, making a “run‑car” (1000 TPS) essential.

Capabilities enabled by 1000 TPS

Complex tasks (code generation, web‑page creation, multi‑agent coordination) finish in seconds instead of minutes.

Real‑time applications such as voice dialogue, ad bidding (≤ 100 ms decision window), and customer‑service bots become feasible because model latency is virtually invisible.

Hundreds of AI components can run in parallel without pipeline stalls, supporting large‑scale AI orchestration.

Technical stack achieving the speed

FP4 quantization : Reduces model size and memory bandwidth by quantizing parameters that are insensitive to precision (e.g., MoE expert modules) while preserving accuracy through quantization‑aware training.

DFlash decoding : Performs block‑level speculative decoding—generating a chunk of tokens in parallel with a lightweight decoder, then validates the chunk against the full model. Only correctly predicted blocks are kept, turning per‑token verification into block‑wise verification.

TileRT inference system : A custom GPU kernel and scheduler that keeps the hardware continuously busy by eliminating microsecond‑scale gaps between kernel launches, data movement, and scheduling. Resident kernels and heterogeneous pipelines remove idle periods.

This model‑system co‑design runs on commodity GPUs, avoiding the limited generality of wafer‑scale or SRAM‑based ASIC solutions.

Preservation of model quality

FP4 quantization is applied only to components tolerant of reduced precision; DFlash validates every generated block against the original model, discarding incorrect predictions; TileRT optimizes execution without altering model outputs. Consequently, UltraSpeed retains the same capability as the baseline MiMo model.

Reference illustration