Lafa: Block-Level Asynchronous Flow LLMs Achieve 2.5x Throughput Over Autoregressive Models

Today's large language models are almost exclusively autoregressive (AR), generating one token per forward pass, which inherently limits latency. To enable parallel generation, the industry has pursued three paths: multi-token prediction (e.g., DeepSeek's MTP), discrete diffusion models (e.g., Dream, LLaDA), and continuous flow models that evolve from noise to text in embedding space. Continuous flow models can borrow mature acceleration and control techniques from image and video generation, but prior work only validated them on small models and simple tasks; no one had scaled them to billion-parameter levels.

Lafa (Large Flow Language Models) from UC Berkeley, Amazon, and CMU fills this gap. It directly adapts pretrained AR models (Qwen2.5, Qwen3) into block denoisers, training to 8B scale with under 30B tokens. At matched quality, Lafa achieves 2.02–2.54x the throughput of AR baselines, outperforming all MTP and diffusion baselines.

Core Design: Block-Level Asynchronous Flow

Lafa's architecture can be summarized as: autoregressive across blocks, bidirectional asynchronous denoising within blocks.

Text is split into blocks; blocks maintain causal order, allowing prefix KV cache reuse.

Each token inside a block has an independent noise time τ; left positions denoise first, naturally aligning with left-to-right text directionality.

A single network performs two tasks simultaneously: AR prediction on clean prefixes and denoising on noised blocks.

This design enables three inference modes: multi-step flow decoding for quality, a distilled flow map for single-step generation, and self-speculative decoding that uses the model's own AR head to verify flow drafts, pushing speed to the limit.

Low-Cost Adaptation from Pretrained AR Models

Lafa does not train from scratch. It converts Qwen2.5/Qwen3 into block denoisers with four key modifications:

Time conditioning: AdaLN injects per-position noise times; prefix positions are fixed at τ=1, initialized as identity mapping to preserve pretrained weights.

Attention mask: Bidirectional within blocks, causal for prefixes. Prefix keys/values are decoupled from block state, enabling caching across denoising steps and across blocks.

Embedding constraint: Word vectors keep fixed L2 norm, learning only direction, preventing embedding explosion that would trivialize denoising.

Joint loss: AR cross-entropy, denoising cross-entropy, and embedding MSE are optimized together.

Training is fully parallelized: a clean sequence and a block-wise noised copy are processed in one forward pass, computing all blocks' AR and denoising losses simultaneously.

Crucially, tokenwise times are not hand-designed but learned online via per-position denoising loss. The model spontaneously learns a left-first, right-later denoising order that matches text directionality (Figure 5).

Experiment 1: Quality First

Flow decoding accuracy rises steadily with integration steps; at block size 4, 0.5B matches and 7B surpasses AR SFT.

Flow map distillation markedly improves 1–2 step generation quality; 7B two-step student approaches four-step teacher.

Larger blocks increase parallelism but slightly reduce quality at fixed steps, a classic quality-speed trade-off.

Surprisingly, single-token decoding (Table 1) shows Lafa exceeds AR SFT at all three scales. Flow training acts as implicit multi-token prediction, strengthening AR capability — the foundation for subsequent acceleration.

Experiment 2: Self-Speculative Decoding Maximizes Speed

Since one model houses both flow and AR heads, the flow head drafts a block in one step, then the AR head verifies left-to-right, accepting the longest consistent prefix. Output matches greedy AR exactly, but much faster.

Each forward pass commits 2.5–5 tokens on average; larger blocks yield higher parallelism.

At 7B with block size 16, single H100 throughput reaches 2.54x AR SFT with near-identical accuracy.

Compared to DeepSeek-MTP and MARS (max 1.4x speedup) and diffusion baselines (slow with quality drop), Lafa leads decisively (Figure 8, Table 2).

Ablation Study

Ablations confirm every design contributes: block bidirectional attention beats full causal by 5.3 points; asynchronous time outperforms synchronous; norm freezing beats freezing or unconstrained embeddings; all three loss terms are essential; self-conditioning and prefix guidance each add ~4 points (Table 3).

Conclusion

Flow models are no longer toys but a scalable language modeling paradigm. Future work should better integrate self-conditioning and prefix guidance into flow map distillation to further close the few-step quality gap.

https://arxiv.org/abs/2610.28559
github.com/lafa-gh/large-flow-lms
Large Flow Language Models