The paper systematically derives scaling laws for mixture-of-experts diffusion language models, trains a 30B-parameter LLaDA MoE v2 model guided by these laws, and demonstrates that it matches or exceeds state-of-the-art autoregressive models while using far less compute.
LLaDA2.2 introduces Levenshtein‑based edit operations, 128K native context, and block routing to turn diffusion language models into reliable agents, achieving competitive scores on 17 benchmarks, up to 1.64× higher throughput than comparable autoregressive models, and demonstrating a new path for agentic AI.
LLaDA2.2 introduces a Levenshtein‑based edit mechanism and the L‑EBPO reinforcement framework, allowing diffusion language models to delete and insert tokens during agent interactions, achieving near‑autoregressive accuracy (53.83 vs 55.74) and 1.64× higher BF16 throughput, plus an 8.6 % gain on SWE‑bench.
The article analyzes the trade‑off in masked diffusion language models where larger generation blocks increase parallelism but degrade reasoning, and shows how the T* progressive block‑scaling method using trajectory‑aware reinforcement learning stabilizes training and boosts accuracy across block sizes, with up to 15 % gains on MATH500.
OmniVoice introduces a single‑stage, diffusion‑style language model that maps text directly to multi‑codebook acoustic tokens, achieving zero‑shot voice cloning for over 600 languages with high intelligibility and real‑time factor as low as 0.025, making it suitable for large‑scale multilingual deployment.
The Tencent WeChat AI team introduces WeDLM, a diffusion language model that, through topological reordering, surpasses autoregressive models on the industrial‑grade vLLM engine with over threefold speedup on math reasoning and up to tenfold in low‑entropy scenarios, and provides a step‑by‑step online tutorial with GPU compute credits.
Tencent WeChat AI introduces WeDLM, a diffusion language model that works with standard causal attention and KV caching, achieving up to ten‑fold speedups over autoregressive models while maintaining or improving generation quality across math reasoning and open‑ended tasks.
The SPG algorithm introduces a sandwiched policy gradient that uses computable lower and upper evidence bounds to align reinforcement learning for discrete diffusion language models, achieving faster convergence, higher peaks, and lower variance on four major reasoning benchmarks.
Ant Group's open‑source dInfer framework dramatically speeds up diffusion language model inference—achieving more than a ten‑fold boost over Fast‑dLLM, surpassing autoregressive baselines, and delivering 1011 tokens per second on HumanEval—by tackling computational cost, KV‑cache invalidation, and parallel decoding challenges through modular system‑level innovations.
Ant Group and Renmin University unveiled LLaDA‑MoE, the industry’s first native MoE‑based diffusion language model trained on 20 TB of data, achieving performance comparable to Qwen2.5 while delivering several‑fold faster inference, and the model will be fully open‑sourced to accelerate global AI research.
