LLaDA2.2 Released: Levenshtein Editing Enables Diffusion Language Models to Correct On-the-Fly

In the landscape of large language models, autoregressive (AR) architectures have long dominated while diffusion models have been viewed as a challenging, non‑consensus path. LLaDA2.2 marks a breakthrough by adding a Levenshtein‑style edit mechanism and environment‑feedback reinforcement learning, giving diffusion language models (dLLM) the ability to correct while acting.

Levenshtein Editing Mechanism

The core innovation is a Levenshtein‑style edit that provides true structural freedom: the model can keep, substitute, DELETE redundant or erroneous tokens, and INSERT missing tokens by creating a masked slot for later denoising. This expands the edit action space beyond the "replace‑only" approach of earlier models such as LLaDA2.1.

During training, the longest common subsequence (LCS) aligns drafts with answers, generating four edit commands. Under a fixed‑block length constraint, DELETE removes tokens and shifts subsequent content left, while INSERT creates a [MASK] slot that is later filled, preserving block length through padding or truncation.

L‑EBPO: Trajectory‑Level RL for Editable dLLM

LLaDA2.2 extends the earlier EBPO (Edit‑Based Policy Optimization) by adding a two‑layer control structure. The outer layer optimizes trajectory‑level decisions across multiple Agent interactions, while the inner layer handles per‑block INSERT and DELETE actions. Both layers share a unified action space V ∪ {DELETE, INSERT}, allowing token prediction and structural edits to be optimized together.

During each diffusion step, the system aligns noisy sampled rollouts with the original action sequence, restoring instantaneous edit labels so gradients can flow through insertions and deletions. The environment reward comprises three parts: correct tool‑call execution, valid output format, and overall task completion.

On the SWE‑bench Verified benchmark, using the same base model and trajectory fine‑tuning but toggling Levenshtein editing raised the solve rate from 35.8 % to 44.4 %, an absolute gain of 8.6 percentage points.

128K Context and MoE Block Routing

Agent tasks often require ingesting massive codebases and dozens of interaction turns. LLaDA2.2 starts from an 8K‑token base, expands to 64K with 300 B tokens, and finally reaches native 128K context after an additional 200 B tokens. This context is built directly into the model rather than being an inference‑time extrapolation.

To handle the engineering challenges of long context in block diffusion models, LLaDA2.2 introduces a Block Routing mechanism consisting of:

Token Racing: each token scores all experts; the highest score per expert across the block forms the block‑level admission score.

Fixed Capacity Limit: from 256 global routing experts, the top‑48 by block score are selected as the block’s candidate pool.

Second‑Stage Fine Dispatch: each token then performs a top‑k routing within this candidate pool, preserving token‑level specialization.

This design yields a predictable O(C) upper bound on expert work per block and avoids flooding all tokens into a single expert set. Reported results show negligible quality loss after switching to block routing, while MoE block diffusion becomes more stable and easier to plan.

Agent Benchmark Results

Overall performance close to AR models: LLaDA2.2‑flash scores 53.83 average, only 1.91 points behind Ling‑2.6‑flash (55.74).

Specific scenarios surpass AR: on τ²‑Bench, PinchBench, and MCP‑Atlas, LLaDA2.2‑flash exceeds Ling‑2.6‑flash; on Claw‑Eval the gap is minimal.

Throughput advantage: BF16 average decode throughput is 1.64× that of Ling‑2.6‑flash; after quantizing to FP8, throughput improves an additional 18.6 %.

Conclusion

LLaDA2.2 demonstrates that diffusion language models can match autoregressive accuracy on complex Agent benchmarks while delivering superior efficiency. The Levenshtein edit mechanism turns dLLM from a "draft‑only" editor into a true structural editor capable of deletions and insertions. The L‑EBPO framework teaches the model to act, receive feedback, and correct in a closed loop. Native 128K context and block routing remove major engineering bottlenecks for large‑scale Agent deployment.

The community is invited to try LLaDA2.2 and explore the possibilities of diffusion models in the emerging Agent era.