Can Diffusion LLMs Replace Transformers? Inside Mercury Coder’s Speed Surge

Background

Large language models (LLMs) commonly exhibit hallucinations, limited context windows, outdated knowledge bases, and slow generation, which hampers smooth interaction.

Why Generation Speed Matters

Interactive use requires low latency; a model that generates tokens slowly can make users wait several seconds per response.

Autoregressive vs. Diffusion Models

Most LLMs are autoregressive : they predict the next token sequentially from left to right, requiring a full forward pass for each token and thus incurring high latency.

Diffusion models simulate a forward diffusion process that gradually adds noise to data until it becomes pure random noise, and a learned reverse diffusion that denoises step‑by‑step. Because the reverse steps can be parallelized, diffusion models can theoretically produce all tokens simultaneously, offering much higher throughput.

Diffusion LLM (dLLM) Mechanism

During training a mask predictor p_θ(·|x(t)) learns to predict masked tokens using a cross‑entropy loss computed only on the masked positions. After training, generation starts from pure noise and iteratively applies the learned reverse diffusion to obtain a coherent token sequence.

The recent paper LLaDA (arXiv:2502.09992) demonstrates this approach. LLaDA retains a Transformer backbone but removes causal masking, allowing the model to attend to the entire input sequence when predicting masked tokens, which enables parallel token prediction while preserving Transformer expressiveness.

Mercury Coder Performance

Official benchmarks on an NVIDIA H100 GPU report up to 1000 tokens/second , far surpassing the few‑hundred‑token rates of typical autoregressive models.

In a task that writes LLM inference code, a traditional autoregressive model required 75 iterations, whereas Mercury Coder completed the same task in just 14 iterations, showing a substantial speed advantage.

On the Copilot Arena code‑completion benchmark, Mercury Coder Mini ranked second overall, outperforming GPT‑4o Mini and Gemini‑1.5‑Flash and approaching the performance of the much larger GPT‑4o model.

Access and Usage

Mercury Coder is publicly available at https://chat.inceptionlabs.ai. The model runs without custom hardware acceleration, relying on the diffusion‑based architecture.

Future Outlook

Diffusion LLMs could challenge the dominance of autoregressive Transformers if they continue to improve in three key areas: generation speed (through parallel denoising), diversity (avoiding the monotonicity of left‑to‑right generation), and controllability (producing more precise outputs). A likely evolution is hybrid pipelines that use diffusion models for rapid drafting followed by autoregressive refinement.