How Diffusion Language Models Deliver 5× Faster Agent Inference Than Token‑by‑Token Generation

Most mainstream large models still expand along the autoregressive, token‑by‑token route, but diffusion language models (dLLM) are emerging as a competing paradigm that could replace GPT‑style models within two years, with companies such as Google, Inception and Ant Group actively pursuing the approach.

On September 1, a joint test by Diffusion Intelligence’s DiffuSpace and the Asian Agent Computing Platform (Acrab) demonstrated that, in edge scenarios, dLLM can accelerate an agent’s runtime by roughly fivefold. Both parties plan to push the technology into AI PCs, smart cars, robots and smart‑home applications.

dLLM differs from GPT‑style models by borrowing the diffusion mechanism from image generation: it first creates a global content framework and then iteratively refines the result with contextual information. This “global generation” yields higher consistency and can achieve 5–10× inference acceleration.

On edge devices, the speed advantage is amplified because traditional autoregressive models generate only one token per step, under‑utilizing parallel GPU compute. dLLM, by processing multiple positions simultaneously, converts the chip’s parallelism into actual latency reduction.

Agent workflows typically involve planning, retrieval, tool invocation, execution and verification, with the model called repeatedly. Each generation latency accumulates along the task chain, so a fivefold speedup can markedly shorten the total time required. For example, an AI PC agent that reads email, organizes a schedule, retrieves files and drafts a plan can run these steps in parallel with a diffusion model.

Interview – Why edge is a prime scenario? DiffuSpace explains that edge inference usually serves a single local user who is highly sensitive to response time, making the parallel token generation of dLLM especially valuable. The speed gain comes from generating several tokens at once, akin to how humans form ideas before spelling them out.

Interview – How does speed translate to user experience? In a coding agent, a user edit forces a GPT‑style model to regenerate from the edit point onward, discarding previously generated content. dLLM can adapt the downstream output, preserving unchanged parts and delivering a more editable, flexible, and intelligent interaction.

Looking further ahead, diffusion models may raise the ceiling of artificial general intelligence by excelling at global search and diverse candidate generation, which benefits reasoning‑search‑inference (RSI) tasks. Training dLLM can be more demanding—requiring exposure to more noise and position combinations—but the one‑time training cost is offset by long‑term inference efficiency.

Current challenges include the lack of mature operators, inference systems and infrastructure compared with the well‑established autoregressive stack (e.g., FlashAttention). However, the growing number of open‑source projects and community contributions are gradually closing this gap.

DiffuSpace predicts that within the next one to two years dLLM could replace GPT‑style models, with a new larger diffusion model slated for release later this year and plans to scale parameters by an order of magnitude in the following year.