Fast Generation, Weak Intelligence? The Harsh Reality of Diffusion Models for Agents

Autoregressive language models have demonstrated strong performance in agentic workflows but are limited by high inference cost. Diffusion language models promise faster parallel decoding, yet a comprehensive evaluation by Tao et al. (NTU, Southeast University, Alibaba) reveals systematic capability gaps.

In embodied agent tasks (AlfWorld, ScienceWorld, BabyAI), diffusion models achieve significantly lower success and progress rates than autoregressive counterparts, often failing to produce any correct examples. Analysis shows they lack causal reasoning and repeatedly enter retry loops.

In tool‑calling tasks evaluated with the Berkeley Function‑Calling benchmark (BFCL v3), diffusion models lag behind in both single‑turn and multi‑turn scenarios, especially struggling with multi‑turn workflows. Their outputs are frequently ill‑formatted or semantically ambiguous, making strict JSON‑based calls unreliable.

The study attributes these failures to the parallel decoding mechanism, which sacrifices causal coherence and precise formatting for throughput. While this mechanism boosts raw generation speed, it impairs long‑chain reasoning and accurate output structuring.

To probe the true potential of diffusion models, the authors introduce DiffuAgent, a modular evaluation framework that isolates agentic sub‑tasks: memory, self‑verification, tool selection, and format correction. Experiments show diffusion models perform comparably or better than autoregressive models in static memory and self‑verification modules, but remain weak in format‑correction for tool‑calling.

Key findings indicate diffusion models excel at static information extraction but falter on dynamic, causally‑linked reasoning. The authors recommend strengthening causal and structural training data, adopting adaptive hybrid decoding (autoregressive for critical reasoning steps, parallel for static generation), and developing agent‑centric benchmarks beyond MMLU/GSM8K.