How PRISM Enables Efficient Test‑Time Scaling for Discrete Diffusion Language Models

Test‑time scaling challenge for discrete diffusion language models

Best‑of‑N and Self‑Consistency improve reasoning for autoregressive LLMs but assume left‑to‑right generation. Discrete diffusion language models (dLLMs) generate answers by iteratively denoising a masked sequence, providing global context but making traditional search and reward‑based methods inefficient.

PRISM framework

PRISM (Pruning, Remasking, and Integrated Self‑verification Method) introduces a hierarchical trajectory search (HTS) that splits inference into three phases:

Early random exploration with a wide candidate set to preserve diversity under high noise.

Mid‑stage progressive pruning: when a “logical skeleton” emerges, a self‑verification signal ranks trajectories and discards low‑quality ones, reallocating compute to promising candidates.

Late‑stage refinement: only a small number K of trajectories are kept for fine‑grained denoising.

Complexity drops from O(N T) for naïve Best‑of‑N (N trajectories, T denoising steps) to approximately O(N + K T) with K ≪ N.

Local Branching via Partial Remasking

During the mid‑stage, high‑confidence tokens form a stable answer backbone. PRISM selectively re‑masks low‑confidence tokens and lets the model generate new branches only in those regions, preserving the backbone while exploring alternative details.

Self‑Verified Feedback (SVF)

PRISM reuses the same dLLM as a binary verifier. After generating a candidate answer, a Yes/No verification prompt is issued; the model’s Yes/No logits are normalized to a score that ranks and prunes trajectories. SVF consumes less than 10 % of the total denoising function‑evaluation budget.

Experimental results

Benchmarks: GSM8K, MATH‑500 (math reasoning), HumanEval, MBPP (code generation). Models: LLaDA‑8B‑Instruct, Dream‑7B‑Instruct, LLaDA‑2.0‑mini.

LLaDA‑8B‑Instruct, K = 8: GSM8K accuracy ↑ from 67.58 % to 85.30 %; MATH‑500 ↑ from 26.40 % to 42.80 %.

Code tasks: HumanEval ↑ 24.39 points, MBPP ↑ 16.40 points.

PRISM reaches 85.30 % on GSM8K with 1,048 NFE, while Best‑of‑16 needs 4,096 NFE for comparable performance, a >4× reduction.

Speed‑up at comparable accuracy: GSM8K ≈ 2.9×, MATH‑500 ≈ 6.5×, HumanEval ≈ 1.8×, MBPP ≈ 1.7×.

TruthfulQA ROUGE‑1/2/L: 31.8/35.5/31.9 with 1,048 s inference; LLaDA‑ReMDM scores 29.5/31.8/29.5 with 1,354.8 s.

External verifier (Qwen3‑8B) yields 87.35 % on GSM8K but requires loading an additional 8 B model (total 16 B parameters). SVF achieves 85.30 % using only the original 8 B dLLM.

Implications

PRISM demonstrates that test‑time scaling for non‑autoregressive models can be achieved by integrating search, pruning, local branching, and self‑verification into the denoising dynamics, allocating compute where the answer structure forms and exploring alternatives only in low‑confidence regions, without extra models.

Paper: Prism: Efficient Test‑Time Scaling via Hierarchical Search and Self‑Verification for Discrete Diffusion Language Models , arXiv:2602.01842.

Code repository: https://github.com/viiika/Prism

Code example

此前，团队曾提出 Meissonic [1]，探索 masked generative transformer 在高分辨率文本到图像生成中的潜力；随后进一步提出 Muddit [2]，将离散扩散建模从图像生成推进到更统一的多模态生成框架。此次入选 ICML 2026 的 PRISM，则将这一研究脉络进一步延伸到推理阶段，关注如何通过层次化搜索、自验证反馈和局部 remasking，让离散扩散模型在无需额外 verifier 的情况下实现高效 Test-Time Scaling。
[1] Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis, ICLR 2025, https://arxiv.org/abs/2410.08261
[2] Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model, ICLR 2026, https://arxiv.org/abs/2505.23606