How SADA Boosts Diffusion Model Sampling Speed by Up to 1.8× Without Losing Quality

Background

Diffusion models achieve high‑quality generative results but require many iterative denoising steps and quadratic‑complexity attention, leading to large inference costs. Existing training‑free acceleration methods reduce per‑step computation but often cause noticeable fidelity loss.

Problem Analysis

The fidelity gap stems from two observations:

Different text prompts follow distinct denoising trajectories, so a uniform sparsity pattern cannot preserve all details.

Most acceleration techniques ignore the underlying ordinary differential equation (ODE) that governs the diffusion process and the numerical properties of its solvers.

Proposed Method: SADA (Stability‑guided Adaptive Diffusion Acceleration)

SADA introduces a unified stability criterion that guides the allocation of sparsity both temporally (per sampling step) and spatially (per token). The method consists of two complementary components.

1. Stability‑guided adaptive sparsity

For each sampling step, SADA evaluates a stability metric (e.g., the norm of the predicted noise residual) on the current trajectory. Tokens with higher instability receive a denser computation pattern, while stable tokens are processed with increased sparsity (e.g., reduced attention heads or lower‑resolution feature maps). This dynamic allocation adapts to the specific prompt‑dependent trajectory, preserving details where needed.

2. Approximate gradient from the ODE solver

SADA leverages the numerical ODE solver (such as EDM or DPM++) to compute an approximate gradient of the diffusion ODE without extra forward passes. By differentiating the solver’s update rule, SADA obtains a cheap estimate of the true gradient, which is used to correct the sparsified update and maintain the theoretical stability of the underlying ODE integration.

Experimental Setup

Evaluations were performed on three widely used diffusion backbones:

Stable Diffusion‑2 (SD‑2)

Stable Diffusion‑XL (SDXL)

Flux

Each model was paired with two ODE solvers:

EDM (Euler‑based Diffusion Model)

DPM++ (higher‑order solver)

Speedup was measured as the ratio of original to accelerated wall‑clock time. Fidelity was quantified with LPIPS (lower is better) and FID (lower is better).

Results

SADA consistently achieved at least 1.8× faster sampling across all model‑solver combinations while keeping fidelity loss negligible (LPIPS ≤ 0.10, FID ≤ 4.5). Additional findings include:

ControlNet could be accelerated without any architectural changes, demonstrating the method’s plug‑and‑play nature.

MusicLDM, a diffusion model for audio spectrogram generation, received a 1.8× speedup with spectrogram LPIPS ≈ 0.01, indicating high‑quality audio preservation.

Conclusion

SADA provides a principled, stability‑driven framework for adaptive sparsity in ODE‑based generative models. By coupling trajectory‑aware sparsity with an ODE‑solver‑derived gradient approximation, it delivers substantial inference acceleration without compromising visual or audio fidelity.

Resources

Project code: https://github.com/Ting-Justin-Jiang/sadaicml

Code example

来源：专知
本文
约1000字
，建议阅读
5
分钟
本文提出了一种新颖的加速范式 SADA，该方法通过统一的稳定性准则，在每一步与每个token粒度上动态分配稀疏性，实现对基于ODE的生成模型（如扩散模型与流匹配模型）的高效采样加速。