Accelerating Diffusion Model Sampling with OLSS: A Linear Subspace Approach

Background

Recent advances in image generation have highlighted the impressive results of diffusion models, which unlike GANs require many forward passes to gradually denoise a Gaussian noise vector into a high‑quality image. This iterative process, while producing superior results, suffers from severe computational inefficiency due to the large number of sampling steps.

Unified Analysis of Acceleration Algorithms

Formally, a diffusion model starts from Gaussian noise

and proceeds through a sequence of sampling steps

to produce the final image. Existing works introduce a “scheduler” that selects a decreasing subsequence of steps and approximates the full generation process. By modeling these schedulers as linear subspace expansions, the paper derives a unified analytical framework that reveals the core of scheduler design.

Algorithm Architecture

Building on this analysis, the authors propose a new scheduler whose iteration coefficients are trainable, allowing a more accurate approximation of the diffusion process. The optimal parameters are obtained by solving a least‑squares problem using QR decomposition, avoiding gradient‑based training due to the small number of parameters. This results in the OLSS (Optimal Linear Subspace Search) scheduler, which searches for the best linear subspace to approximate the full generation trajectory.

Experimental Results

Experiments on Stable Diffusion 1.4 and 2.1 compare eight schedulers, including OLSS and a version without path planning (OLSS‑P), across 5, 10, and 20 sampling steps against the standard 100‑step baseline. FID scores show that OLSS consistently achieves higher image quality at the same step count. Visual examples further illustrate the superior performance of OLSS in extremely low‑step regimes.

References

Bingyan Liu et al., Rapid Diffusion: Building Domain‑Specific Text‑to‑Image Synthesizers with Fast Inference Speed, ACL 2023 (Industry Track).

Chengyu Wang et al., EasyNLP: A Comprehensive and Easy‑to‑use Toolkit for Natural Language Processing, EMNLP 2022 (Demo Track).

Jiaming Song, Chenlin Meng, Stefano Ermon, Denoising Diffusion Implicit Models, ICLR 2020.

Tero Karras et al., Elucidating the Design Space of Diffusion‑Based Generative Models, NeurIPS 2022.

Luping Liu et al., Pseudo Numerical Methods for Diffusion Models on Manifolds, ICLR 2021.

Qinsheng Zhang, Yongxin Chen, Fast Sampling of Diffusion Models with Exponential Integrator, ICLR 2022.

Cheng Lu et al., DPM‑Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps, NeurIPS 2022.

Cheng Lu et al., DPM‑Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models, arXiv 2022.