Training One‑Step Generative Models Without CFG, DMD, GAN, Drifting, or MeanFlow

Generating images in a single forward pass is attractive, but the main difficulty lies in the training stage: how to provide a stable and simple supervision signal that drives the generated distribution toward the real one.

Existing approaches each have structural trade‑offs: autoregressive, diffusion, and flow‑matching models require sequential or multi‑scale decoding; GANs support one‑step inference but depend on a discriminator and min‑max optimization; consistency models and Distribution Matching Distillation (DMD) need noise‑time supervision, a teacher model, or additional estimators; drifting models rely on batch‑wide pairing, which is unsuitable for “one description → one image” data.

TBSM reframes the problem as “Can we train a small model that directly tells each generated sample where to go?” It introduces a three‑sample scattering event consisting of a real sample, a generated source (with gradients), and a second generated sample (stop‑gradient). The Tracker learns the instantaneous direction from the generated sample toward the real sample while repelling the other generated sample, providing a per‑sample fake‑to‑real movement field.

Formally, the direction is derived from the energy distance between distributions. For a given position, the direction vector is the sum of a unit vector pointing to the real sample and a negatively weighted unit vector pointing to the second generated sample; the two terms correspond to the real‑attract and fake‑self‑interaction components of the energy distance. Under finite first‑order moments, this direction is non‑negative and its expectation yields the conditional mean field that the Tracker aims to learn.

During training, many inexpensive three‑sample events are sampled. The Tracker aggregates these noisy observations to estimate the conditional expectation of the direction at each location. The final direction used to update the generator is a weighted mixture of the instantaneous direction and the Tracker’s output, controlled by two scalar weights (ρ and λ). The generator is then updated by regressing the generated sample to the position obtained after moving along this direction.

Algorithm 1 describes the basic three‑body scattering regression: given a real sample, a generated source, and a stop‑gradient generated sample, compute the direction and use it as a regression target. Algorithm 2 shows the full TBSM training loop, where the weighted instantaneous direction, the Tracker query point (uniformly sampled along the real‑fake line), and the Tracker output are combined to produce the final direction, after which both the generator and Tracker parameters are updated.

Experiments on ImageNet‑256 with the 800 M‑parameter PixelDiT‑XL/16 model (Tracker has < 30 M parameters and is used only during training) achieve FID 1.63 in latent space and 2.23 in pixel space with a single function evaluation (NFE = 1). By contrast, limiting the same architecture to NFE = 1 without TBSM yields FID 398.20, while after TBSM training the same architecture reaches FID 1.92. Ablation studies show that even without the generator‑generator interaction term the model performs well, indicating a GAN‑like behavior of the learned field.

For text‑to‑image generation, a 20 B‑parameter Qwen‑Image model pretrained on large‑scale data is fine‑tuned with TBSM; the resulting model produces high‑quality one‑step samples, as illustrated in the accompanying figures.

The discussion highlights that TBSM provides a sample‑level distribution‑matching interface without an online teacher, allowing mature diffusion models to be post‑trained into one‑step generators. Future work may explore random initialization at ImageNet scale, progressive transition from multi‑step to single‑step generation, and dynamic weighting based on the noise in instantaneous directions.

In conclusion, TBSM extracts the local “real‑attract – fake‑self‑interaction” direction from the energy distance, trains a Tracker to learn its conditional mean, and after training discards the auxiliary network so that the generator alone can generate images with a single forward pass.