Can Generative Models Be Trained End‑to‑End? The Secret Is a Simple For Loop

Why Generative Models Lag Behind End‑to‑End Training

Since AlexNet proved that end‑to‑end learning beats handcrafted pipelines, most deep‑learning tasks have become fully end‑to‑end, except generative models. Current autoregressive and diffusion models predict a tiny step during training but must unroll that step hundreds of times at inference, causing a mismatch between training and sampling distributions.

This mismatch creates exposure bias : errors from one step feed into the next, pushing the input away from the training distribution and accumulating errors. Because generative targets contain many valid modes, reconstruction losses (e.g., L2) push the model toward the average of those modes, producing mode blurring —a blurry image or repetitive text.

Explorative Modeling (XM): A One‑For‑Loop Solution

A recent UIUC‑Harvard paper proposes Explorative Modeling (XM). Instead of splitting the generation process, the authors split the training loop: at each step the model generates K candidates and back‑propagates only on the one closest to the real data. The authors implement this with a 3‑to‑5‑line for loop (Algorithm 1).

By allowing the model to explore K possibilities, each candidate can “claim” a different mode, eliminating the need for averaging and thus solving mode blurring.

Forward and Reverse Exploration

The paper defines two directions:

Forward : fix a real target and search the model’s outputs for the nearest candidate (high recall, covers many modes).

Reverse : fix a generated sample and search the real data for the nearest target (high precision, low compute, may collapse to few modes).

Both can be combined for complementary benefits.

Scaling the Third Axis

Experiments on image, video, and language tasks show monotonic performance gains as the exploration factor K grows. Reported gains include:

Data‑scale: +7 % → +36 % improvement.

Model‑scale: +13 % → +23 % improvement.

Compute‑scale: tripling FLOPs more than doubles efficiency.

FLOP efficiency ↑ 4.1×, sample efficiency ↑ 6.2×, parameter efficiency ↑ 47 %.

On ImageNet, XM pushes the best RAE recipe to an unconditional FID of 1.43, rivaling the state‑of‑the‑art.

End‑to‑End Applications

When applied as a standalone end‑to‑end model, XM excels in robot control tasks. In behavior cloning, the Explorative Policy matches or exceeds a Diffusion Policy that requires 100 forward passes with only a single forward pass. In goal‑directed world modeling, Explorative World Model achieves better average performance with 16‑to‑256× less inference compute than Diffuser.

Limitations and Future Work

The authors acknowledge that best‑of‑K is not new; their contribution is clarifying that a simple loop can amplify generative expressivity without breaking the generation process. Autoregressive language models remain challenging for pure end‑to‑end XM due to the high number of modes, leaving room for future research.

They argue that as models and data grow beyond being bottlenecks, generative expressivity—this third scaling knob—will become the dominant factor.