How TDM‑R1 Boosts Few‑Step Image Generation: GenEval Jumps from 61% to 92% and Beats GPT‑4o

Few‑step diffusion models excel in speed and deployment cost but suffer from weak instruction following, unstable text rendering, and poor compositional generation. These limitations stem from the inability to incorporate non‑differentiable, discrete rewards into reinforcement learning.

Core Idea of TDM‑R1

TDM‑R1 separates learning into two independent tracks: an agent reward model that translates vague, non‑differentiable feedback (e.g., correct spelling, accurate counting, user preference) into fine‑grained learning signals, and a few‑step generator that maximizes these signals under a strict 4‑step sampling constraint. The design avoids hard reward back‑propagation and leverages deterministic sampling trajectories to estimate step‑wise rewards precisely.

Key Innovations

Deterministic Trajectory : Fixed sampling paths allow accurate reward estimation for each denoising step, reducing estimation error and accelerating convergence.

Group Preference Optimization (GRPO/DGPO) : Uses a Bradley‑Terry model to assign higher weights to superior sample groups, turning discrete preferences into stable training signals.

Dynamic Reference Model : Employs an EMA version of the agent reward model as a moving reference, preserving stability while adapting to the generator’s distribution.

Experimental Results

On the GenEval benchmark, which tests compositional abilities such as multi‑object counting, positional relations, and attribute binding, TDM‑R1 achieves the following scores:

Baseline 4‑step model (TDM‑SD3.5‑M): 61%

TDM‑R1 (4 steps): 92%

80‑step SD3.5‑M baseline: 63%

GPT‑4o: 84%

Sub‑metric breakdown: single‑target 1.00, dual‑target 0.96, counting 0.88, positional relation 0.93, attribute binding 0.91. The improvement is consistent across all categories, demonstrating genuine enhancement of instruction compliance rather than score hacking.

Additional quality metrics on the DrawBench suite (Aesthetic Score, DeQA, ImageReward, PickScore, UnifiedReward) all improve, e.g., Aesthetic = 5.42 and DeQA = 4.07, exceeding both the 4‑step baseline and the 80‑step model. OCR accuracy rises from 55% to 95%, solving the long‑standing text‑rendering problem.

Generalization to Larger Models

TDM‑R1 is not model‑specific. Applied to the 6B‑parameter Z‑Image model, it yields:

Z‑Image (100 steps): GenEval 0.66, OCR 0.74

Z‑Image‑Turbo (4 steps): GenEval 0.73, OCR 0.78

TDM‑R1‑Z‑Image (4 steps): GenEval 0.77, OCR 0.79

Across multiple quality metrics, TDM‑R1‑Z‑Image outperforms both the 100‑step baseline and the Turbo variant, confirming the framework’s scalability.

Ablation Study

Adding traditional diffusion‑RL loss directly to a few‑step model leads to blurry outputs and unstable performance because the weighted denoising loss conflicts with the reverse‑KL distillation objective required for few‑step training.

Industry Implications

TDM‑R1 demonstrates that few‑step diffusion models can undergo LLM‑style RL after‑training, breaking the belief that speed‑only optimization is the endpoint. Non‑differentiable rewards—such as binary preferences, product‑side feedback, or click data—can now be systematically integrated, paving the way for low‑cost, verifiable tasks to drive broader capability gains and move image models toward general alignment.

Conclusion

By decoupling reward translation from generation and exploiting deterministic trajectories, TDM‑R1 turns non‑differentiable human feedback into effective learning signals, achieving a leap from 61% to 92% on GenEval, surpassing GPT‑4o, and delivering high‑quality, instruction‑following images with minimal inference steps.