VA‑π: Pixel‑Level Alignment Achieves 50% FID Reduction with 25‑Minute Fine‑Tuning

Introduction

VA‑π (Variational Policy Alignment) addresses a long‑standing gap in visual autoregressive (AR) models: tokenizers perfectly reconstruct images, yet AR generators produce distorted outputs with unnatural artifacts. The authors argue that this “elephant in the room” stems from a mismatch between pixel‑level supervision and token‑level likelihood optimization.

Key Contributions

Training efficiency : On 8 × A100 GPUs, fine‑tuning with only 1% of ImageNet data finishes in ~25 minutes.

Quality leap : On LlamaGen‑XXL, FID drops from 14.36 to 7.65 (≈50% reduction) and Inception Score rises from 86.55 to 116.70.

Mathematical elegance : Introduces policy gradients into a variational inference (VI) framework, solving the non‑differentiable reconstruction term and exposure‑bias explosion.

Methodology

VA‑π treats the AR generator as a policy and optimizes a variational evidence lower bound (ELBO) that combines a pixel‑level reconstruction term and a prior regularization term. The reconstruction term forces the generator, under teacher‑forcing, to reproduce the original image, while the prior term preserves the pretrained token distribution.

The ELBO is derived by introducing a variational posterior built from teacher‑forced token predictions. This posterior concentrates on token sequences that can faithfully reconstruct the image, avoiding off‑manifold drift during free‑running sampling.

The ELBO yields two concrete training signals:

Reconstruction term : Provides pixel‑level supervision.

Prior regularization term : Keeps the token distribution close to the pretrained AR model.

1. Solving the “computational explosion” with ELBO

Directly maximizing the pixel‑space likelihood is intractable. By borrowing the VAE trick, VA‑π defines a variational posterior via teacher‑forcing, which eliminates error accumulation and off‑manifold drift.

2. Eliminating exposure bias

The KL term in the ELBO measures the divergence between teacher‑forced and free‑running token distributions. Minimizing this KL directly reduces exposure bias. VA‑π injects contextual noise into the prefix, turning the KL regularizer into a noisy next‑token prediction loss.

3. Overcoming non‑differentiability with RL

The reconstruction term involves a quantizer and discrete sampling, which blocks gradients. VA‑π reframes the AR generator as a policy and maximizes a reconstruction reward (negative reconstruction loss) using reinforcement learning. The reward is computed from the reference image, the true token sequence, and the decoded output.

4. Unified policy optimization (GRPO)

VA‑π combines the RL‑based reconstruction reward and the noisy next‑token regularizer within the Generalized Reinforcement Policy Optimization (GRPO) algorithm, yielding a stable training objective.

Experiments

VA‑π is evaluated on two challenging visual generation tasks: class‑to‑image (C2I) and text‑to‑image (T2I). The base models include LlamaGen‑XL (775 M), LlamaGen‑XXL (1.4 B), and the multimodal Janus‑Pro 1 B.

C2I : Using only 12.8 K ImageNet‑1K samples and 100 fine‑tuning steps, VA‑π achieves a 50% FID drop on LlamaGen‑XXL (14.35 → 7.65) and raises IS from 86.55 to 116.70. On LlamaGen‑XL, IS reaches 299.63, a 7.5× speed‑up over AR‑GRPO.

T2I : Without any text‑alignment or human‑preference reward, VA‑π outperforms AR‑GRPO on most GenEval sub‑tasks, including color understanding, counting, and dual‑objective composition. When inserted into Janus‑Pro 1 B, VA‑π improves the combined GenEval score to 0.725‑0.744.

Ablation Studies

To understand VA‑π’s efficiency, the authors ablate reward components, regularization terms, and noise ratios.

Reward + regularization (Table 4): Using only pixel‑level reward causes collapse (FID ≈ 38.76). Adding the prior regularizer stabilizes training and yields the best FID = 7.65.

Cross‑entropy vs. KL (Fig 4): CE regularization offers superior stability across a wide weight range, eliminating the need for delicate hyper‑parameter tuning.

Contextual noise ratio (Table 5): Moderate noise injection achieves the highest GenEval composite score (0.339); too little or too much noise degrades performance.

Visualization

ImageNet‑1K C2I : Qualitative comparisons show sharper textures and more faithful class semantics.

GenEval T2I : Samples demonstrate improved attribute binding and dual‑objective composition without explicit text‑alignment training.

Conclusion

VA‑π achieves pixel‑level alignment by (1) introducing a pixel‑aware reward that bridges token likelihood and visual fidelity, (2) using teacher‑forced ELBO to avoid computational explosion, and (3) employing a natural ELBO regularizer that preserves the pretrained distribution while guiding the policy toward perfect reconstruction.

These design choices enable a lightweight post‑training procedure that dramatically improves generation quality with minimal compute.

References

[1] Liao, X., He, Q., Xu, K., Qu, X., Li, Y., Wei, W., & Yao, A. (2026). VA‑π: Variational Policy Alignment for Pixel‑Aware Autoregressive Generation. arXiv preprint arXiv:2512.19680.