How VA‑π Bridges Tokenizers and Autoregressive Generators for Pixel‑Perfect Images

Why Tokenizers Fail When Paired With Autoregressive Generators

Current visual AR models treat the tokenizer and generator as a perfect black box, but generated images often exhibit structural distortions and unnatural artifacts because the tokenizer learns perfect pixel reconstruction while the AR generator only optimizes token likelihood in a discrete space, leading to off‑manifold token sequences.

Key Contributions of VA‑π

Training Efficiency : Eliminates costly RLHF clusters; fine‑tunes on 1% of ImageNet using 8 A100 GPUs in about 25 minutes.

Quality Leap : On LlamaGen‑XXL, FID drops from 14.36 to 7.65 and Inception Score rises from 86.55 to 116.70.

Mathematical Elegance : Introduces policy gradients within a variational inference framework to solve the non‑differentiable pixel‑level feedback and exposure‑bias problems.

Methodology Deep Dive

1. Solving the "Computation Explosion" with an ELBO

The goal is to maximize the true image likelihood in pixel space, which is intractable in token space. VA‑π adopts a VAE‑style variational posterior built via teacher‑forcing, ensuring the posterior concentrates on token sequences that faithfully reconstruct the image. This yields the following evidence lower bound (ELBO) objective:

The ELBO provides two training signals:

Reconstruction Term : Supplies pixel‑level supervision, forcing the AR model under teacher‑forcing to generate sequences that can be decoded back to the original image.

Prior Regularization Term : Constrains the token distribution to preserve the pretrained AR model’s language‑modeling ability.

2. Eliminating Exposure Bias with Noisy Next‑Token Prediction

The KL regularizer in the ELBO measures the divergence between the teacher‑forced posterior and the model’s free‑running distribution. Minimizing this KL directly reduces exposure bias. VA‑π injects contextual noise into the real prefix, turning the regularizer into a simple noisy next‑token loss:

This forces the model to generate high‑quality tokens even under perturbed contexts, greatly improving inference robustness.

3. Breaking the Non‑Differentiability Barrier with RL

Although the ELBO defines a direction, the reconstruction term involves a quantizer and teacher‑forced sampling, which block gradients. VA‑π reframes the AR generator as a policy and maximizes a reconstruction reward (negative reconstruction loss). Given a reference image, its true token sequence, and the decoded output, the reward is defined as:

VA‑π uses the same noisy token sequence during training, so maximizing this reward directly aligns token generation with pixel‑level fidelity.

4. Final Fusion: Policy Optimization with GRPO

Combining the reconstruction reward (Eq. 10) and the noisy next‑token regularization (Eq. 9) fits naturally into the GRPO algorithm, which jointly optimizes the policy and a KL penalty. The final objective is:

Experimental Validation

VA‑π was evaluated on two challenging visual generation tasks: class‑conditional image generation (C2I) and text‑conditional image generation (T2I). The base models included LlamaGen‑XL (775 M), LlamaGen‑XXL (1.4 B), and the multimodal Janus‑Pro 1 B.

1. C2I Results – 25‑Minute Fine‑Tuning Cuts FID by ~50 %

On the ImageNet‑1K validation set (50 k images), VA‑π outperformed AR‑GRPO and STE baselines. For LlamaGen‑XXL, FID dropped from 14.35 to 7.65 and IS increased by 30.16 points. For LlamaGen‑XL with CFG = 2.0, VA‑π achieved an IS of 299.63, surpassing AR‑GRPO while training 7.5× faster (≈20 min).

2. T2I Results – No External Reward Needed

Even without any text‑alignment or human‑preference reward, VA‑π excelled on the GenEval benchmark. On LlamaGen‑XL, it beat AR‑GRPO on most sub‑tasks, especially “color understanding”, “counting”, and “dual‑object composition”. When inserted into Janus‑Pro 1 B, VA‑π raised the combined GenEval score to 0.725 / 0.744, showing strong generalization to multimodal generation.

3. Ablation Studies – Proving Each Component Matters

Reward + Regularization Required (Table 4) : Using only the pixel‑level reconstruction reward causes FID to explode to 38.76; adding the prior regularization stabilizes the token distribution and brings FID down to 7.65.

Cross‑Entropy Regularization Beats KL (Fig 4) : CE regularization yields more stable training across a wide weight range, eliminating the need for delicate hyper‑parameter tuning.

Optimal Noise Ratio (Table 5) : Moderate contextual noise gives the highest GenEval composite score (0.339); too little or too much noise degrades performance.

Result Visualization

Class‑Conditional Generation (ImageNet‑1K) : Samples generated with CFG = 1.0, temperature = 1.0, top‑k = 0, top‑p = 1.0 show clear improvements in fidelity and diversity.

Text‑Conditional Generation (GenEval) : Samples with CFG = 5.0 demonstrate superior attribute binding and dual‑object composition.

Takeaway

Pixel‑level rewards directly connect token probability optimization with physical visual quality, eliminating the hidden misalignment between tokenizer and generator.

ELBO‑based teacher‑forcing reduces multi‑step online search to a single forward pass, avoiding computation explosion.

The built‑in regularization preserves the original AR distribution while pursuing pixel‑level fidelity, preventing manifold drift.

References

[1] VA‑π: Variational Policy Alignment for Pixel‑Aware Autoregressive Generation.