GRC: Self-Diagnosis and Correction for Generative Recommendation (CIKM'26 Oral)

Background: Generative Recommendation and Its Limitations

Generative recommendation (GR) encodes items into discrete semantic IDs (SIDs) via RQ-VAE and reformulates recommendation as an autoregressive sequence generation task. An encoder-decoder Transformer conditions on the user's interaction history to generate the target item's SID token by token, using beam search at inference. This paradigm avoids massive item embedding tables and allows semantically similar items to share tokens.

However, standard one-pass decoding suffers from error accumulation: once an early token deviates into a wrong semantic branch, subsequent tokens can only extend that erroneous branch. Teacher forcing trains only on ground-truth prefixes, so the model never learns to detect or correct its own mistakes. Recent reasoning-augmented paradigms—reason-then-generate and reason-while-generating—improve quality but still reason before a complete draft exists, preventing global diagnosis of where and at which semantic level the generation went wrong.

Challenges

(C1) No explicit trajectory correction mechanism. Existing GR models lack the ability to diagnose and fix errors after a full draft is produced.

(C2) Mismatch with free-form natural language reflection. Free-text reflections cannot align with discrete SIDs, guide token-level edits, or meet strict latency budgets in large-scale real-time recommendation.

(C3) Expanded reflection-correction space. Adding explicit reflection and correction vastly increases possible decoding behaviors; limited supervised trajectories cannot cover this space, requiring reinforcement learning for robust policy exploration.

GRC: Generation–Reflection–Correction Paradigm

GRC replaces single-pass decoding with a three-stage process within a single trajectory: generate a complete draft, reflect on it with structured labels, then correct conditioned on both draft and reflection. Three components address the challenges:

2.1 Structured Reflection-Correction Template (SFT)

Instead of free text, reflection is formalized as fixed-length, discrete labels that map directly to retrieval actions. For each training sample, the pretrained backbone produces multiple candidates via beam search. Two labels are annotated:

Token-level error location label : records the first position where a candidate diverges from the target SID (Eq. 3). A value of L+1 indicates a fully correct sequence.

Semantic-level consistency label : judges whether the generated item matches the target on key attributes (e.g., category, brand) by mapping items to attribute buckets (Eq. 4). This dimension is extensible by adding new attribute judgment functions.

The initial draft, reflection labels, and corrected sequence are concatenated into one supervised sequence separated by special tokens (Eq. 5). A weighted loss jointly trains generation, reflection, and correction in a single pass.

2.2 GRPO Reinforcement Learning for Self-Correction Policy

SFT relies on offline labels and cannot provide end-to-end signals for the whole trajectory or encourage exploration beyond the template. The full trajectory is treated as an RL episode optimized with GRPO. The reward decouples generation quality from correction behavior (Eq. 7):

Task reward : measures retrieval quality by token-level hits on the final sequence, assigning higher weight to the corrected sequence to anchor optimization on the original next-item prediction objective.

Correction reward : shapes the reflection-correction behavior—rewarding accurate error location, correct semantic consistency judgments, and only giving positive incentive when the corrected sequence actually outperforms the draft. This forces the model to produce real improvements, not just detect errors.

Training initializes from SFT weights and computes group-relative advantages per user.

2.3 Entropy-Guided Reflection Scheduling (EGRS)

Running full reflection-correction on every beam is computationally prohibitive. The reflection step naturally yields an uncertainty signal: the average entropy of reflection tokens. EGRS adds an entropy calibration term to the standard beam score (Eq. 8) and prunes using the calibrated score. A hyperparameter controls calibration strength. Under a fixed beam budget, high-uncertainty paths survive and receive more correction opportunities, while confident drafts skip regeneration. The mechanism integrates seamlessly with existing beam search interfaces and is tunable under latency constraints.

Experiments

On public benchmarks and a large-scale industrial dataset, GRC achieves the best performance across all metrics. Online A/B testing confirms significant lifts in advertising revenue and GMV. Thanks to EGRS, the online inference latency increases only marginally, striking a practical balance between effectiveness and deployability.