How Generative Models Transform Re‑ranking Architecture for Faster, More Diverse Recommendations

Background

In multi‑stage recommendation pipelines (recall → coarse ranking → fine ranking → re‑ranking), the re‑ranking stage receives a limited candidate set (typically Top 100–500 items) and must output an ordered list that maximizes a global utility function. Pointwise scoring suffers from diversity constraints, position bias, and lack of contextual modeling.

Generation‑Evaluation Architecture

The system uses a two‑stage Generation‑Evaluation (G‑E) framework:

Generation : efficiently produce multiple high‑quality candidate permutations using heuristic rules, random perturbations, beam‑search or generative models.

Evaluation : score each candidate with a refined model and select the globally optimal permutation.

Exhaustive enumeration is infeasible; increasing candidate count yields diminishing returns and higher latency.

Generative Model Choices

Two families are explored:

Autoregressive : generate items sequentially, each step conditioned on previously generated items. Pros: strong sequence dependency modeling, stable training, high quality. Cons: inference latency O(L) for list length L, error propagation.

Non‑autoregressive : predict the entire sequence in parallel with a single forward pass. Pros: very fast inference. Cons: strong conditional independence assumption limits modeling of item interactions.

Non‑autoregressive Model for Production

The deployed model consists of a Candidates Encoder (standard Transformer) and a Position Encoder with cross‑attention, allowing simultaneous attention to candidate items while modeling position‑specific effects. Features include user profile, item attributes, positional data, and upstream ranking scores. The model outputs an n×L score matrix (n candidates, L target length) for parallel inference.

Training uses a log‑loss objective that maximizes the probability of positive label sequences and minimizes that of negative sequences:

Inference Performance Optimizations

CPU latency was reduced by moving the DScatter model service to GPU inference on NVIDIA L20 cards and decoupling it from the main re‑ranking service. Additional optimizations:

Export model to ONNX and accelerate with TensorRT (quantization, layer fusion, dynamic memory).

Cache static item embeddings.

Reuse KV‑Cache for autoregressive models to compute only incremental token values.

Apply LLM acceleration techniques such as GQA.

Future End‑to‑End Sequence Generation

The next‑generation architecture will replace the local two‑stage paradigm with an end‑to‑end generator that directly optimizes global business objectives (e.g., dwell time, interaction depth, ecosystem health). The design includes a unified Transformer‑based generator with a hierarchical mixed‑generation strategy:

Coarse parallel stage predicts a high‑level skeleton (category distribution, content density).

Fine autoregressive stage refines individual items under the skeleton constraints.

Reward modeling will incorporate multi‑objective functions covering:

Engagement – weighted CTR based on scroll depth.

Diversity – entropy across categories/creators.

Fairness – exposure for cold‑start and long‑tail creators.

Training Paradigm Upgrade

Autoregressive models will be fine‑tuned with PPO/DPO reinforcement learning and contrastive learning. A near‑line system generates high‑quality candidate lists scored by DCG‑based metrics. Gains are defined as:

click = +1.0

like/favorite = +1.5

view > 5 s = +0.8

otherwise = 0

Preference pairs are constructed from win/lose lists with a margin δ (e.g., 0.1). The DPO loss aligns the policy model (current autoregressive generator) with a fixed reference model (pre‑trained checkpoint):

This approach aims to break the “quality‑latency‑diversity” triangle, achieving simultaneous improvements and enabling AIGC‑driven recommendation where content generation and sequencing are unified.

References

Gloeckle F. et al., “Better & faster large language models via multi‑token prediction,” arXiv:2404.19737, 2024.

Ren Y. et al., “Non‑autoregressive generative models for reranking recommendation,” KDD 2024.

Meng Y. et al., “A generative re‑ranking model for list‑level multi‑objective optimization at Taobao,” SIGIR 2025.

Zhao X. et al., “Deep reinforcement learning for page‑wise recommendations,” RecSys 2018.

Feng Y. et al., “GRN: Generative Rerank Network for Context‑wise Recommendation,” arXiv:2104.00860, 2021.

Pang L. et al., “Setrank: Learning a permutation‑invariant ranking model for information retrieval,” SIGIR 2020.