Skip to content
5x Inference Speedup: Inside Tencent's HiGR Generative Recommendation Framework

5x Inference Speedup: Inside Tencent's HiGR Generative Recommendation Framework

Tencent's HiGR framework tackles slate recommendation with hierarchical semantic IDs (PCRQ-VAE), a split planner-generator decoder (HSD), and list-level ORPO alignment, achieving over 5x faster inference than OneRec-Beam while lifting watch time 1.22% and video plays 1.73% in production across news, video, and novel platforms.

Sohu Tech Products
Sohu Tech Products
Sohu Tech Products

At CIKM 2026, Tencent's TRS team introduced HiGR (Hierarchical Generative Recommendation), a list-level generative recommendation framework now serving hundreds of millions of users across Tencent News, Video, and Web Novels. HiGR belongs to the TGR (Tencent Generative Recommendation) family, an integrated generative recommendation solution deployed in the TRS recommendation engine platform.

Three Core Challenges in Generative Slate Recommendation

Generative recommendation encodes items into discrete Semantic IDs (SIDs) that the model generates token by token. Moving from single-item generation to whole-list generation surfaces three problems:

SID prefixes lack stable semantics. In standard residual quantization (RQ-VAE), semantically similar items do not necessarily share similar prefixes, making it hard for a planner to steer list-level diversity and relevance.

Token-level beam search slows list generation. A list of M items each with D SID tokens requires decoding M×D tokens. Beam search over the full sequence incurs high latency and conflates intra-item SID relations with inter-item transitions.

Training objectives don't target list-level experience. Next-token prediction alone does not teach the model which ordering is better, which items match true interest, or how to avoid repetitive slates.

HiGR addresses each with PCRQ-VAE for semantic prefixes, HSD (Hierarchical Slate Decoder) for decoupled planning and generation, and list-level ORPO for preference alignment.

PCRQ-VAE: Building Hierarchical Semantic IDs

PCRQ-VAE maps item embeddings into a multi-layer discrete code via residual quantization. The encoder projects items into a latent space; each layer selects a codebook vector to represent the residual. A 3-layer codebook with 1024 entries per layer is used in experiments.

Unlike vanilla RQ-VAE which only minimizes reconstruction loss, PCRQ-VAE adds a global quantization loss that aligns the sum of all codebook vectors with the original latent representation, ensuring the full code carries the reconstruction burden. Crucially, it applies InfoNCE contrastive learning on the first D−1 layers : semantically neighboring or co-consumed items are pulled together in prefix space, while other batch items serve as negatives. The final layer is left unconstrained to preserve fine-grained discrimination.

Table 2 quantifies the effect. With contrastive constraints on the first two layers, collision rate drops from 2.98% to 2.37%, category concentration reaches 93%, and prefix consistency rises from 55.77% to 66.47%. Extending contrast to the last layer boosts consistency to 69.32% but raises collisions to 8.73%, confirming the trade-off: HiGR constrains only the first D−1 layers.

PCRQ-VAE constructs semantic IDs
PCRQ-VAE constructs semantic IDs
Encoding quality under different prefix contrastive constraints
Encoding quality under different prefix contrastive constraints

HSD: Decoupling Planning from Generation

The Hierarchical Slate Decoder (HSD) raises the autoregressive granularity from token-level to item-level. A Context Encoder produces a shared context C from user profile and history. A Planner (14 Transformer layers by default) attends to C via cross-attention and predicts a preference vector for the next slate position. A lightweight Item Generator (2 layers) conditions on this vector to emit the item's SID tokens. The generated SID embeddings are summed to form the item representation, which feeds back into the Planner for the next position.

Inference proceeds as: (1) Planner emits first preference from BOS + context; (2) Generator produces that item's SIDs; (3) SID embeddings summed → item representation; (4) Repeat until slate complete. The same Generator is reused across positions, supporting variable slate lengths.

Search strategy changes accordingly: the Planner uses greedy decoding, while beam search is confined to the short SID sequence inside the Generator. This avoids maintaining a large beam over the entire M×D sequence. A diversity regularizer on Planner outputs (lowering cosine similarity between position preference vectors) prevents premature clustering.

Ablation (Table 3) shows removing context drops NDCG@5 from 0.0753 to 0.0664; removing Planner diversity leaves NDCG unchanged but cuts ILD (intra-list diversity) from 0.62 to 0.59. Giving each position a separate Generator yields NDCG 0.0748 vs. 0.0753 for shared, confirming parameter sharing is beneficial.

Layer allocation (Table 5): 14 Planner + 2 Generator layers gives the best quality-latency trade-off. Shifting layers to the Generator (12+4) raises NDCG@5 marginally (0.0631→0.0635) but increases relative latency to 1.355×; further shifts degrade both.

Hierarchical Slate Decoder architecture
Hierarchical Slate Decoder architecture
Decoder component ablation results
Decoder component ablation results
Planner vs. Generator layer allocation
Planner vs. Generator layer allocation

List-Level ORPO: Aligning with Whole-Slate Feedback

HiGR converts implicit user feedback into positive/negative slate pairs. Positive slates come from actually consumed items, ordered by feedback strength. Three negative slate types target distinct failure modes:

Order corruption: same items, shuffled order → teaches ranking fidelity.

Content replacement: swap in negatively-exposed or non-clicked items → teaches true interest discrimination.

Repetition injection: fix first item, fill rest with semantically nearest neighbors → teaches diversity.

ORPO computes log-odds ratios over the full SID sequence for each slate, summing per-step log-odds to obtain a slate-level score. Training jointly optimizes supervised likelihood on positive slates and the ORPO margin pushing positive scores above negatives — without a separate reference model, saving forward compute and memory.

Table 6 isolates each objective: diversity-only yields ILD 0.64 but NDCG@5 0.0780; all three together achieve NDCG@5 0.0831 and ILD 0.62 (vs. 0.0753/0.52 without alignment). Table 4 shows ORPO outperforms DPO on intra-list diversity (0.62 vs. 0.57 on industrial data; 0.63 vs. 0.60 on KuaiRec).

List-level preference alignment
List-level preference alignment
Contribution of each preference objective
Contribution of each preference objective
Intra-list diversity across alignment methods
Intra-list diversity across alignment methods

Recommendation Quality Improvements

Experiments use Tencent industrial data (1B pretraining samples, 3% for post-training) and public KuaiRec (leave-five-out list evaluation). Metrics: Impressions (exposure modeling), Effective Views and NDCG (consumption modeling).

At 25M parameters on industrial data, HiGR beats OneRec: Effective Views Hit@5 0.1603→0.1945, NDCG@5 0.0589→0.0714 (~21% gains). Without ORPO, HiGR already reaches NDCG@5 0.0631; ORPO adds the rest. On KuaiRec, NDCG@5 improves from 0.1496 to 0.1574 (+5.2%). Scaling to 100M pushes industrial NDCG@5 to 0.0831. Figure 4 shows converging loss and rising NDCG across 0.05B–2B parameter sweeps.

Industrial dataset results (Table 1)
Industrial dataset results (Table 1)
Model scale vs. loss and NDCG
Model scale vs. loss and NDCG

5× Inference Speedup and Deployment Gains

Under identical model settings and hardware, without KV cache, HiGR achieves over 5× throughput improvement over OneRec-Beam while exceeding its Recall@5 by 5%+. OneRec with greedy decoding matches speed but sacrifices recall; HiGR occupies the top-right quality-efficiency frontier. With KV cache enabled, the advantage persists.

Deployment results: at equal throughput, P99 latency ≤ 50 ms, and GPU demand drops ~60% versus OneRec . Additional optimizations include FlashAttention, KV cache, and beam-search deduplication.

HiGR vs. OneRec quality-efficiency comparison
HiGR vs. OneRec quality-efficiency comparison

Online A/B Test Results

HiGR is integrated into the TGR pipeline: PCRQ-VAE encoding → HSD pretraining → ORPO alignment. At serving time, the model generates SIDs from real-time user features; an index retrieves actual items, followed by deduplication, availability, safety, and business filters.

On Tencent's commercial media platform (5% traffic), A/B against the legacy multi-stage system shows: average dwell time +1.03%, average watch time +1.22%, average video plays +1.73%, average requests +1.57%. Other business tests report CTR +0.68% / ad revenue +0.56% in one scenario, and watch time +1.14% / video plays +0.88% in another. Exposure and consumption logs flow back into the training loop for continuous list-level preference learning.

HiGR online results across three business scenarios
HiGR online results across three business scenarios

References

HiGR paper v5: https://arxiv.org/abs/2512.24787v5

CIKM 2026 official site: https://cikm2026.diag.uniroma1.it

TGR technical report: https://arxiv.org/abs/2609.00986

TRS recommendation engine platform: https://trs.tencent.com

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

TencentGenerative RecommendationSemantic IDCIKM 2026Hierarchical DecodingORPOPCRQ-VAESlate Recommendation
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.