Personalized World Knowledge Lets Large Models Truly Understand Users in Generative Recommendation

Generative Recommendation (GR) models encode items as discrete Semantic IDs (SID) and treat recommendation as a sequence generation task, offering scalability compared to traditional discriminative models.

Recent advances in Large Language Models (LLM) provide world knowledge that can enhance GR, but existing LLM‑augmented methods rely on fixed handcrafted prompts and directly fuse LLM outputs, leading to two core challenges: (1) fixed prompts cannot capture the multi‑dimensional heterogeneity of user interests; (2) knowledge from LLM may conflict with observed behavior signals.

LWGR Framework

LWGR addresses these challenges with two modules.

Knowledge Extraction

Instead of a single handcrafted prompt, LWGR generates a personalized soft instruction for each user using a parallel codebook. The GR encoder first produces a user context vector, which is projected into multiple sub‑spaces. For each sub‑space a codebook supplies a discrete Codeword; the selected Codewords are concatenated to form the soft instruction embedding. Because Codeword selection is discrete, LWGR employs a Straight‑Through estimator based on IBQ: the forward pass uses a hard argmax index, while the backward pass propagates gradients through the softmax probabilities.

The soft instruction is concatenated with the tokenized user history (item titles/descriptions) and fed to the LLM decoder. The LLM’s output, conditioned on the personalized instruction, is treated as the user‑specific world‑knowledge representation.

Knowledge Fusion

LWGR injects the extracted knowledge into the GR decoder via Cross‑Attention: the decoder’s initial [BOS] token serves as the query, while the LLM knowledge embeddings act as key and value, producing an enhanced [BOS] representation that influences the entire auto‑regressive generation.

To prevent negative transfer, LWGR formulates knowledge fusion as a constrained optimization problem. A reference GR model (trained without LLM knowledge) is frozen; the fused model must not reduce the average token log‑probability of the true target SID beyond a tolerance. A non‑negative Lagrange multiplier λ is introduced, turning the constraint into a saddle‑point objective. Training alternates between gradient descent on model parameters and gradient ascent on λ (primal‑dual updates), automatically adjusting the strength of the degradation penalty.

Experiments

Extensive experiments on several public benchmarks and a large‑scale industrial Lazada advertising dataset show that LWGR consistently outperforms prior methods across all metrics. An online A/B test on Lazada’s recommendation ads platform reports significant lifts in advertising revenue and GMV.

The paper “LWGR: Lagrangian‑Constrained Personalized World Knowledge for Generative Recommendation” (arXiv:2605.18771) details the full methodology and results.