Generative Retrieval for E‑commerce Search: Lexical and Semantic ID Approaches

The authors, Wang Hui‑mu and Li Ming‑ming, participated in DataFunsummit2024 and introduced generative retrieval based on large language models to address the challenges of traditional dual‑tower retrieval in massive product catalogs.

Background and challenges : Traditional recall pipelines struggle with efficiency and semantic matching, especially for long‑tail queries, due to limited tower interaction and index maintenance costs.

Advantages of generative retrieval : It avoids link loss, simplifies index management, enhances model performance with advanced LLMs, and leverages world knowledge for better personalization.

Lexical‑based approach : The authors propose a Preference‑Optimized Generative Retrieval framework that redefines the task from Query‑to‑Title to Query‑to‑MultiSpan, using supervised fine‑tuning, DPO preference optimization, and constrained beam search to generate concise spans that are later matched against an efficient FM index.

Semantic‑ID‑based approach : They explore numeric semantic IDs (SID) generated via residual quantization, identifying a “sand‑glass” distribution where middle‑layer tokens become overly concentrated, causing sparsity and long‑tail performance degradation.

Experimental findings : Experiments on models such as LLaMA, Qwen, and Baichuan show that the sand‑glass effect harms head‑token performance while benefiting tail tokens; swapping token layers or removing dominant middle‑layer tokens improves overall recall.

Proposed solutions : Two methods are introduced – a heuristic removal of large routing nodes and an adaptive variable‑length token strategy (top‑K removal) – both effectively mitigating the sand‑glass issue and boosting recall metrics.

Future work : The authors aim to enhance SID representations, integrate multimodal features, and unify generative retrieval with ranking to reduce pipeline loss and improve real‑time search performance.

Team and contact : The work is conducted by JD.com’s Search Algorithm Team, with contact email [email protected] for interested collaborators.