Engineering Exploration of Generative Recall: Leveraging Large Models for Next‑Gen Recommendation Retrieval

The article analyzes the limitations of traditional ID‑based and vector recall, proposes a generative Semantic‑ID‑based recall framework (ASGR), details a series of inference‑time optimizations—including a custom SDPA‑enabled optimizer, novel beam search, static‑cache compilation, tree‑aware attention, sliding‑window attention, quantization, ONNX/Optimum conversion, and TensorRT‑LLM integration—and reports concrete latency and throughput gains that enable large‑scale, low‑latency recommendation serving.

beam searchgenerative recallinference optimization

0 likes · 25 min read

Engineering Exploration of Generative Recall: Leveraging Large Models for Next‑Gen Recommendation Retrieval

Tencent Advertising Technology

Jan 9, 2025 · Artificial Intelligence

Applying Large Language Models to Search Advertising: End‑to‑End Generative Recall and System Optimizations

This report details how large language models (LLMs) were integrated into Tencent's search advertising pipeline—from early extraction‑distillation experiments in 2023 to a 2024 end‑to‑end generative recall architecture—showing significant improvements in relevance, diversity, and revenue through knowledge injection, supervised fine‑tuning, constrained beam‑search decoding, and high‑performance inference services.

AILLMbeam search

0 likes · 11 min read