Technical Trends in Recommendation Systems: From Retrieval to Re‑ranking

The article reviews the current technical trends of recommendation systems, noting that although their development pace lags behind NLP and computer vision, recent deep‑learning breakthroughs have sparked noticeable progress across all stages of the pipeline.

It first clarifies the macro architecture: traditional industrial systems are often described as two stages—recall (fast, coarse filtering) and ranking (accurate, personalized scoring). A more detailed view splits the process into four stages: recall, coarse ranking, fine ranking, and re‑ranking, each with distinct performance goals.

In the recall stage, the shift is toward model‑based approaches. Embedding‑driven two‑tower models (e.g., FM, DNN) replace handcrafted multi‑route recall, allowing supervised learning and multi‑feature fusion. The article discusses head‑bias issues and the need for consistent multi‑objective optimization between recall and downstream stages.

User‑behavior sequence recall is highlighted as a powerful way to capture fine‑grained interests. By aggregating the items a user has interacted with—using CNN, RNN (especially GRU), or Transformer encoders—systems generate user interest embeddings for vector‑based retrieval. Multi‑interest models (e.g., MIND) further split a user’s interests to mitigate head‑bias.

Knowledge‑graph fusion is presented in two forms: embedding‑based (KGE) and path‑based (Meta‑Path). While embedding methods lack interpretability, path‑based methods provide clear explanations but often underperform. The article suggests using knowledge graphs mainly in the recall stage for cold‑start and sparsity scenarios.

Graph neural networks (GNNs) extend the idea of graph‑based recall beyond static knowledge graphs. By learning node embeddings through GraphSAGE or PinSage, GNNs can incorporate heterogeneous user‑item interactions at scale, offering a promising direction for future recall systems.

The ranking stage is described as the most technically intensive part. The evolution from linear models (LR) to factorization machines (FM) and deep neural networks (DNN) is traced, emphasizing explicit feature interaction (DeepFM, Deep⨯, XDeepFM) versus implicit interaction via MLPs. The article argues that higher‑order explicit interactions have diminishing returns.

Feature extractors are examined: CNNs and RNNs are less suited for unordered feature vectors, while Transformers’ self‑attention can model arbitrary feature interactions but have not yet shown clear superiority in practice. AutoML attempts (e.g., ENAS‑based search) to discover better architectures are discussed, noting limited gains so far.

Reinforcement learning is framed as a natural fit for recommendation, offering exploration‑exploitation balance, dynamic interest modeling, and long‑term reward optimization. However, practical deployment challenges and modest online impact are highlighted.

Multi‑objective optimization is explored through various model sharing schemes: Share‑Nothing (independent models), Share‑Bottom (shared bottom layers), and MMOE (Mixture‑of‑Experts) that adaptively allocate sub‑networks per task. Pareto‑optimal weight search is mentioned as a way to balance conflicting business metrics.

Multimodal fusion is described as embedding different modalities (text, image, video, audio) into a unified semantic space, with the main difficulty being engineering efficiency rather than algorithmic complexity.

The separation of long‑term and short‑term user interests is outlined: long‑term interests are often represented by simple user ID embeddings, while short‑term interests rely on recent behavior sequences processed by sequence models (RNN, CNN, Transformer). The article suggests more sophisticated long‑term modeling remains an open research area.

Finally, re‑ranking trends focus on listwise optimization. Listwise loss functions consider the entire ranked list, and context‑aware models (RNN, Transformer) re‑score top‑k items using their mutual context. Listwise approaches, though computationally heavier, aim to improve overall list quality and are increasingly adopted in production.