Expert Insights on Recommendation System Architecture, Data, Features, Recall, Ranking and Evaluation

Introduction – Recommendation systems are mature but still face many practical difficulties; three experts share their views to help practitioners avoid pitfalls.

Technical Architecture – The system typically follows a pipeline: recall → coarse ranking → fine ranking → re‑ranking, as illustrated by the diagram.

Data Sources – Core data comes from event tracking (埋点). Challenges include noisy data, difficulty in debugging front‑end instrumentation, and high resource consumption for data cleaning.

User Portrait – Combines basic and interest profiles; interest portraits are derived from offline (long‑, mid‑, short‑term) and real‑time signals.

Content Structuring – Structured information varies by domain (e.g., e‑commerce items include category, brand, price, specs). Multi‑modal content adds complexity.

Feature Engineering – Transforms structured information into model‑ready features. Experts stress the difficulty of sample construction, dirty data handling, and the importance of feature selection, cross‑features, embeddings, sequence, context and industry‑specific features.

Recall – Retrieves a large candidate set from the whole item pool. Key points: handle massive data, ensure speed, keep models simple, and use few features. Difficulties include coupling recall with downstream ranking, offline‑online metric consistency, and evaluating hit‑rate.

Recall Algorithms – Popular methods include dual‑tower models, graph neural networks, knowledge‑graph recall, embedding‑based recall, and expert rules. Trends point to GNN and causal inference.

Ranking – Scores items to surface the most relevant ones. Includes coarse ranking (large candidate set, fast, often dual‑tower) and fine ranking (smaller set, richer features, often DIN). Multi‑objective optimization balances objectives such as click‑through rate and conversion rate, commonly using Pareto‑optimal weighting or PLE.

Multi‑Modal Fusion – Challenges arise when items have heterogeneous modalities (text, image, video) or belong to different business lines, requiring separate models or manual weighting.

Re‑ranking – Adjusts final order to satisfy user satisfaction and creator traffic, balancing user experience with commercial goals.

Other Steps & Considerations – Cold‑start solutions include collaborative filtering, meta‑learning, rule‑based approaches, graph neural networks, cross‑domain data, and prompting users for preferences. Evaluation metrics differ by stage: hit‑rate for recall/coarse ranking, AUC/NDCG for fine ranking, plus CTR, CVR, dwell time, and retention for business impact.

Applications – Recommendation logic varies across UI contexts (home page, product detail, post‑purchase). Experts emphasize the need for surprise and discovery alongside precision.

Overall Expert Views – Emphasize data quality, model robustness, balancing precision vs. surprise, handling new scenarios, supporting new content creators, and fostering academia‑industry collaboration.