Challenges and Best Practices in Recommendation Systems – Expert Interview

Introduction – Recommendation systems are mature, but their deployment still faces many challenges; three experts share their insights to help practitioners avoid pitfalls.

Technical Architecture – The system typically consists of a recall module that generates candidate sets, followed by coarse ranking, fine ranking, and re‑ranking before presenting items to users.

Data Sources

Tracking (埋点) is easy to implement, but constructing clean, accurate samples, especially for real‑time social scenarios like live streaming, is difficult.

Challenges include noisy data, complex front‑end reporting, and resource‑intensive back‑tracking.

User Profiles – Include basic and interest profiles; interest profiles are derived from offline (long‑term, mid‑term, short‑term) and real‑time data.

Content Structuring – Different domains require different structuring (e.g., e‑commerce items need category, brand, price, specifications). Multi‑modal content adds significant computational cost.

Feature Engineering

Feature generation is hard due to sample stitching and dirty data cleaning.

Popular techniques involve embedding‑based feature crossing.

Feature trends: sequential features, contextual features, and embeddings.

Recall features are a subset of ranking features; recall focuses on efficiency and large‑scale data, while ranking can use richer cross features.

Recall

Key points: handle massive data, ensure speed, keep models simple, use few features.

Challenges: coupling recall with downstream ranking, aligning offline and online metrics (e.g., hit rate).

Popular algorithms: dual‑tower models, graph neural networks, knowledge‑graph recall, embedding‑based matching.

Emerging trends: GNN recall, knowledge‑graph recall, causal inference.

Ranking

Coarse ranking outputs a large candidate set for fine ranking; both require efficient models, often dual‑tower.

Fine ranking focuses on top items, using richer features and models such as DIN.

Multi‑objective optimization balances click‑through rate, conversion rate, and other business goals, often using Pareto‑optimal weight search or PLE models.

Multi‑Modal Fusion – Combining text, images, video, or different business lines is difficult; current solutions rely on manual weighting and slot allocation.

Re‑ranking – Adjusts final order to balance user satisfaction and creator exposure, considering newcomer support and commercial product placement.

Cold‑Start Solutions

Collaborative filtering with similar users.

ML models trained on historical data to bootstrap new users/items.

Rule‑based business logic.

Graph neural networks for item cold‑start.

Cross‑domain data collection.

Evaluation Metrics

Recall and coarse ranking: hit rate.

Fine ranking: AUC, NDCG.

Business metrics: CTR, CVR, dwell time, retention.

Practical Insights – Balancing precision and surprise, handling data scarcity, ensuring system robustness, and fostering industry‑academia collaboration are essential for successful recommender systems.