What Makes a Good Recommendation System?

A good recommendation system cannot be defined by a single absolute metric; instead, its quality is assessed through a full‑stack evaluation that spans the entire user interaction journey. While algorithms are central, the system’s success also depends on supporting technologies, architecture, and interaction design.

The article highlights the importance of customer experience value, emphasizing that different product features require distinct evaluation indicators, with the core question being how well user needs are satisfied.

Key influencing factors include user demand, data quality, algorithm strategy, module placement, presentation style, and alignment with product goals.

From a product‑domain perspective, evaluation criteria differ across stages: early‑stage products focus on user onboarding and cost‑performance (e.g., new‑user promotions), while mature products prioritize metrics such as browsing depth, click‑through rate, conversion rate, average order value, and overall GMV.

Information‑retrieval considerations stress path optimization and accurate item delivery to streamline user behavior and facilitate discovery of new items.

From the recommendation‑system angle, essential tasks involve long‑tail mining, helping users discover unknown items, and providing persuasive, trustworthy suggestions.

Interaction aspects cover user experience (retention, loyalty, word‑of‑mouth), educational guidance, and building user trust in recommendations.

Commercially, a good system should generate tangible business value, optimize sales boundaries and profit, and expand product reach through diverse consumption paths.

The article outlines a five‑W framework (When, Where, Who, What, Why) to guide product positioning, followed by a set of macro and micro evaluation dimensions such as the HEART model (Happiness, Engagement, Adoption, Retention, Task Success) and various algorithmic accuracy and diversity metrics.

Offline metrics (MSE, RMSE, AUC, NDCG, ROC) and online metrics (A/B testing, CTR, CVR, GMV) are discussed for system assessment.

Finally, a systematic iteration workflow is proposed: clarify product needs, define recommendation goals, select appropriate solutions, estimate targets, monitor effects, and continuously refine the system through stages of requirement analysis, design, architecture, algorithm development, and evaluation.