How Recommendation Systems Work: Architecture, Flow, and Evaluation

What Is a Recommendation System?

In the era of information overload, solutions like directories and search engines have emerged, and recommendation systems are among the most representative approaches.

Products such as NetEase Cloud Music and Toutiao use recommendation systems, yet many cannot describe their components and operation, which this article details.

Purpose of Recommendation Systems

Examples: after buying a book on Amazon, you may be recommended a related title; after watching a short video on Baixing.com, you may see a related post.

Recommendation systems discover the binary relationship between users and items to surface content of interest, helping users quickly find high‑quality information, increasing dwell time, and reducing exposure to irrelevant or disliked content.

Help users discover interesting, high‑quality information quickly, improving experience.

Potentially increase user dwell time within the product.

Reduce negative impact of repeated or unwanted content.

Difference Between Search and Recommendation Systems

Search serves explicit user queries by matching keywords to a database, while recommendation operates when users have no explicit intent, leveraging historical behavior to infer interests and provide intelligent suggestions.

Typical Architecture of a Recommendation System

A complete recommendation system consists of multiple functional modules; algorithms are only one part.

Using Baixing.com short‑video recommendation as an example, the system includes five layers: foundation, model, recall, fusion, and application.

Data Collection Layer

This layer gathers data and performs basic processing and statistics. User actions are captured as JSON, streamed via Kafka to HDFS for storage, and used for real‑time profile updates. Offline scripts periodically compute video PV, CTR, and other metrics.

Model Layer

The model layer generates content features for videos and builds user profiles.

Since videos lack sufficient textual information, multiple frames are extracted, images are recognized, and titles are tokenized to create a content feature vector.

User profiles are built by aggregating visited videos and their features; for example, a user who watched 100 videos (70 car‑related, 30 phone‑related) yields a raw interest set {car, phone} and weighted profile {car: 0.7, phone: 0.3}.

Recall Layer

Early systems used simple “Top‑list” rules; modern recall combines hot items, geographic, temporal, content, interest, and association strategies to generate a candidate set tailored to each user.

Fusion Layer

Candidate items from various recall algorithms must be ranked and made comparable. Simple normalization or more complex models such as LR or GBDT are used; LR combines historical behavior, user attributes, and item data to predict click probability and sort candidates.

Application Layer

This layer handles client interaction, including data exchange protocols, operation management (e.g., inserting promotional items), and filtering of already‑seen items.

Operational Flow of a Recommendation System

When a user visits, the system retrieves the user ID, fetches the profile (region, preferred categories, device), performs strategy recall, merges results, de‑duplicates, filters, and finally delivers recommendations.

De‑duplication ensures users are not shown items they have recently seen, typically using a one‑week history.

User Behavior Feedback

User actions (open list, play video, like, dislike, comment, etc.) trigger offline recall updates, profile refreshes, and hot‑item recomputation. Behaviors are classified as explicit (e.g., like, dislike) or implicit (e.g., view, play, comment), with explicit feedback being rarer but more valuable.

Weight hierarchy: likes and collections have highest weight, followed by clicks, searches, filters, and finally browsing.

Offline Computation Process

The core offline tasks are content‑feature extraction and user‑profile calculation.

Content Feature Calculation Different media types require different methods: text uses TF‑IDF keywords; images use classification/recognition; videos are sampled frame‑by‑frame, each frame processed, and results aggregated into a video feature vector.

User Profile Calculation By analyzing clicks, collections, likes, and combining them with video features, the system derives interest vectors; demographic attributes add regional features.

How to Evaluate Recommendation Effectiveness?

Common metrics include:

Accuracy – measured by RMSE for rating prediction or correct‑recommendations/total recommendations for top‑N.

Recall – correct recommendations divided by the number of items the user actually likes.

Diversity – ensuring varied content across different interest weights.

User Satisfaction – gathered via surveys (explicit buttons) or inferred from click‑through rate, dwell time, conversion.

Timeliness – relevance of time‑sensitive content such as news.

Novelty – recommending items the user has not seen before.

Surprise – items that seem unintuitive yet match the user’s taste.

In practice, accuracy and recall are the most frequently used metrics.

Conclusion

A good recommendation system requires comprehensive consideration across data collection, algorithmic modeling, and product design. By designing robust models, presenting results effectively, and continuously tracking user feedback, overall recommendation performance can be improved.