Recommendation System Architecture and Engineering Overview

Recommendation systems are embedded in many everyday apps, such as showing related books or videos after a user browses or purchases computer‑related items, and they appear in feed streams, "For You", "You May Like", and similar scenarios.

They exist to filter information and predict user preferences, addressing the growing difficulty of manually selecting relevant content from massive data; unlike search engines where users actively query, recommendation systems passively deliver personalized items based on historical behavior.

The engineering architecture consists of six main modules: data source, data computation, data storage, online learning, recommendation service layer, and recommendation access layer (see diagram). Each module collaborates to deliver low‑latency, personalized results.

Data source: items and feature data come from the production platform and various logs (exposure, click, business data) and are stored in ES , HDFS , HIVE , or streamed via message queues.

Data computation: includes offline tasks (feature generation, user profiling, model training) and real‑time tasks (online feature updates, monitoring) to support both low‑frequency and latency‑critical scenarios.

Data storage: separates hot data (user/profile/content vectors) stored in Redis with LRU eviction from cold data stored in HBase, reducing storage cost while ensuring high‑throughput access.

Online learning: uses TensorFlow for model training and TensorFlow Serving to deploy deep‑learning models online; traditional models (XGBoost, LR, LGBM) are also supported.

Recommendation service layer: follows four steps—Prepare, Trigger, Rank, Display. The Prepare stage normalizes requests and fetches AB‑test and profile information; Trigger performs multi‑channel recall; Rank applies machine‑learning or deep‑learning models; Display handles diversification and business‑level control, often via configuration files.

Access layer adds routing, parameter packaging, and strong business controls before the service layer.

Compared with traditional backend services, recommendation services tightly couple data computation with request handling and must process massive data volumes, requiring high performance and robust scalability.

Standardization of data generation and feature processing.

Unified online/offline feature preprocessing.

Real‑time data processing using Spark Streaming and Flink‑based T‑Streaming.

Distributed model training to handle large datasets.

The overall request flow is divided into four modules, allowing custom logic insertion: Prepare (request preprocessing), Trigger (recall), Rank (ranking), and Display (final presentation).

In summary, the system integrates offline computation, near‑line processing, and online serving, leveraging platforms such as T‑Mining, T‑Streaming, and BRPC, with components like Redis, LRU caching, and various monitoring and configuration services. Ongoing work includes refining offline standards, completing memory‑dictionary features, and improving stability and alerting mechanisms.