Fine-Grained Personalized Recommendation System Architecture for E-commerce

As the internet enters its second half, the demands on the three core elements of "people", "goods", and "scenes" increase, and the explosive growth of data volume and traffic makes traditional manual operations inadequate for complex business scenarios. Achieving precise, accurate, efficient, and intelligent connections among these elements has become a key focus for platforms, making personalized recommendation systems essential.

Two Core Components: Feature Data and Algorithm Engine

This article mainly introduces the architecture of a personalized recommendation system in an e‑commerce scenario. Other aspects such as feature engineering and model training will be covered in separate articles. Below is the overall technical architecture diagram.

The diagram shows that a complete recommendation system consists of two core parts: Feature Data and Algorithm Engine . Feature data is divided into real‑time and offline features, processed in parallel with different technologies. The algorithm engine includes recall and ranking, which are sequentially dependent.

Feature Data

Feature data is categorized by timeliness into offline and real‑time features, and its large volume requires appropriate processing technologies and storage media.

For offline data, distributed databases are used; mature solutions such as MongoDB or Elasticsearch are recommended for storage due to their elastic scalability.

Real‑time data, such as user behavior, relies on streaming technologies like Kafka combined with Spark Streaming, Flink, or Storm. For fast access, cache stores such as Redis are used to hold processed features like user preferences and real‑time user profiles.

As data volume grows, read/write I/O pressure becomes a bottleneck. To alleviate this, the team compresses data using Google’s Protobuf, achieving roughly 50% storage reduction while maintaining high parsing efficiency compared to JSON.

Algorithm Engine

The algorithm engine’s two core components are Recall and Ranking . Recall quickly retrieves a relatively small set (e.g., around a thousand items) from massive data, emphasizing speed and accuracy. Ranking then finely orders the recalled items using complex models.

Recall methods include content‑based, similarity, relevance, and model‑based approaches. To meet high‑concurrency and low‑latency requirements, massive inverted indexes are built and pushed into caches (e.g., Redis). Distributed stores like MongoDB and Elasticsearch support flexible index updates and service scaling.

Similar/relevant recall data, generated from user behavior, orders, clicks, and text information, are stored in caches such as Redis for high throughput.

Rule‑based recall selects items based on query rules. To improve throughput, the team uses high‑performance hardware, distributed databases, and a cache‑centric design that splits large queries into atomic sub‑queries, allowing shared cache entries across users and higher cache‑hit rates.

Ranking

Ranking focuses on model training and deployment. The team has experience with linear models, tree models (GBDT, XGBoost), and deep learning models (TensorFlow). Tree models are popular in e‑commerce due to strong performance and easy integration with Java‑based back‑ends.

Deep learning models are served via TensorFlow Serving, offering high inference accuracy but requiring more complex feature engineering and training pipelines.

For hot deployment, the team uses Zookeeper for synchronizing model updates, enabling online model refreshes without service interruption. TensorFlow Serving also supports hot deployment for deep models.

Conclusion

This article shares foundational engineering solutions for a recommendation system, acknowledging that a complete system involves many more details. The authors invite readers to discuss, provide feedback, or contact [email protected], and announce upcoming articles on feature engineering and algorithmic aspects.