Inside E‑Commerce Recommendation Systems: From Data Collection to Real‑Time Personalization

Overview

Recommendation is a crucial traffic source for e‑commerce platforms. Traditionally it appeared in transaction stages, but now it spans all traffic entrances and expands to activities, categories, and operation slots.

Product recommendation improves conversion rates by analyzing users' browsing, collection, and purchase history, clustering and labeling users to present items they may want, even enabling personalized marketing outside the site.

Types of Recommendation

Regular Recommendation

Fixed or rule‑based items placed in recommendation slots, e.g., “customers who bought a baby bottle also bought formula”. Configured by operators, often based on sales or collection rankings.

Personalized Recommendation

Intelligent recommendation based on individual user behavior, preferences, and location, aiming to present relevant items before the user clicks, and even delivering ads through partner platforms.

Key Challenges

Building user profiles, collecting comprehensive behavior data, storing and processing massive logs, and modeling user interests are technically demanding.

Components of a Personalized Recommendation System

User Profiling – constructing models from interests, behavior, and attributes; continuously updating tags.

Data Collection – embedding invisible probes in web and app, gathering logs, ETL processing, and storing in data services.

Data Storage – cleaning duplicates, structuring semi‑structured data, handling ETL pipelines.

Data Modeling – using collaborative filtering, rule‑based, concept‑based, and vector‑space models to represent users.

Recommendation Process

Three stages: preprocessing → recall → ranking.

Preprocessing

Feature extraction from various sources, such as content features and user behavior profiles.

Recall

Train models on extracted features to generate candidate sets using content‑based or collaborative‑filtering methods.

Ranking

Adjust candidate order with click‑through rate estimation, relevance matching, and manual weights.

System Architecture

The architecture consists of offline computation, real‑time computation, and online service layers, plus configuration management and scheduling services.

Online services include recommendation engine, ranking system, AB‑test, and field‑completion services.

Real‑time layer extracts user features and trains models online; offline layer processes historical behavior for relevance training and feature extraction.

Ranking Models

Supports Logistic Regression (LR) and Gradient Boosting Decision Tree (GBDT). The linear model formula is shown below.

Conclusion

Personalized recommendation transforms e‑commerce from search‑driven to discovery‑driven, delivering unexpected yet logical results and enhancing user experience.