How Our Recommendation Engine Evolved from V1.0 to V3.0

1. Introduction

Recommendation has become the core competitive edge for e‑commerce platforms, driving user experience and revenue across many entry points such as home pages, product detail pages, carts, and checkout success pages. As business volume and algorithmic capabilities grew, the recommendation system at "DaoJia" underwent three major architectural revisions.

2. DaoJia Recommendation Framework V1.0

V1.0 adopted a simple strategy‑plus‑factory pattern to quickly launch recommendation services for various business lines. The design allowed rapid iteration but introduced several risks:

All business logic ran in a single JVM, causing poor fault isolation, resource contention, and limited scalability.

Increasing business complexity made the simple pattern a bottleneck for development efficiency.

Recall relied on direct Redis access, creating a performance bottleneck for high‑traffic scenarios.

A single Redis cluster stored all data, so heavy traffic from one business could affect others.

These issues motivated a redesign.

3. DaoJia Recommendation Framework V2.0

V2.0 introduced vertical business splitting and horizontal division of the recommendation pipeline into distinct stages (recall, filter, coarse ranking, merge, fine ranking, intervention, etc.). Each business now had its own application and storage instances, improving fault isolation and resource utilization. A pipeline scheduler was added to modularize each stage, allowing configuration‑driven assembly of the recommendation flow. The configuration service could manage AB‑testing flags and experiment parameters per handler.

Key improvements:

Business‑level isolation reduced the impact of failures and enabled independent scaling.

Modular pipeline reduced code redundancy and lowered coupling between stages.

AB‑testing capabilities were embedded in handlers, allowing dynamic experiment control without code changes.

4. DaoJia Recommendation Framework V3.0

V3.0 further separated configuration logic into a dedicated configuration server and client. The server stores the full recommendation workflow, while the client periodically polls for updates, validates them, and applies them to local execution. This dynamic configuration enables:

Centralized management of all recommendation scenarios.

Real‑time strategy changes without redeploying services.

Separate, scalable recall and prediction services (recall moved to an Elasticsearch‑backed pool, prediction became a microservice supporting multiple models and versions).

Improved Redis usage by sharding the original monolithic cluster into multiple smaller clusters, reducing risk of single‑point failures.

The pipeline can now be reconfigured on‑the‑fly, supporting new business scenarios (A, B, C) while sharing common components.

5. Outlook

Future work aims to enhance fine‑grained personalization, build an explanation platform for recommendations, and integrate real‑time feature feeds. The goal is to achieve truly individualized recommendations with clear rationale, while maintaining high availability, scalability, and low latency.