Design of a Generalized Recommendation Platform for Xianyu Marketplace

Background

In today’s information‑overloaded Internet, recommendation systems are ubiquitous – from product browsing on Taobao to video feeds on Douyin. This article examines Xianyu’s product recommendation system and the challenges of maintaining multiple recommendation scenarios, heavy engineering effort, and difficulty of propagating model improvements across scenarios.

Recommendation System

When users cannot precisely express their needs, a recommendation system acts as an intermediary, selecting the most relevant items from massive candidate pools.

Problems

Xianyu now supports over 10 recommendation scenarios, with new ones added frequently. Each new scenario currently requires building a full recommendation pipeline from scratch, leading to high marginal costs and duplicated effort. Model updates also need case‑by‑case tuning.

Design Solution

Design Goals

The goal is to build a recommendation middle‑platform that serves many small‑to‑medium scenarios with a single set of core components, enabling rapid scenario onboarding (days instead of weeks) and allowing model improvements to benefit multiple scenarios.

Overall Architecture

The platform relies on a feature center that combines user and item features to generate candidate pools for each scenario. Offline data processing, sample collection, model training, and index construction feed the online service, which handles recall, scoring, and re‑ranking. All strategy configurations are managed through an experiment platform.

Candidate Pool

Candidate pools are defined by flexible feature expressions. By merging user and item features, the system tags each item with scenario identifiers, enabling a single item to belong to multiple pools.

Recall Engine

Three recall methods are provided: i2i (item‑to‑item co‑click similarity), x2i (attribute‑based inverted index), and deep recall (vector‑based ANN using user/item embeddings). Each method outputs item IDs with recall scores, and multiple channels can be configured per scenario.

Scoring Engine

The scoring engine enriches candidate items with features and applies a multi‑objective deep model (CTR, CVR, interaction) to produce personalized scores. Scoring can be customized per scenario via weighted expressions.

Experiment System

AB testing is supported with independent traffic models per scenario and hierarchical layering for multiple experiments within a scenario, ensuring isolation.

Stability

The platform is deployed across two data centers for high availability. Scenario‑level rate limiting and circuit breaking isolate failures, preserving overall service stability.

Conclusion

The generalized recommendation platform reduces scenario onboarding time to days, improves click‑through rate by over 8%, and increases per‑user impressions by more than 10%. Ongoing work focuses on improving model accuracy, multi‑scenario joint modeling, and further automation of scenario integration.