Xianyu Recommendation System: Architecture, Challenges, and Deployment

This document introduces the speaker, Wan Xiaoyong (Wu Bai), a backend expert at Xianyu, and outlines the role of recommendation technology in the platform.

Unlike deterministic search, recommendation in Xianyu faces high uncertainty due to four C2C characteristics: shallow inventory, lightweight publishing, an open market, and rapid growth.

The recommendation system has evolved through four stages: (1) offline scoring with no personalization, (2) limited algorithms with day‑level latency, (3) expanded algorithm coverage with hour‑level latency, and (4) large‑scale infrastructure upgrades enabling full‑graph, automated model compression and universal recommendation.

Offline challenges include the need for a public real‑time data warehouse, unified feature management, and efficient sample construction. Three main pain points are data timeliness, data accuracy, and redundant storage/computation.

Feature engineering is centralized through a feature center that provides low‑cost, high‑performance access, supports diverse downstream consumption, enforces lifecycle governance, and enables fast sample generation.

Offline‑to‑online model transition requires addressing differences in network structure, input data, and compute architecture. Model compression techniques such as graph splitting, moving compute offline, and operator merging are applied.

Online deployment follows a staged recall‑ranking pipeline (recall, coarse‑ranking, fine‑ranking, contextual re‑ranking) built on a serverless platform with a DAG engine, supporting rapid experiments and resource‑agnostic scaling.

The system emphasizes flexibility for business (lightweight algorithms, fast rollout, low‑cost large‑scale experiments) and stability for algorithms (performance, resource utilization).

To handle rapid business growth, the platform introduces a strategy layer for traffic allocation, a six‑quadrant governance model, and a hierarchical experiment framework with blank layers and global buckets to measure cumulative effects.

Future work includes automated metric attribution, white‑box recommendation pipelines, model interpretability, and large‑scale negative sampling.