Overview of Baidu's Wanxiang System for Large‑Scale Rich Media Processing

In the era of rich media, information is no longer limited to plain web pages; images and videos dominate user experience, creating new challenges for search engines that must process, index, and retrieve massive amounts of multimedia content.

The Wanxiang system (named after the Chinese phrase for “all‑encompassing”) was built by Baidu to handle the large‑scale ingestion, processing, and indexing of image and video data, supporting billions of daily processing tasks and powering various Baidu products such as image search, video search, and recommendation.

The system emphasizes two core design goals: scalability (processing tens of millions of media items using hundreds of thousands of CPU cores, GPUs, and FPGAs) and timeliness (producing features, filters, and indexes within product iteration cycles).

Wanxiang consists of four major subsystems:

Qianren (Blades) : extracts basic features from individual media entities (e.g., object detection, OCR, clarity) using a DAG execution engine that balances CPU‑intensive and GPU‑intensive tasks.

Chuyu (Initial) : analyzes relationships between entities (e.g., similarity, duplication, event grouping) by comparing fingerprint‑level signatures generated by Qianren.

Danding (Athanors) : stores and aggregates feature data, merging attributes of duplicate entities so that downstream retrieval can operate on content‑level signals rather than page‑level signals.

Auxiliary services : handle tasks such as cropping, transcoding, and editing.

These components enable the extraction of both low‑level attributes (size, clarity) and high‑level semantic tags (scene, object, text), as well as the aggregation of feedback signals (clicks, play counts, likes) from multiple platforms (web, mobile apps, vertical apps) into a unified content‑centric representation.

The article also discusses the evolving search challenges: multi‑modal input (text, image, video, semantic queries), fragmented user feedback across devices, and the need for content‑level indexing rather than traditional page‑level inverted indexes.

By integrating these capabilities, Wanxiang provides Baidu with a robust foundation for rich‑media search, ensuring that billions of daily queries can retrieve relevant images and videos with high accuracy and freshness.