This article details Bilibili's transformation of its search offline indexing pipeline, moving from manual MySQL‑based processes to a high‑capacity, distributed KV store and Spark‑driven builds, addressing performance, maintenance, and scalability challenges while improving resource efficiency and iteration speed.
Facing billions of video entities and edges, Youku’s engineering team replaced traditional relational stores with a graph‑based knowledge platform, leveraging Alibaba’s Blink streaming engine and Lindorm to enable real‑time, incremental updates, unified UDF logic, and scalable feature computation for search and recommendation.
58.com tackles the challenge of storing and searching billions of heterogeneous posts by employing a unified post center, a category‑attribute service, and an external search engine, using vertical table splitting, JSON‑based extensible fields, compressed keys, and horizontally sharded indexes to achieve massive scalability and high throughput.
The article reviews the evolution of large‑scale website architecture, explaining how business complexity, multi‑server deployment, session handling, load balancing, database read/write separation, caching, and search indexing together address availability, concurrency, and performance challenges in modern web systems.
