1 views collected around this technical thread.
This article explains how Baidu's vertical search platform tackled massive data growth and scaling challenges by redesigning its data management system, introducing elastic scheduling, decoupling ETCD access, implementing auto‑scaling, and advancing shard expansion to improve performance, stability, and cost efficiency.
Bilibili’s large‑scale recall system separates online processing into a two‑tier merge service and an index service, supports multi‑channel text, item‑to‑item and vector indexes with real‑time updates, uses horizontal sharding, robust CI/CD, monitoring and degradation mechanisms, and is being extended toward model‑based recall and greater automation.
AIAPI, Baidu’s AI‑native retrieval platform for large language models, tackles hallucination, slow domain updates, and output opacity by delivering authoritative, timely, full‑content data through a dual‑channel architecture that combines traditional search and RAG, employs reusable ranking, graph‑enhanced data layers, dynamic caching that cuts storage by 70 %, and QueryPlan‑based QoS, achieving markedly higher retrieval quality and a 34 % speed gain with Wenxin 4.0.
This article describes how to combine Elasticsearch’s keyword matching with Milvus’s vector‑based semantic search to build a scalable document search service, covering data preprocessing, architecture, query handling, custom scoring, DSL configuration, and result merging.
Exgraph, Baidu Search’s graph execution engine, uses a human‑readable description language and a robust execution core with dependency injection, object pooling, and interruption handling to orchestrate complex, parallel or conditional tasks, improving code readability and unifying diverse execution scenarios in search architecture.
The article chronicles Baidu Search’s Model Architecture Group’s evolution of deep‑learning‑driven search, detailing the shift from inverted‑index to semantic vector indexing, the use of transformer‑based models for text and image queries, large‑scale offline/online pipelines, and extensive GPU‑centric optimizations such as pruning, quantization and distillation, all aimed at delivering precise, cost‑effective results to hundreds of millions of users.
This article presents a cloud‑native redesign of Baidu's search middle‑platform that introduces intelligent data management, elastic scaling, on‑demand resource allocation, precise fan‑out, and localized computation to address efficiency, cost, stability, and performance challenges of large‑scale search workloads.
Cloud-native redesign of Baidu's search middle platform introduces partition, shard, replica, and addressing controllers that enable elastic scaling, on-demand resource allocation, precise fan‑out, and localized computation, reducing capacity adjustment time from weeks to hours, cutting costs by 30‑80%, raising availability above 99.9% and halving query latency.
Idlefish’s new multi‑engine concurrent search architecture replaces the tightly‑coupled single‑engine pipeline with deep engine isolation, asynchronous multi‑engine recall, and unified result merging, cutting dump build time from 14 h to 5 h, shrinking memory use dramatically, improving latency by only ~15 ms, and boosting exposure by 50 % and orders by 33 %.
iQIYI Douya’s emoji search evolved from ElasticSearch to a pure Lucene implementation and added semantic vector retrieval, enabling fast, scalable, and more accurate text‑based search of AI‑generated images for small‑to‑medium businesses by combining custom tokenization, dense embeddings, and hybrid ranking.
This article surveys how major Chinese internet companies such as JD.com, Ctrip, Qunar, 58.com and Didi have adopted Elasticsearch and the Elastic Stack for high‑volume order queries, log analysis, real‑time monitoring, security analytics, and large‑scale distributed search, describing their architecture evolution, shard strategies, and operational practices.
This article explains how Fangdd leverages Elasticsearch to boost search performance across consumer, broker, and internal products, detailing a platformized architecture that separates indexing and querying, addresses operational challenges, and outlines design patterns for index management and incremental updates.
The Youzan Search Platform evolved from a simple Elasticsearch cluster in 2015 to a modular, message‑driven architecture with proxy validation, caching, and management tools, and now plans a cloud‑native, Kubernetes‑based 4.0 version that automates data sync, isolates workloads, and scales elastically to support billions of records.
This article introduces Mogujie's end‑to‑end search system architecture, detailing its online and offline components such as Topn, ABTest, QR, fine‑ranking, search engine, UPS, and feature platforms, and then walks through a real‑world online request example to illustrate how queries are processed, rewritten, personalized, and finally ranked.