0 views collected around this technical thread.
This article explains why traditional databases like MySQL struggle with massive data, introduces Elasticsearch’s advantages, and details a practical architecture using Hive, Canal, and Otter to achieve near real‑time indexing of petabyte‑scale datasets with minimal latency.
The article details Xianyu’s end‑to‑end product‑search architecture—covering tokenization, indexing, online request flow, offline index building, multi‑datacenter active‑active deployment, and supporting ad and debugging systems—and explains how expanding searcher capacity, separating query engines, grading updates, and diffusing auxiliary‑table writes together reduced latency from hours to near‑zero, enabling real‑time search.
This article explains how to construct near real‑time Elasticsearch indexes for petabyte‑level datasets by comparing MySQL limitations, describing Elasticsearch fundamentals, and detailing a pipeline that uses Hive, wide tables, MySQL binlog, Canal, and Otter to achieve second‑level index updates.
The article describes a real‑time video recommendation indexing system that replaces 30‑minute batch builds with an Elasticsearch‑based service, integrates prior and posterior data pipelines, ensures consistency via locking and version checks, enables zero‑downtime upgrades, smooths write spikes, and boosts recall performance through multi‑level caching and ES tuning, delivering sub‑40 ms latency and significant business growth.
This article explains how to build a real-time search engine indexing pipeline using Flink, covering background, batch versus incremental indexing strategies, a hybrid architecture that merges both approaches, and a concrete cloud‑based implementation involving MySQL binlog, Logtail, SLS, and Elasticsearch.
The article summarizes the evolution and redesign of Zhihu's search engine, details 58 Group's high‑performance uesearch architecture, real‑time indexing mechanisms, cloud‑native deployment with Kubernetes, and highlights key technical insights and future directions for large‑scale search systems.
This article describes how Ctrip's hotel R&D team tackled growing order‑volume challenges by sharding the database, building a real‑time Elasticsearch index, implementing a custom replication pipeline, and applying various write‑ and read‑optimizations to achieve low latency and stable performance.
This article explains how 58.com’s in‑house C++ search kernel ESearch was architected to provide second‑level real‑time indexing, high‑concurrency low‑latency querying, flexible ranking models, and efficient storage structures for billions of daily queries across massive classified data.