How Meituan Scaled Kafka to 15k Nodes and Overcame Latency & Management Challenges

Meituan’s data platform uses Kafka as a unified cache and distribution layer, and as the cluster grew beyond 15,000 brokers and daily traffic exceeded 30 PB, the team tackled slow nodes, read/write latency, large‑scale cluster management, full‑link monitoring, service lifecycle, and future cloud‑native deployment challenges.

KafkaLarge-Scale ClusterRead‑Write Latency

0 likes · 22 min read

How Meituan Scaled Kafka to 15k Nodes and Overcame Latency & Management Challenges

DataFunTalk

Feb 1, 2022 · Big Data

Kafka at Meituan: Practices, Challenges, and Optimizations for Large‑Scale Data Platforms

This article presents Meituan's large‑scale Kafka deployment, describing the current state and challenges of massive data ingestion, detailing latency‑reduction techniques, cluster‑level optimizations, SSD‑based caching, isolation strategies, full‑link monitoring, lifecycle management, and future directions for high availability.

Cluster ManagementKafkaMeituan

0 likes · 22 min read

Kafka at Meituan: Practices, Challenges, and Optimizations for Large‑Scale Data Platforms