Meituan Instant Logistics: Evolution of Distributed System Architecture and Technical Challenges

Background

Meituan's instant logistics has been developed for over five years, accumulating extensive experience in building distributed high‑concurrency systems. Two main takeaways are the need for ultra‑low fault and latency tolerance as business complexity grows, and the integration of AI across pricing, ETA, dispatch, capacity planning, subsidies, accounting, voice interaction, LBS mining, operations, and monitoring to boost scale, preserve experience, and reduce cost.

Meituan Instant Logistics Architecture

The platform focuses on three core tasks: providing SLA guarantees such as ETA and pricing, matching riders under multi‑objective optimization (cost, efficiency, experience), and offering rider assistance through intelligent voice, route recommendation, and store arrival reminders.

These services rely on a robust technical foundation and a distributed system architecture that ensures high availability and high concurrency.

Distributed architecture, compared with centralized architecture, follows the CAP theorem and involves deploying services across multiple peer nodes that communicate over the network to form a highly available, consistent service cluster.

Initially, Meituan used vertical service architectures per business domain; later, it adopted layered services for availability, and eventually evolved to micro‑services, emphasizing gradual evolution rather than premature design.

Distributed System Practices

The typical distributed system structure in Meituan leverages public components and services to achieve partition scaling, disaster recovery, and monitoring. Front‑end traffic is balanced by HLB; services communicate via OCTO for registration, discovery, load balancing, fault tolerance, and gray releases, with optional message queues like Kafka or RabbitMQ. Storage accesses distributed databases via Zebra, monitoring uses the in‑house CAT system, caching combines Squirrel+Cellar, and task scheduling is handled by Crane.

Key challenges include cluster scalability—stateful clusters scale poorly—and hotspot nodes causing uneven resource and CPU usage.

Solutions include converting stateful nodes to stateless ones and using parallel computation to enable rapid scaling; ensuring data consistency by using Databus, a high‑availability, low‑latency, high‑concurrency change‑data‑capture system that propagates binlog changes to downstream stores; and guaranteeing high availability through pre‑emptive capacity testing, periodic health checks, fault injection drills, real‑time alerts, rapid fault localization, and post‑incident rollback, throttling, circuit breaking, and degradation strategies.

Rapid Single‑IDC Deployment & Disaster Recovery

After a single IDC failure, entry services detect faults and automatically switch traffic; rapid IDC expansion synchronizes data and services in advance, enabling traffic to be opened once ready. All data‑sync and traffic‑distribution services must support automatic fault detection and removal, with scaling performed per IDC.

Multi‑Center Attempts

To overcome resource saturation in a single IDC partition, Meituan groups multiple IDCs into a virtual center, deploying services uniformly across the center; when capacity is insufficient, new IDCs are added to expand.

Unit‑Based Attempts

Unit‑based deployment offers finer‑grained partition disaster recovery and scaling. Traffic routing is based on regions or cities, while cross‑region data sync may introduce latency. SET disaster recovery ensures rapid failover to other SETs when a local or remote SET encounters issues.

Core Intelligent Logistics Technologies and Platform Accumulation

The Machine Learning Platform provides an end‑to‑end environment for model training and algorithm deployment, addressing repetitive development and inconsistent data quality across online and offline sources.

JARVIS is an AIOps platform focused on stability, consolidating noisy alerts, reducing manual fault analysis, and improving response speed and reliability in high‑concurrency distributed environments.

Future Challenges

Future challenges include the growing complexity of micro‑services, network amplification effects from minor latency, rapid fault localization in intricate service topologies, and the shift from cluster‑level to unit‑level operations, which demands advanced deployment capabilities.