MGW: Meituan's High‑Performance Four‑Layer Load Balancer

This article, originally presented at Meituan‑Dianping Technical Salon, introduces MGW, a self‑developed four‑layer (L4) load balancer that replaces the legacy LVS solution for handling tens of Gbps and millions of concurrent connections.

What is load balancing? Early Internet services used single servers; as traffic grew, multi‑server scaling and fault tolerance became necessary. Load balancers sit at the traffic entry point, distributing requests to backend servers and performing health checks.

Four‑layer load balancing works at the transport layer (forwarding packets by rewriting address fields), while seven‑layer operates at the application layer (proxying traffic). Four forwarding modes are described: DR (direct routing), NAT, TUNNEL, and FULLNAT, each with different network requirements.

Why develop a custom L4 balancer? Hardware LB costs are high, and LVS showed performance bottlenecks and rising operational overhead as Meituan’s traffic exploded.

Achieving high performance :

Interrupt reduction: processing 6 M packets/s would otherwise generate 1 M hardware interrupts, causing cache misses. MGW uses a polling‑mode driver and kernel bypass via DPDK to eliminate most interrupts.

Lock elimination: RSS distributes packets across NIC queues, but FULLNAT can break flow affinity. MGW assigns a dedicated localip per CPU and aligns SNAT ports so that request and response packets stay on the same CPU, removing the need for session locks.

Context‑switch isolation: control‑plane and data‑plane run on separate CPU sets, with data‑plane threads pinned to dedicated cores, preventing interference.

Ensuring high reliability :

Cluster reliability: MGW forms an OSPF+ECMP cluster. ECMP spreads flows, while OSPF quickly removes failed nodes. Session state is fully synchronized across nodes, so flow path changes do not break existing connections.

Failover and expansion: a pre‑online state allows new nodes to receive a complete session sync before becoming visible to the network, avoiding traffic loss during scaling.

Node smooth offline: existing connections continue while new ones are stopped; once drained, the node can be safely upgraded.

Consistent source‑IP hash scheduler: a custom hash ring with virtual nodes guarantees that only the affected subset of connections remap when backend servers change, preserving client‑to‑server affinity.

Summary : MGW demonstrates that a software‑based L4 load balancer built on DPDK can meet the high‑performance, high‑reliability demands of large‑scale Internet services, supporting diverse workloads from long‑lived connections to web traffic, and is poised to evolve toward L7 capabilities.