Load Balancing: Concepts, Mechanisms, and Enterprise Practices

Introduction

High‑performance, high‑availability systems typically employ multiple layers of load balancing, implemented either in hardware or software. The article adopts a top‑down approach to explain each layer.

Stateless Service vs. Stateful Data Load Balancing

Stateless services (e.g., web or app servers) can route requests to any backend because the servers are interchangeable; routing policies are usually business‑agnostic and based on backend load. Stateful data services (e.g., cache, MQ, databases) cannot be load‑balanced in the same way due to write‑pressure and consistency constraints, so sharding is used and routing becomes business‑aware.

Stateless Service Load Balancing

1. Front‑end Web Server Load Balancing

First layer often uses DNS‑based load balancing (e.g., round‑robin DNS or GSLB). Requests then reach a load‑balancer server, which can be hardware (F5) or software (HAProxy, LVS, etc.). Software balancers may run in user space (HAProxy, Nginx – Layer‑7) or be compiled into the kernel (LVS – Layer‑4).

2. Backend Application Server Load Balancing

Backend services communicate via RPC and often use a configuration‑center plus client‑side load balancing (CLB). Agents cache service registry data locally, and dynamic weights are calculated from request success rate and latency, providing fault tolerance, overload protection, and automatic failover (e.g., via ZooKeeper).

Stateful Data Load Balancing

Database Load Balancing

SQL parsing can be treated as stateless, but the real bottleneck is storage I/O. Scaling out is achieved through sharding (vertical then horizontal using hash‑based keys). Middleware such as Cobar or HAProxy can hide sharding complexity, but routing keys must be included in every query.

Cache Load Balancing

Caches behave like key‑value stores; consistent hashing is used for mapping. Write‑capacity can be increased by re‑sharding, while read‑capacity scales by adding more shard groups, balancing the trade‑off between write performance and complexity.

File System Load Balancing

Large binary files are stored in distributed file systems (e.g., MogileFS, GFS). Metadata is often kept in a database, while file data is distributed and replicated across multiple nodes.

Enterprise Case Studies

Alibaba B2B

Global DNS + F5 hardware LB, Apache+JBoss web/app servers, Dubbo for service routing, Memcached for distributed cache, MySQL sharding for DB scale‑out, and NFS‑based file storage.

Tencent ECC

Global GSLB+DNS, LVS kernel‑level LB, Nginx+Tomcat separated web/app layers, internal IDL framework for service routing, Memcached cache, MySQL sharding, and TFS distributed file system.

Additional Topics

LVS

LVS is a fast IP‑based load balancer compiled into the Linux kernel, often paired with Keepalived for health checking.

DNS Records

Explanation of common DNS record types (A, CNAME, MX, TXT, TTL) and their role in GSLB.

GSLB

Global Server Load Balancing distributes traffic across geographic regions using DNS, redirection, or routing‑protocol methods, with static or dynamic IP‑to‑node mappings.

Triangular Load‑Balancing Strategy

Describes client → balancer → backend flow and two return paths: via balancer (store‑and‑forward/NAT) or direct routing. Direct routing and IP tunneling are explained as ways to avoid the balancer becoming a bottleneck.

ECC L5 Component Features

Provides name service, dynamic load balancing based on success rate and latency, fault tolerance, overload protection, and local agent caching. Core algorithms include dynamic weight calculation, threshold scaling, and failure detection.