Mastering Load Balancing: Lessons from Alibaba’s VIPServer Journey

Introduction

The author, an Alibaba Cloud senior technical expert, shares practical experiences and reflections on load balancing, inspired by the legacy of LVS and the development of Alibaba’s own VIPServer product.

What Is Load Balancing?

Load balancing distributes incoming traffic across multiple servers to improve availability, performance, and fault tolerance. The article uses a storytelling analogy to illustrate why naïve scaling can lead to overload.

Scaling Strategies

Scale Up (Vertical) : Adding more powerful servers, often limited by budget and hardware constraints.

Business Splitting : Separating distinct business functions into independent services to reduce contention.

Scale Out and Replicas : Deploying multiple identical instances (replicas) to achieve horizontal scaling and redundancy.

Common Load‑Balancing Techniques

DNS Round‑Robin

DNS can map a single domain name to multiple IP addresses, rotating the response order to spread traffic.

niubility.com.     IN     A     172.168.1.101 IN     A     172.168.1.102 IN     A     172.168.1.103 IN     A     172.168.1.104

Advantages : Easy to implement, no intrusion into applications.

Disadvantages : Session stickiness cannot be guaranteed, DNS caching and TTL cause delayed updates, and fault tolerance is limited.

Dedicated Load Balancer

A dedicated LB sits between clients and backend servers, handling traffic distribution and health monitoring.

Pros : Centralized routing logic, support for algorithms such as round‑robin, weighted round‑robin, and random selection.

Cons : Becomes a single point of failure, adds an extra network hop, and can become a bottleneck for response traffic.

Solutions include moving response streams out of the LB path and deploying active‑standby or active‑active LB clusters.

Health Checks

Health detection is essential for graceful scaling, graceful shutdown, and automatic failover. In elastic cloud environments, precise, multi‑level health checks prevent large‑scale outages.

Why Alibaba Built VIPServer?

Despite LVS’s success, several challenges remain: LB overload, single‑point‑of‑failure risks, extra network hop, multi‑region disaster recovery, handling traffic spikes during events, and cost concerns. VIPServer was created to address these six pain points.

VIPServer Overview

VIPServer is a mid‑tier, layer‑7 load balancer built on a P2P model. It provides dynamic DNS resolution, intelligent traffic scheduling (e.g., same‑room, same‑region priority), multiple health‑check protocols, fine‑grained weight control, and multi‑level disaster‑recovery capabilities. It is widely used across Alibaba’s services such as Alibaba Cloud, DingTalk, and major e‑commerce platforms.

For detailed architecture, refer to the paper “VIPServer: A System for Dynamic Address Mapping and Environment Management.”

Current Limitations and Future Work

VIPServer, being a relatively new system, still has shortcomings. The team encourages community feedback to continuously improve the product.