Load Balancing: Principles, Types, and Algorithms

Facing massive user traffic, high concurrency, and large data volumes, a system needs high‑performance servers, large databases, and efficient programming languages (e.g., Go, Scala). When a single machine reaches its limits, business decomposition and distributed deployment become necessary to handle large‑scale website traffic.

Moving from a single‑machine site to a distributed one hinges on splitting services and deploying them across multiple machines, forming a large‑scale distributed system. While this solves centralization issues, each independent service still faces single‑point failures and a unified entry point. Redundancy (multiple instances) mitigates failures, and placing a load‑balancing device before the cluster distributes traffic.

1. Load‑Balancing Principles

System expansion can be vertical (adding CPU, memory, storage to a single server) or horizontal (adding more servers). Horizontal scaling addresses the limitations of vertical scaling for high‑traffic, high‑concurrency sites. A typical cluster‑and‑load‑balancer architecture consists of an application cluster and a load‑balancing device that distributes requests based on algorithms.

The application cluster runs the same service on multiple machines, handling requests forwarded by the load balancer. The load‑balancing device selects a target server according to its algorithm.

Load balancing solves several problems:

Alleviates concurrency pressure and improves throughput.

Provides fault‑tolerance for high availability.

Enables horizontal scaling by adding or removing servers.

Offers security features such as filtering and black‑/white‑listing.

2. Load‑Balancing Classifications

Based on implementation technology, load balancing can be divided into DNS, HTTP, IP, link‑layer, and hybrid types.

2.1 DNS Load Balancing

The earliest technique uses multiple A records in DNS to map a domain to several servers, distributing traffic at the DNS level. Advantages include simplicity and performance gains by directing users to nearby servers. Drawbacks are poor availability (DNS propagation delay) and limited control over algorithms and server health.

Practical advice: use DNS as the first tier, pointing A records to an internal load balancer that then distributes requests to real web servers.

2.3 IP Load Balancing

At the network layer, the load balancer rewrites the destination IP address of incoming packets to a real server's IP, then forwards the packet. The response is sent back to the balancer, which rewrites the source IP to its own address before returning to the client.

Two implementation modes exist: (1) source NAT (SNAT) where the balancer also changes the source address, and (2) the balancer acts as a gateway for the server cluster. Advantages are kernel‑level performance; disadvantages include the balancer becoming a throughput bottleneck.

2.4 Link‑Layer Load Balancing

This method modifies the destination MAC address instead of the IP, allowing servers to share a virtual IP identical to the balancer's IP. Responses can be sent directly to clients, avoiding the balancer’s bandwidth limits (direct‑routing mode).

Pros: high performance. Cons: complex configuration. This mode is widely used today.

2.5 Hybrid Load Balancing

Different server groups may use the most suitable load‑balancing technique, and a higher‑level balancer can distribute traffic among those groups, achieving optimal performance. Hybrid approaches are common in large internet companies.

3. Load‑Balancing Algorithms

Common algorithms include round‑robin, random, least‑connections, source‑IP hash, and weighted methods.

3.1 Round‑Robin

Requests are distributed sequentially across servers. Works well when servers have identical hardware.

3.2 Random

Requests are assigned to servers randomly. Simple but not optimal for heterogeneous environments.

3.3 Least Connections

The balancer selects the server with the fewest active connections, adapting to current load.

3.4 Hash (Source‑IP)

Requests from the same client IP are consistently routed to the same server, providing session stickiness.

3.5 Weighted

Combines basic algorithms with server weights to favor more powerful machines.

4. Hardware Load Balancers

Dedicated appliances (e.g., F5, A10) offer comprehensive features, high performance, stability, and built‑in security (firewall, DDoS protection). They are expensive and have limited scalability, so most internet companies prefer software solutions like Nginx, using hardware only for critical global traffic.

Summary

Hardware load balancers usually require dual‑node high‑availability setups, increasing cost. Software load balancers are widely adopted in internet companies, with hardware used selectively for specific scenarios.