Mastering Load Balancing: Types, Architectures, and Algorithms Explained

Single servers have a performance ceiling; when they cannot meet business demand, a high‑performance cluster is required to increase overall processing capacity.

The essence of a high‑performance cluster is to add more servers, but the complexity lies in task distribution, which requires a task allocator (commonly called a load balancer) and an appropriate allocation algorithm.

Load balancing is not only about equalizing load across compute units; different algorithms may prioritize load, performance, or business requirements.

Load Balancing Types

Common load‑balancing systems include DNS load balancing, hardware load balancing, and software load balancing.

DNS Load Balancing

DNS is the simplest and most common method, typically used for geographic‑level balancing (e.g., users in the north are directed to a Beijing data center, users in the south to a Shenzhen data center). DNS can return different IP addresses for the same domain name.

Simple DNS load‑balancing diagram:

Advantages:

Simple and low cost: the DNS server handles balancing without additional development or hardware.

Proximity improves access speed: DNS can resolve to the nearest server based on the client’s IP.

Disadvantages:

Updates are slow due to DNS caching.

Poor scalability: control resides with the domain registrar.

Limited algorithms and no awareness of backend server health.

Some companies implement private HTTP‑DNS systems to overcome these drawbacks for latency‑sensitive services.

Hardware Load Balancing

Hardware load balancers are dedicated network devices (e.g., F5, A10) that provide comprehensive, high‑performance balancing.

Advantages:

Powerful features: support for all OSI layers, many algorithms, and global balancing.

High performance: can handle millions of concurrent connections.

High stability: enterprise‑grade testing and large‑scale deployment.

Integrated security: firewall and DDoS protection.

Disadvantages:

Expensive: even entry‑level units cost thousands of dollars.

Limited extensibility: hardware cannot be easily customized.

Software Load Balancing

Software load balancers use programs such as Nginx (layer‑7) and LVS (layer‑4). Nginx supports HTTP, SMTP, etc., while LVS operates at the transport layer and is protocol‑agnostic.

Performance comparison (approximate): Nginx handles tens of thousands of requests per second, LVS up to hundreds of thousands, and high‑end hardware devices reach millions.

Advantages:

Simple deployment and maintenance.

Low cost: a standard Linux server is sufficient.

Flexibility: choose layer‑4 or layer‑7, extend via plugins (e.g., Nginx modules).

Disadvantages compared with hardware:

Lower performance ceiling.

Fewer built‑in features.

Typically lacks firewall and DDoS protection.

Software solutions can be customized or self‑developed for special business needs.

Typical Load‑Balancing Architecture

Three layers are often combined:

Geographic level: DNS directs users to the nearest data center.

Cluster level: hardware load balancers (e.g., F5) distribute traffic among servers within a data center.

Machine level: software load balancers (e.g., Nginx) route requests to individual servers.

Example diagram:

This three‑layer model is suitable for large‑scale services; smaller applications may use only a subset (e.g., a single Nginx instance).

Load‑Balancing Algorithms

Algorithms can be grouped by their primary goal:

Task‑distribution (e.g., round robin, weighted round robin).

Load‑aware (e.g., least connections, CPU‑based).

Performance‑optimal (e.g., fastest response time).

Hash‑based (e.g., source‑IP hash, session‑ID hash).

Round Robin

Requests are assigned to servers in a fixed order, ignoring server state.

Pros: extremely simple to implement.

Cons: cannot account for server load or capacity differences; a failing server may still receive traffic.

Weighted Round Robin

Each server is assigned a weight (often based on hardware capacity) and receives a proportionate number of requests.

Pros: better utilization of heterogeneous servers.

Cons: still ignores real‑time server health.

Least Load (Least Connections / CPU Load)

Requests are sent to the server with the lowest current load, measured by connections, CPU, or I/O.

Pros: avoids overloading busy servers.

Cons: requires monitoring infrastructure and adds algorithmic complexity.

Performance‑Optimal

Requests are directed to the server with the fastest observed response time.

Pros: improves client‑perceived latency.

Cons: needs frequent response‑time sampling and careful selection of sampling rate and evaluation period.

Hash‑Based

Requests are hashed on a key (source IP, session ID, etc.) so that the same key consistently maps to the same server, useful for stateful services.

Pros: maintains session affinity without external storage.

Cons: may lead to uneven distribution if hash keys are skewed.