Mastering Load Balancing: From Basics to Advanced Algorithms and Tools

What is Load Balancing

Load balancing distributes incoming client requests across a set of backend servers (or other resources) to maximize throughput, minimize response latency, and avoid overload. A single public entry point (e.g., www.example.com) forwards traffic to a cluster of real servers transparently.

OSI Layers for Load Balancing

Layer 2 (Data‑Link) : The balancer presents a virtual IP (VIP) while each server keeps a unique MAC address. The balancer rewrites the destination MAC address to forward packets.

Layer 3 (Network) : Servers have distinct IP addresses. The balancer selects a target IP according to the configured algorithm.

Layer 4 (Transport) : Operates on TCP/UDP ports. The balancer modifies the IP + port tuple before forwarding to the chosen server.

Layer 7 (Application) : Inspects application‑level protocols (HTTP, DNS, etc.) and can route based on URL, headers, browser type, language, etc.

Common Software Load‑Balancers

LVS (Linux Virtual Server)

LVS is an open‑source, high‑performance layer‑4 balancer that runs on Linux (kernel 2.6+ without special configuration) or FreeBSD. It consists of three logical layers:

Load‑Balancer (Director) : Receives client traffic, holds routing tables, and monitors real servers via ldirectord.

Server Array (Real Servers) : The actual application hosts (Web, Mail, DNS, etc.).

Shared Storage : Optional shared filesystem (NFS, GFS, OCFS2) for content consistency.

Nginx

Nginx is a lightweight web server that also functions as a reverse proxy and layer‑7 load balancer. It supports up to 50 k concurrent connections out‑of‑the‑box, hot configuration reloads, and a rich set of load‑balancing methods (round‑robin, least‑connections, IP‑hash, etc.).

HAProxy

HAProxy is a C‑based, open‑source solution providing high availability and both TCP and HTTP load balancing. It is widely used for high‑traffic sites and operates primarily at layer 7, offering features such as health checks, stickiness, and ACL‑based routing.

Load‑Balancing Algorithms

Algorithms are classified as static or dynamic.

Static algorithms (do not consider runtime metrics):

Round Robin – cycles through servers sequentially.

Weighted Round Robin – each server receives a weight; higher‑weight servers get proportionally more requests.

Priority – traffic is sent to the highest‑priority group; lower groups are used only when the higher group fails.

Dynamic algorithms (use real‑time metrics such as connection count, response time, CPU, I/O):

Least Connections – forwards to the server with the fewest active connections.

Fastest Response – selects the server with the lowest recent response latency.

Observed – balances based on a combination of connection count and response time.

Predictive – forecasts future performance from recent metrics.

Dynamic Ratio‑APM – adjusts traffic share according to application‑level performance counters.

Dynamic Server Add‑On – automatically adds backup servers when the primary pool shrinks.

QoS / ToS – routes based on service‑class or type‑of‑service fields.

Rule‑Based – custom ACLs or header‑based routing.

Java Reference Implementations of Common Algorithms

The following images illustrate concise Java implementations for four typical algorithms. The code is self‑contained and can be copied into a utility class that implements a selectServer() method returning the index of the chosen backend.