Mastering Software Load Balancing: Nginx, LVS, and HAProxy Deep Dive

Presentation Outline

Software load balancing overview

Nginx load balancing

LVS load balancing

HAProxy load balancing

Summary

1. Software Load Balancing Overview

Hardware load balancers are high‑performance but expensive, so software load balancers such as Nginx, LVS, and HAProxy are widely used on the Internet. This article references many documents, some copied directly.

2. Nginx Load Balancing

Nginx is a lightweight web server / reverse‑proxy that operates at the HTTP layer. It offers high performance, high concurrency, low memory usage, and can handle up to 50,000 concurrent connections.

Supported operating systems: Linux, Windows, FreeBSD, Solaris, macOS, AIX, etc.

Implemented in C.

Officially supports 50,000 concurrent requests per second; typical deployments achieve 20,000, with optimizations reaching 100,000.

2.1 Features

1. Modular design – easy extension via modules.

2. High reliability – master and worker processes synchronize; a failing worker is immediately restarted.

3. Low memory consumption – 10,000 keep‑alive connections use only ~2.5 MB.

4. Hot deployment – configuration can be reloaded without stopping the server.

5. Strong concurrency – up to 50,000 requests per second.

6. Rich functionality – powerful reverse‑proxy and flexible load‑balancing strategies.

2.2 Functions

2.2.1 Basic Functions

Static web serving.

Reverse proxy and load balancing for HTTP, SMTP, POP3.

FastCGI (FPM) support.

Modular filters for compression, SSL, image resizing, etc.

Built‑in health checks.

Name‑ and IP‑based virtual hosts.

Custom access logs.

Smooth upgrades.

Keep‑alive support.

URL rewrite.

Path aliasing.

IP‑ and username‑based access control.

Rate limiting and connection limiting.

2.2.2 Extended Functions

(Content omitted for brevity.)

2.2.3 Performance

Official tests show 50,000 concurrent connections; production environments typically reach 20‑30 k. 10,000 idle keep‑alive connections consume about 2.5 MB. On a 24 GB machine, Nginx can handle up to 2 million concurrent requests.

2.3 Architecture

2.3.1 Basic Working Model

A master process (running as root to bind port 80) spawns one or more worker processes. The master loads configuration and performs graceful upgrades; workers handle the actual request processing. Modules are chained in a pipeline, each performing a specific task such as header parsing, data lookup, or compression.

Hot deployment works by the master reloading the configuration, allowing existing workers to finish current requests before being replaced by new workers with the updated settings.

2.3.2 Sendfile Mechanism

Sendfile lets the kernel transfer file data directly to the network socket, bypassing user‑space copying and reducing overhead, which is essential for high‑concurrency scenarios.

2.3.3 I/O Multiplexing Model

Supports epoll, kqueue, rt signals, /dev/poll, event ports, select, and poll, as well as features like sendfile, AIO, DIRECTIO, accept‑filters, and TCP_DEFER_ACCEPT.

2.4 Load‑Balancing Strategies

Nginx provides built‑in strategies (weighted round‑robin, IP hash) and many extensible ones (fair, generic hash, consistent hash). The following examples use source code from nginx‑1.0.15.

2.4.1 Weighted Round Robin

The algorithm first assigns requests to higher‑weight servers until their weight drops below others, then moves to the next server. If all back‑ends are down, Nginx resets flags to avoid a total timeout.

2.4.2 IP Hash

IP hash distributes requests based on client IP, ensuring the same client is directed to the same backend.

2.4.3 Fair

The fair module selects the backend with the lowest response time, offering strong adaptability but may be unsuitable for unstable networks.

2.4.4 Generic Hash & Consistent Hash

Generic hash uses a variable as the key; consistent hash implements a hash ring, useful for memcached.

2.5 Deployment Scenarios

2.5.1 Front‑End Load Balancing

Nginx sits at the edge, forwarding requests to application servers or a secondary load balancer such as LVS.

2.5.2 Internal Load Balancing

LVS receives external traffic and distributes it to multiple Nginx instances, which then route to specific application servers.

2.5.3 High Availability

Typical HA setup includes two Nginx nodes (master and backup) monitored by Keepalived, sharing a virtual IP (VIP) that clients connect to.