How to Choose Between LVS, Nginx, and HAProxy: A Practical Performance Comparison

Choosing Based on Traffic Path Instead of Product Name

First map where requests are processed. LVS operates in DR/TUN/NAT mode at L4 without HTTP awareness, suitable for stable L4 distribution and large connection counts; Nginx excels at static files, TLS, HTTP routing, caching, and integration with the web ecosystem; HAProxy offers mature L4 health checks, connection control, HTTP routing, runtime statistics, and configuration validation. Real‑world architectures often combine them, e.g., cloud LB or LVS for entry distribution and Nginx/HAProxy for TLS and L7 policies.

Recording Node Resources Before Testing

#!/usr/bin/env bash
set -euo pipefail
DIR="./lb-baseline-$(hostname)-$(date +%Y%m%d%H%M%S)"
mkdir -p "$DIR"
uname -a > "$DIR/uname.txt"
nproc > "$DIR/nproc.txt"
free -h > "$DIR/memory.txt"
ulimit -n > "$DIR/nofile.txt"
ip -s link show dev <net‑if> > "$DIR/link.txt"
ethtool -l <net‑if> > "$DIR/channels.txt" 2>&1 || true
sysctl net.core.somaxconn net.ipv4.ip_local_port_range > "$DIR/sysctl.txt"

Keep backend version, service logic, request payload, client count, network path, TLS version, and session reuse consistent. Avoid mixing local curl with remote load‑generator traffic to prevent client‑side bottlenecks.

Establish Identical Backends and Acceptance Endpoints

Backends must return a minimal, distinguishable response so the scheduler’s distribution can be verified. Example Nginx backend for testing (not for production):

server {
    listen 8080;
    server_name _;
    location = /healthz { return 200 "ok
"; }
    location = /whoami {
        default_type text/plain;
        return 200 "$hostname
";
    }
}

Backup and test configuration before reload; preserve existing virtual hosts.

sudo cp -a /etc/nginx/conf.d/backend.conf "/etc/nginx/conf.d/backend.conf.bak.$(date +%Y%m%d%H%M%S)"
sudo nginx -t
sudo systemctl reload nginx
curl -fsS http://127.0.0.1:8080/healthz

Directly query each LB node to record connection errors, handshake time, and response codes, using a short connect timeout to avoid hanging on a dead node.

for host in <backend1IP> <backend2IP> <backend3IP>; do
  curl -sS --connect-timeout 2 --max-time 5 \
       -w "host=${host} code=%{http_code} connect=%{time_connect}
" \
       "http://${host}:8080/healthz" -o /dev/null
done

If the backend requires Host header, TLS SNI, or authentication, ensure all three test configurations provide identical conditions; otherwise the comparison becomes meaningless.

wrk.method = "GET"
wrk.path = "/whoami"
wrk.headers["Host"] = "<test‑domain>"
wrk.headers["Connection"] = "keep-alive"

wrk -t <threads> -c <concurrency> -d <duration> \
    -s request.lua http://<entry‑addr>:<port>/whoami

Confirm wrk’s default behavior, connection reuse, and HTTP version. For HTTP/2, gRPC, long‑lived connections, or large uploads, switch to a tool that fully covers the protocol.

LVS: Kernel L4 Forwarding Capabilities and Prerequisites

LVS uses IPVS in the Linux kernel for L4 scheduling. It does not terminate TLS or inspect URLs; high connection capacity relies on correct director, return path, ARP/ND suppression, session persistence, and health checks. LVS only manages the forwarding table and is usually paired with keepalived for VIP management and health checking.

sudo ipvsadm -Ln --stats --rate

If the command is missing, install ipvsadm via the distro package manager. The --stats --rate output helps verify whether packets pass through the scheduler but does not alone explain business success rates.

sudo ipvsadm -A -t <VIP>:80 -s rr
sudo ipvsadm -a -t <VIP>:80 -r <backend1IP>:8080 -g -w 1
sudo ipvsadm -a -t <VIP>:80 -r <backend2IP>:8080 -g -w 1
sudo ipvsadm -Ln

DR mode changes affect live traffic; production should manage the table declaratively with keepalived and validate changes on a standby VIP or gray‑scale address. Rolling back requires restoring a previously saved IPVS table, not merely deleting a single real server.

sudo ipvsadm-save > ipvs.before.txt
sudo ipvsadm-restore < ipvs.before.txt

Nginx: L7 Capabilities, TLS Termination, and Connection Model

Nginx’s value lies in its ability to perform HTTP‑level routing, caching, compression, rate limiting, rewrites, and TLS termination. When comparing, state whether TLS termination is enabled, gzip/cache is on, clients keep‑alive, and backend connections are reused. Comparing Nginx with HAProxy/LVS that only handle raw TCP yields cost differences, not like‑for‑like performance.

upstream api_pool {
    least_conn;
    server <backend1IP>:8080 max_fails=3 fail_timeout=10s;
    server <backend2IP>:8080 max_fails=3 fail_timeout=10s;
    keepalive 128;
}
server {
    listen 80;
    location / {
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Request-ID $request_id;
        proxy_pass http://api_pool;
    }
}

When Nginx returns 499/502/504, separate client aborts, upstream connection failures, and upstream timeouts. Structured logging helps correlate status codes with upstream timings.

log_format upstream_timing '$request_id $remote_addr "$request" $status '
    'rt=$request_time uct=$upstream_connect_time '
    'uht=$upstream_header_time urt=$upstream_response_time';
access_log /var/log/nginx/access.log upstream_timing;

HAProxy: Observable Scheduling Between L4 and L7

HAProxy can operate in mode tcp for TLS pass‑through or generic TCP proxying, and in mode http for L7 routing. Its maxconn, queue, health checks, slow start, and stats make traffic control explicit. The mode must match the health‑check type; TCP pass‑through cannot use HTTP‑based routing rules.

global
    maxconn 50000
    log /dev/log local0
defaults
    mode http
    timeout connect 5s
    timeout client 30s
    timeout server 30s
frontend fe_http
    bind :80
    default_backend be_api
backend be_api
    balance leastconn
    option httpchk GET /healthz
    http-check expect status 200
    server app1 <backend1IP>:8080 check
    server app2 <backend2IP>:8080 check
listen stats
    bind <mgmt‑IP>:8404
    stats enable
    stats uri /stats

Expose the stats port only to a management‑network ACL; reload configuration without interruption by backing up, testing syntax, and reloading.

sudo cp -a /etc/haproxy/haproxy.cfg "/etc/haproxy/haproxy.cfg.bak.$(date +%Y%m%d%H%M%S)"
sudo haproxy -c -f /etc/haproxy/haproxy.cfg
sudo systemctl reload haproxy
sudo systemctl is-active haproxy

Use the stats socket to query backend state instead of relying on the HTML stats page.

echo 'show stat' | sudo socat stdio /run/haproxy/admin.sock | \
    awk -F, 'NR==1 || $1=="be_api" {print $1,$2,$18,$34}'

For TLS pass‑through with SNI‑based routing, enable TLS hello checks and evaluate encryption‑traffic limits; HTTP ACLs do not apply in mode tcp.

frontend fe_tls
    mode tcp
    bind :443
    default_backend be_tls
backend be_tls
    mode tcp
    balance roundrobin
    option tcp-check
    server tls1 <backend1IP>:443 check
    server tls2 <backend2IP>:443 check

Unified Performance and Stability Measurement Method

A trustworthy comparison includes at least four phases: warm‑up, steady‑state, fault injection, and recovery. Warm‑up clears one‑time effects (connection establishment, cache, JIT). Steady‑state observes multiple concurrency levels. Fault injection validates how the system reacts to backend loss and recovers. Recovery confirms no connection leaks, queue residue, or retry storms. Persist client output, LB configuration hash, kernel baseline, and backend log time range for each run.

#!/usr/bin/env bash
set -euo pipefail
URL="http://<entry‑addr>:<port>/whoami"
THREADS="<threads>"
DURATION="<duration>"
OUT="results-$(date +%Y%m%d%H%M%S)"
mkdir -p "$OUT"
for connections in 100 500 1000 5000; do
  wrk -t "$THREADS" -c "$connections" -d "$DURATION" "$URL" \
      | tee "$OUT/c${connections}.txt"
done

Do not cherry‑pick the best run across rounds. Repeat each concurrency level multiple times, record median and dispersion. When error rate > 0, packet loss on the load‑generator, backend CPU saturation, or outbound bandwidth ceiling occurs, QPS is no longer a usable capacity metric.

Failover and Rollback Should Not Be an Appendix

In an isolated environment, remove a backend and observe whether new requests stop routing to that instance, how existing long‑lived connections behave, and how the system recovers (slow‑start or predefined policy). Do not shut down production backends for testing. Nginx open‑source relies on passive health checks; HAProxy/keepalived can perform active checks, but the check path must reflect true business availability.

echo 'set server be_api/app1 state maint' | sudo socat stdio /run/haproxy/admin.sock
echo 'show servers state' | sudo socat stdio /run/haproxy/admin.sock

After verification, bring the server back to ready state:

echo 'set server be_api/app1 state ready' | sudo socat stdio /run/haproxy/admin.sock

Runtime commands do not automatically persist to configuration files; change records must indicate whether declarative configuration needs updating. LVS/keepalived removal should be performed via an approved configuration change to avoid drift.

Final decision tables should consider more than peak QPS. Qualitative or measured comparisons should include: need for HTTP routing and TLS termination, need for transparent L4 forwarding, backend health‑check semantics, connection queuing and rate‑limiting capability, configuration‑release risk, log and metric visibility, ops team expertise, and failover rollback time. LVS fits clear high‑performance L4 paths; Nginx fits web‑gateway duties; HAProxy fits scenarios requiring fine‑grained proxy control and unified L4/L7 handling. Hybrid deployments are often the correct answer, provided responsibilities, timeouts, retries, and observability do not conflict.