Designing Million‑QPS Architecture: LVS + Nginx + Keepalived Three‑Layer Load Balancing

Problem Statement

In large‑scale services, a single server or a simple load‑balancer cannot absorb traffic spikes nor guarantee continuous availability when the request volume reaches the million‑QPS level. The article therefore proposes a layered architecture that separates traffic scheduling, application processing, and failover responsibilities.

Three‑Layer Architecture

The solution consists of three logical layers:

Layer 1 – LVS + Keepalived : Provides four‑layer (TCP/UDP) load balancing with extremely high packet‑forwarding efficiency. Keepalived ensures high‑availability by moving the virtual IP (VIP) when any node fails.

Layer 2 – Nginx Cluster : Acts as a seven‑layer reverse proxy, handling HTTP/HTTPS, performing fine‑grained routing based on domain, URL or header, and offering features such as SSL termination, caching and rate limiting.

Layer 3 – Backend Service Cluster : Hosts business services (e.g., Tomcat, Golang, Spring Boot). Nginx can further distribute traffic among these instances.

LVS + Keepalived Configuration (Layer 1)

Two LVS nodes are deployed in master‑slave mode using Direct‑Routing (DR) for maximum performance. Keepalived monitors the Nginx processes and performs VIP failover via VRRP.

global_defs {
    router_id LVS_MASTER
}

vrrp_script chk_nginx {
    script "/etc/keepalived/check_nginx.sh"
    interval 2
    weight -20
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.100/24
    }
    track_script {
        chk_nginx
    }
}

virtual_server 192.168.1.100 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    protocol TCP
    real_server 192.168.1.10 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
    # Add other Nginx real_server entries here
}

Nginx Cluster Configuration (Layer 2)

The Nginx upstream uses a least‑connection (or ip_hash) strategy, with health‑check parameters and keep‑alive connections to reduce latency.

upstream backend {
    least_conn;               # or ip_hash
    server app1:8080 weight=5 max_fails=3 fail_timeout=30s;
    server app2:8080 weight=5;
    keepalive 32;              # connection reuse
}

server {
    listen 80;
    server_name _;
    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
    # static files, cache, rate‑limit etc. can be added here
}

Kernel Tuning for LVS/Nginx Nodes (Layer 3)

Key sysctl parameters are adjusted to enlarge the port range, increase the maximum connection backlog, and optimize ARP handling to avoid split‑brain scenarios.

# Expand port range
net.ipv4.ip_local_port_range = 1024 65535

# Increase connection backlog
net.core.somaxconn = 65535

# ARP optimization
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2

By combining these three layers—high‑performance packet forwarding (LVS), flexible application‑level routing (Nginx), and robust failover (Keepalived)—the architecture can sustain million‑level concurrent connections while maintaining low latency and high availability.