Master Nginx, Keepalived, and LVS: Build a High‑Availability Load‑Balancing Cluster

Nginx Installation

1. Download the appropriate Nginx package from the official website (prefer the stable version).

2. Upload the package to the Linux server.

3. Install required dependencies:

yum install gcc-c++

yum install -y pcre pcre-devel

yum install -y zlib zlib-devel

yum install -y openssl openssl-devel

4. Extract the source code: tar -zxvf nginx-1.16.1.tar.gz 5. Create a temporary directory for Nginx: mkdir -p /var/temp/nginx 6. Configure the build (creates the Makefile):

./configure \
    --prefix=/usr/local/nginx \
    --pid-path=/var/run/nginx/nginx.pid \
    --lock-path=/var/lock/nginx.lock \
    --error-log-path=/var/log/nginx/error.log \
    --http-log-path=/var/log/nginx/access.log \
    --with-http_gzip_static_module \
    --http-client-body-temp-path=/var/temp/nginx/client \
    --http-proxy-temp-path=/var/temp/nginx/proxy \
    --http-fastcgi-temp-path=/var/temp/nginx/fastcgi \
    --http-uwsgi-temp-path=/var/temp/nginx/uwsgi \
    --http-scgi-temp-path=/var/temp/nginx/scgi

7. Compile and install:

make
make install

8. Manage Nginx in the sbin directory:

nginx          # start
./nginx -s stop   # stop
./nginx -s reload # reload

Configure Reverse Proxy

upstream backend {
    server 192.168.1.173:8080;
    server 192.168.1.174:8080;
    server 192.168.1.175:8080;
}

server {
    listen 80;
    server_name www.tomcats.com;
    location / {
        proxy_pass http://backend;
    }
}

Load‑Balancing Methods

Nginx uses round‑robin by default. Additional methods include:

Weighted round‑robin (set weight in the upstream block).

IP hash ( ip_hash) – binds a client IP to a specific server.

URL hash – hash based on request URL.

Least connections ( least_conn).

upstream Directive Parameters

max_conns

: limit maximum simultaneous connections. slow_start: gradually increase weight over a period (seconds). down: mark server as unavailable. backup: designate a backup server. max_fails: number of failed attempts before marking a server down. fail_timeout: time window for failure counting.

SSL Configuration (HTTPS)

Ensure the http_ssl_module is compiled. Copy the certificate ( *.crt) and private key ( *.key) to /usr/local/nginx/conf. Add a server block listening on port 443:

server {
    listen 443;
    server_name www.imoocdsp.com;
    ssl on;
    ssl_certificate 1_www.imoocdsp.com_bundle.crt;
    ssl_certificate_key 2_www.imoocdsp.com.key;
    ssl_session_cache shared:SSL:1m;
    ssl_session_timeout 5m;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE;
    ssl_prefer_server_ciphers on;
    location / {
        proxy_pass http://tomcats/;
        index index.html index.htm;
    }
}

High‑Availability with Keepalived

Install Keepalived, then create /etc/keepalived/keepalived.conf:

global_defs {
    router_id keep_171
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 100
    advert_int 2
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.161
    }
}

Start Keepalived:

systemctl start keepalived.service

Health‑Check Script for Nginx

#!/bin/bash
A=$(ps -C nginx --no-header | wc -l)
if [ $A -eq 0 ]; then
    /usr/local/nginx/sbin/nginx
    sleep 3
    if [ $(ps -C nginx --no-header | wc -l) -eq 0 ]; then
        killall keepalived
    fi
fi

Make the script executable and reference it in Keepalived:

chmod +x /etc/keepalived/check_nginx_alive_or_not.sh

vrrp_script check_nginx_alive {
    script "/etc/keepalived/check_nginx_alive_or_not.sh"
    interval 2
    weight 10
}

track_script {
    check_nginx_alive
}

LVS (Linux Virtual Server) High‑Availability Load Balancing

Why combine LVS with Nginx? LVS operates at layer 4, offering higher throughput; Nginx handles layer 7 processing.

LVS Modes

NAT : All traffic passes through the LVS node.

TUN : Packets are tunneled; responses bypass LVS.

DR : Direct routing; servers reply directly to clients.

DR Mode Setup (Example)

Disable NetworkManager to avoid interface conflicts.

systemctl stop NetworkManager
systemctl disable NetworkManager

Create a virtual IP on the LVS node (e.g., 192.168.1.150):

cp ifcfg-ens33 ifcfg-ens33:1
# edit ifcfg-ens33:1
DEVICE="ens33:1"
ONBOOT="yes"
IPADDR=192.168.1.150
NETMASK=255.255.255.0
BOOTPROTO=static

Install ipvsadm: yum install ipvsadm Configure real servers with a loopback alias (e.g., 192.168.1.150) and set ARP parameters in /etc/sysctl.conf:

net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2

Apply with sysctl -p.

Add a host route for the virtual IP:

route add -host 192.168.1.150 dev lo:1
echo "route add -host 192.168.1.150 dev lo:1" >> /etc/rc.local

Define LVS scheduling rules using ipvsadm (round‑robin example):

ipvsadm -A -t 192.168.1.150:80 -s rr -p 5
ipvsadm -a -t 192.168.1.150:80 -r 192.168.1.171:80 -g
ipvsadm -a -t 192.168.1.150:80 -r 192.168.1.172:80 -g
ipvsadm -S   # save rules

Keepalived + LVS + Nginx HA Cluster

Clear existing LVS rules on each node: ipvsadm -C Configure Keepalived on the master node:

global_defs {
    router_id keep_151
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 41
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress { 192.168.1.150 }
}

virtual_server 192.168.1.150 80 {
    delay_loop 6
    lb_algo rr
    lb-kind DR
    persistence_timeout 5
    protocol TCP
    real_server 192.168.1.171 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 2
            nb_get_retry 2
            delay_before_retry 3
        }
    }
    real_server 192.168.1.172 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 2
            nb_get_retry 2
            delay_before_retry 3
        }
    }
}

Configure the backup node (change state BACKUP and lower priority).

global_defs { router_id keep_152 }

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 41
    priority 50
    advert_int 1
    authentication { auth_type PASS auth_pass 1111 }
    virtual_ipaddress { 192.168.1.150 }
}

# same virtual_server block as master

Restart Keepalived on both nodes:

systemctl restart keepalived