Master ipvsadm: Build LVS/NAT & DR Load‑Balancing Clusters on Linux

Table of Contents

1. ipvsadm Command

The ipvs cluster service can use protocols such as tcp, udp, ah, esp, ah_esp, and sctp.

A single ipvs host can define multiple cluster services.

Each cluster service should have at least one Real Server.

1.1 Syntax

Manage cluster services

# Add or modify a virtual server record
ipvsadm -A|E -t|u|f virtual-service-address:port [-s scheduler] [-p[timeout]] [-M netmask]

# Delete a virtual server record
ipvsadm -D -t|u|f virtual-service-address

# Clear a virtual server record
ipvsadm -C

# Restore a virtual server record
ipvsadm -R

# Save a virtual server record
ipvsadm -S [-n]

Manage Real Servers within a cluster

# Add or modify a real server in a virtual server
ipvsadm -a|e -t|u|f virtual-service-address:port -r real-server-address:port [-g|i|m] [-w weight]

# Delete a real server from a virtual server
ipvsadm -d -t|u|f virtual-service-address -r server-address

Other options

# Show kernel virtual server table
ipvsadm -L|l [options]
# Reset counters (clear current connections)
ipvsadm -Z [-t|u|f virtual-service-address]
# Set connection timeout values
ipvsadm --set [tcp tcpfin udp]
# Start sync daemon
ipvsadm --start-daemon state [--mcast-interface interface]
# Stop sync daemon
ipvsadm --stop-daemon
# Show help
ipvsadm -h

Related terms

scheduler        Scheduling method
real-service-address  Real server IP address
virtual-service-address  Virtual server IP address

1.2 Command Options

1.3 Additional Options

2. LVS/NAT Example

Configuration can be scripted for automation.

2.1 Environment Configuration

Director Server

# DS has two NICs: bridge‑mode VIP and vmnet2 custom‑mode DIP
[root@localhost ~]# vim /etc/sysconfig/network-scripts/ifcfg-eno16777736
... (static IP configuration) ...
BOOTPROTO=static
IPADDR=172.16.100.9
NETMASK=255.255.0.0
GATEWAY=172.16.0.1
NAME=eno16777736
...
# Configure second NIC
[root@localhost ~]# cp ifcfg-eno16777736 ifcfg-eno33554992
[root@localhost ~]# vim ifcfg-eno33554992
... 
IPADDR=192.168.20.1
NETMASK=255.255.255.0
NAME=eno33554992
DEVICE=eno33554992
...
# Restart network on CentOS 7
[root@localhost ~]# systemctl restart network.service

# Install ipvsadm if missing
[root@localhost ~]# yum install -y ipvsadm

Real Server

# RS1: configure static IP on vmnet2
[root@localhost ~]# setup
Name
Device
Use DHCP
Static IP         192.168.20.7
Netmask           255.255.255.0
Default gateway IP 192.168.20.1
...
# Restart network on CentOS 6
[root@localhost ~]# service network restart
# Create test page
[root@localhost ~]# echo 'nodeA' > /var/www/html/index.html

# RS2: configure static IP, adjust cloned interface
[root@localhost ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0
# Remove HWADDR and UUID, set IPADDR=172.16.100.8
...
# Update udev rules to rename eth1 to eth0
[root@localhost ~]# vim /etc/udev/rules.d/70-persistent-net.rules
...
# Reload NIC driver
[root@localhost ~]# modprobe -r e1000
modprobe e1000
# Restart network on CentOS 6
[root@localhost ~]# service network restart
# Create test page
[root@localhost ~]# echo 'nodeB' > /var/www/html/index.html

2.2 Load‑Balancing Configuration

# Enable IP forwarding
# Method 1
[root@localhost ~]# echo '1' > /proc/sys/net/ipv4/ip_forward
# Method 2 (persistent)
[root@localhost ~]# vim /etc/sysctl.conf
net.ipv4.ip_forward = 1

# Add virtual service rule (VIP 172.16.100.9:80, round‑robin)
[root@localhost ~]# ipvsadm -A -t 172.16.100.9:80 -s rr
# View rules
[root@localhost ~]# ipvsadm -L -n
Forwark  Weight  ActiveConn  InActConn
Masq     1       0           0
# Add Real Servers
[root@localhost ~]# ipvsadm -a -t 172.16.100.9:80 -r 192.168.20.7 -m
[root@localhost ~]# ipvsadm -a -t 172.16.100.9:80 -r 192.168.20.8 -m
# Verify
[root@localhost ~]# ipvsadm -L -n

# Persist rules to /etc/sysconfig/ipvsadm
[root@localhost ~]# ipvsadm -S /etc/sysconfig/ipvsadm
# Reload rules
[root@localhost ~]# ipvsadm -R < /etc/sysconfig/ipvsadm
# Save current rules
[root@localhost ~]# ipvsadm-save
# Clear all rules
[root@localhost ~]# ipvsadm -C
# Restore rules
[root@localhost ~]# ipvsadm-restore
# Unit file for CentOS 7
cat /usr/lib/systemd/system/ipvsadm.service

# Test: change scheduler to sh
[root@localhost ~]# ipvsadm -E -t 172.16.100.9:80 -s sh
# Test: delete scheduler
[root@localhost ~]# ipvsadm -D -t 172.16.100.9:80
# View rules
[root@localhost ~]# ipvsadm -L -n
# Test: modify Real Server
[root@localhost ~]# ipvsadm -e -t 172.16.100.9:80 -r 192.168.20.7:8080 -m
# Test: delete Real Server
[root@localhost ~]# ipvsadm -d -t 172.16.100.9:80 -r 192.168.20.7:8080
# View rules
[root@localhost ~]# ipvsadm -L -n

3. LVS/DR Example

Configuration can also be scripted.

3.1 Problem Solving

Problem: How to make only the Director Server (DS) respond to VIP requests?

Method 1: Add a static MAC entry for the VIP on the router (may be ISP‑controlled).

Method 2: On Real Servers, configure arpiptables to block ARP replies for the VIP.

Method 3: Configure network interfaces so DS uses a physical NIC for DIP and an alias for VIP; RS uses a physical NIC for RIP and a loopback alias for VIP.

Set kernel parameters to ignore ARP requests for the VIP: arp_ignore=0: respond to any local address. arp_ignore=1: respond only to requests for the interface address. arp_announce=0: announce all networks. arp_announce=1: avoid announcing non‑local addresses. arp_announce=2: announce only on the local network.

3.2 Environment Configuration

Director Server

# Configure DS DIP interface
[root@localhost ~]# ifconfig eno16777736:0 172.16.100.10/32 broadcast 172.16.100.10 up
# Add route for VIP
[root@localhost ~]# route add -host 172.16.100.10 dev eno16777736:0

Real Server

# Configure RS RIP interfaces
# A: 172.16.100.21 (gateway 172.16.0.1)
# B: 172.16.100.22 (gateway 172.16.0.1)

# Configure loopback VIP on RS
[root@localhost ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@localhost ~]# echo 1 > /proc/sys/net/ipv4/conf/eth0/arp_ignore
[root@localhost ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@localhost ~]# echo 2 > /proc/sys/net/ipv4/conf/eth0/arp_announce
[root@localhost ~]# ifconfig lo:0 172.16.100.10/32 broadcast 172.16.100.10 up
[root@localhost ~]# route add -host 172.16.100.10 dev lo:0
# Verify ARP entries
[root@localhost ~]# arp -a

3.3 Load‑Balancing Configuration

# Create DS virtual server for DR model
[root@localhost ~]# ipvsadm -A -t 172.16.100.10:80 -s rr
# Add Real Servers
[root@localhost ~]# ipvsadm -a -t 172.16.100.10:80 -r 172.16.100.21 -g
[root@localhost ~]# ipvsadm -a -t 172.16.100.10:80 -r 172.16.100.22 -g

4. Extended Exercises

LVS‑NAT

Task: Implement HTTP and HTTPS load‑balancing clusters using the NAT model.

Notes:

HTTPS requires long‑connection handling.

Both web servers must share the same private key and certificate.

In production, Nginx or HAProxy is usually preferred over LVS.

LVS‑DR

Task: Implement DR model load‑balancing clusters for HTTP, HTTPS, and MySQL.

Plan topology where VIP and RIP are on different networks.

Notes:

HTTPS and MySQL are long‑connection services.

When configuring SSL, both Real Servers must use identical private keys and certificates.