Mastering LVS: Complete Guide to Linux Load Balancing (NAT, DR, TUN) and HA

1. Overview of LVS

Load‑balancing clusters, often abbreviated as LVS, are implemented using open‑source tools such as nginx, LVS, and haproxy, or commercial hardware like F5. This guide focuses on LVS, an open‑source project that is now part of the Linux kernel and enables high‑performance, highly available server farms.

2. LVS Working Principle

When a client request arrives, it is first received by the Director Server (DS). The kernel’s PREROUTING chain captures the packet, then INPUT forwards it to the IPVS module. IPVS matches the packet against defined virtual services, rewrites the destination IP/port, and passes the packet to POSTROUTING, which finally routes it to the selected Real Server (RS).

3. LVS Components

LVS consists of two main programs:

ipvs : kernel‑space code that performs the actual scheduling.

ipvsadm : user‑space utility for configuring ipvs rules and defining virtual services and real servers.

4. Key Terminology

DS – Director Server (frontend load balancer).

RS – Real Server (backend worker).

VIP – Virtual IP presented to clients.

DIP – Director Server IP used for internal communication.

RIP – Real Server IP.

CIP – Client IP.

5. LVS NAT Mode

In NAT mode the Director rewrites both source and destination addresses, so all traffic (request and response) passes through the DS, which can become a bottleneck. The packet flow is:

Client → DS (PREROUTING).

IPVS changes destination to RS IP and forwards to POSTROUTING.

RS processes the request and sends the response back to DS.

DS rewrites the source to VIP and returns the response to the client.

Characteristics:

RS must use private addresses; its gateway points to DIP.

DIP and RIP must be in the same subnet.

All traffic traverses DS, creating a potential performance bottleneck.

Supports port mapping.

RS can run any OS.

Limitation: DS bears the full load of both request and response traffic.

6. LVS Direct‑Routing (DR) Mode

DR mode keeps the client’s source IP unchanged and only rewrites the destination MAC address to that of the selected RS. The packet never leaves the DS after the initial routing decision, eliminating the DS bottleneck for response traffic.

Client → DS (PREROUTING).

IPVS changes the destination MAC to the RS’s MAC while leaving IP addresses untouched.

POSTROUTING forwards the packet directly to the RS (layer‑2).

RS replies using its own IP; the response bypasses the DS.

Characteristics:

DS and RS must reside in the same broadcast domain.

RS can use private or public IPs.

No address translation or port mapping.

RS can run any common OS.

Limitation: DS and RS must be in the same data‑center.

7. LVS TUN Mode

TUN encapsulates the original IP packet inside an outer IP header (source DIP, destination RIP). This allows the DS to forward packets across routed networks while keeping the client’s IP unchanged.

8. Scheduling Algorithms

rr – Simple round‑robin.

wrr – Weighted round‑robin (weights 0‑100).

lc – Least connections.

wlc – Weighted least connections.

lblc – Locality‑based least connections.

lblcr – Locality‑based least connections with server‑group mapping.

dh – Destination‑hash.

sh – Source‑hash.

9. Practical NAT Configuration

Environment : One Director (172.16.254.200, 192.168.0.8) and two Real Servers (192.168.0.18, 192.168.0.28). Both RS run nginx; Director runs ipvsadm.

# yum install -y nginx
# yum install -y ipvsadm
# vim /usr/local/sbin/lvs_nat.sh
#!/bin/bash
# Enable IP forwarding
 echo 1 > /proc/sys/net/ipv4/ip_forward
# Disable ICMP redirects
 echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
 echo 0 > /proc/sys/net/ipv4/conf/default/send_redirects
 echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects
 echo 0 > /proc/sys/net/ipv4/conf/eth1/send_redirects
# NAT firewall rules
 iptables -t nat -F
 iptables -t nat -X
 iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -j MASQUERADE
# ipvsadm configuration
 IPVSADM=/sbin/ipvsadm
 $IPVSADM -C
 $IPVSADM -A -t 172.16.254.200:80 -s wrr
 $IPVSADM -a -t 172.16.254.200:80 -r 192.168.0.18:80 -m -w 1
 $IPVSADM -a -t 172.16.254.200:80 -r 192.168.0.28:80 -m -w 1

Run the script on the Director: /bin/bash /usr/local/sbin/lvs_nat.sh. Verify with ipvsadm -ln. Test by setting different index pages on each RS and accessing http://172.16.254.200.

10. Practical DR Configuration

Environment : Same three hosts, but the Director now uses a virtual IP (VIP 192.168.0.38) bound to eth0:0. RS gateways point to the Director’s internal IP.

# vim /usr/local/sbin/lvs_dr.sh
#!/bin/bash
 echo 1 > /proc/sys/net/ipv4/ip_forward
 ipv=/sbin/ipvsadm
 vip=192.168.0.38
 rs1=192.168.0.18
 rs2=192.168.0.28
 ifconfig eth0:0 down
 ifconfig eth0:0 $vip broadcast $vip netmask 255.255.255.255 up
 route add -host $vip dev eth0:0
 $ipv -C
 $ipv -A -t $vip:80 -s wrr
 $ipv -a -t $vip:80 -r $rs1:80 -g -w 3
 $ipv -a -t $vip:80 -r $rs2:80 -g -w 1

RS side script ( lvs_dr_rs.sh) configures the loopback alias and ARP settings to prevent the RS from answering ARP for the VIP:

# vim /usr/local/sbin/lvs_dr_rs.sh
#!/bin/bash
 vip=192.168.0.38
 ifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 up
 route add -host $vip lo:0
 echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
 echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
 echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
 echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce

Execute the Director script and then the RS script on each real server. Test by browsing http://192.168.0.38. In DR mode the RS gateways do **not** need to point to the Director.

11. LVS with Keepalived for High Availability

Four nodes are used: two Director+keepalived pairs (MASTER 192.168.0.48, BACKUP 192.168.0.58) and two Real Servers (192.168.0.18, 192.168.0.28) sharing VIP 192.168.0.38.

# yum install ipvsadm keepalived -y
# vim /etc/keepalived/keepalived.conf   (MASTER)
vrrp_instance VI_1 {
 state MASTER
 interface eth0
 virtual_router_id 51
 priority 100
 advert_int 1
 authentication {
  auth_type PASS
  auth_pass 1111
 }
 virtual_ipaddress { 192.168.0.38 }
}
virtual_server 192.168.0.38 80 {
 delay_loop 6
 lb_algo rr
 lb_kind DR
 persistence_timeout 0
 protocol TCP
 real_server 192.168.0.18 80 {
  weight 1
  TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 }
 }
 real_server 192.168.0.28 80 {
  weight 1
  TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 }
 }
}

Copy the config to the BACKUP node, change state MASTER to state BACKUP and lower priority to 90. Start keepalived on both nodes ( service keepalived start).

Verification :

Stop nginx on one RS; the VIP continues to serve the remaining RS.

Restart the stopped RS; round‑robin traffic returns to both servers.

Stop keepalived on the MASTER; the VIP migrates to the BACKUP and service remains uninterrupted.

These experiments demonstrate LVS’s load‑balancing capabilities combined with keepalived’s HA failover.