Mastering LVS: A Complete Guide to Linux Load Balancing with NAT, DR, and Keepalived

1. Introduction to LVS

Load balancing clusters, often called load‑balance clusters, are commonly implemented with open‑source software such as Nginx, LVS, and HAProxy, or commercial hardware like F5. This article focuses on learning and summarizing LVS.

2. Basic Working Principle of LVS

When a user request arrives, it is first sent to the Director Server, which distributes the request to real servers based on the configured scheduling algorithm. Shared storage ensures consistent data across servers.

LVS (Linux Virtual Server) is an open‑source project that is now part of the Linux kernel, providing high‑performance, high‑availability server clusters.

3. LVS Components

LVS consists of two programs: ipvs (kernel‑space code that performs scheduling) and ipvsadm (user‑space tool for configuring rules).

4. LVS Terminology

DS: Director Server (frontend load balancer node)

RS: Real Server (backend server)

VIP: Virtual IP address presented to clients

DIP: Director Server IP (used for internal communication)

RIP: Real Server IP

CIP: Client IP

5. LVS/NAT Principle and Features

(a) Client request reaches Director Server and enters the PREROUTING chain (source CIP, destination VIP). (b) PREROUTING forwards the packet to INPUT. (c) IPVS modifies the destination IP to the Real Server IP and forwards to POSTROUTING. (d) POSTROUTING sends the packet to the Real Server. (e) Real Server processes the request and replies to Director Server. (f) Director Server rewrites the source IP to VIP before responding to the client.

RS should use private addresses; its gateway must point to DIP.

DIP and RIP must be in the same subnet.

All request and response packets pass through the Director Server, which can become a bottleneck under high load.

Port mapping is supported.

RS can run any operating system.

Drawback: Director Server bears high load because both request and response traverse it.

6. LVS/DR Principle and Features

(a) Client request reaches Director Server (PREROUTING, source CIP, destination VIP). (b) PREROUTING forwards to INPUT. (c) IPVS changes the source MAC to DIP's MAC and destination MAC to RIP's MAC, then forwards to POSTROUTING. (d) POSTROUTING sends the packet directly to the Real Server (layer‑2 transmission). (e) Real Server receives the packet, processes it, and sends the response back to the client (source VIP, destination CIP). (f) Response reaches the client without passing the Director Server.

All client‑targeted packets (VIP) are sent to the Director Server.

RS can use private or public addresses; if public, it can be accessed directly.

RS and Director Server must be on the same physical network.

Response packets bypass the Director Server.

No address translation or port mapping is supported.

RS can run most operating systems.

RS gateway must not point to DIP.

RS configures VIP on its loopback interface.

Drawback: RS and DS must reside in the same data center.

7. LVS/Tun Principle and Features

In Tun mode, an additional IP header is encapsulated: inner header (source CIP, destination VIP) and outer header (source DIP, destination RIP).

(a) Client request reaches Director Server (PREROUTING, source CIP, destination VIP). (b) PREROUTING forwards to INPUT. (c) IPVS encapsulates a new IP header (source DIP, destination RIP) and forwards to POSTROUTING. (d) POSTROUTING sends the packet through the tunnel to the Real Server. (e) Real Server strips the outer header, processes the request, and sends the response back via its loopback VIP. (f) Response reaches the client.

RIP, VIP, and DIP are public addresses.

RS gateway cannot point to DIP.

All request packets traverse the Director Server; responses bypass it.

Port mapping is not supported.

RS must support tunneling.

8. LVS Scheduling Algorithms

rr – Round Robin: simple cyclic distribution.

wrr – Weighted Round Robin: adds weight to servers.

lc – Least Connections: selects server with fewest active connections.

wlc – Weighted Least Connections: combines weight with connection count.

lblc – Locality‑Based Least Connections: prefers servers close to the client IP.

lblcr – Complex Locality‑Based Least Connections: maintains IP‑to‑server mapping.

dh – Destination Hash: hashes destination IP to a server.

sh – Source Hash: hashes source IP to a server.

9. Practical NAT Mode

Experiment Environment

Three machines: one Director (external IP 172.16.254.200, internal IP 192.168.0.8) and two Real Servers (192.168.0.18 and 192.168.0.28) with their gateways set to the Director's internal IP.

Installation

# yum install -y nginx
# yum install -y ipvsadm

Director Configuration Script (lvs_nat.sh)

#!/bin/bash
# Enable IP forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward
# Disable ICMP redirects
echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
# Set up NAT rule
iptables -t nat -F
iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -j MASQUERADE
# Add LVS service
IPVSADM=/sbin/ipvsadm
$IPVSADM -C
$IPVSADM -A -t 172.16.254.200:80 -s wrr
$IPVSADM -a -t 172.16.254.200:80 -r 192.168.0.18:80 -m -w 1
$IPVSADM -a -t 172.16.254.200:80 -r 192.168.0.28:80 -m -w 1

Running the script configures LVS in NAT mode. Testing is done by accessing http://172.16.254.200 and modifying the default Nginx page on each Real Server to distinguish responses.

10. Practical DR Mode

Experiment Environment

Director node: eth0 192.168.0.8, VIP eth0:0 192.168.0.38

Real server1: eth0 192.168.0.18, VIP lo:0 192.168.0.38

Real server2: eth0 192.168.0.28, VIP lo:0 192.168.0.38

Installation

# yum install -y nginx
# yum install -y ipvsadm

Director DR Script (lvs_dr.sh)

#!/bin/bash
# Enable IP forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward
# Set VIP on Director
ifconfig eth0:0 192.168.0.38 netmask 255.255.255.255 up
# Add route for VIP
route add -host 192.168.0.38 dev eth0
# Configure LVS
IPVSADM=/sbin/ipvsadm
$IPVSADM -C
$IPVSADM -A -t 192.168.0.38:80 -s wrr
$IPVSADM -a -t 192.168.0.38:80 -r 192.168.0.18:80 -m -w 1
$IPVSADM -a -t 192.168.0.38:80 -r 192.168.0.28:80 -m -w 1

Real servers configure the VIP on their loopback interface and adjust ARP settings (arp_ignore, arp_announce) to avoid ARP conflicts.

11. LVS with Keepalived (High Availability)

Two keepalived nodes (MASTER and BACKUP) share a virtual IP 192.168.0.38. Each node runs LVS with the same configuration, and keepalived monitors the real servers via TCP checks. When the MASTER fails, the BACKUP takes over the VIP, ensuring continuous service.

Key configuration snippets:

# yum install -y ipvsadm keepalived
# keepalived.conf (excerpt)
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress { 192.168.0.38 }
    virtual_server 192.168.0.38 80 {
        lb_algo rr
        lb_kind DR
        protocol TCP
        real_server 192.168.0.18 80 {
            weight 1
            TCP_CHECK { connect_timeout 10 nb_get_retry 3 }
        }
        real_server 192.168.0.28 80 {
            weight 1
            TCP_CHECK { connect_timeout 10 nb_get_retry 3 }
        }
    }
}

Testing includes stopping Nginx on one real server to verify failover, restarting it to see round‑robin distribution, and stopping keepalived on the MASTER to confirm VIP migration to the BACKUP.

References: LVS tutorial , LVS introduction .