Mastering LVS Load Balancing: From Fundamentals to Practical NAT/DR Configurations

1. Basic Introduction to LVS Load Balancing

Load‑balancing clusters, often abbreviated as load balance clusters, distribute incoming user requests to a director server, which then forwards them to real servers based on configured scheduling algorithms. Shared storage ensures consistent data across servers.

LVS (Linux Virtual Server) is an open‑source project initiated by Dr. Zhang Wensong and is now part of the Linux kernel. It enables high‑performance, highly available server clusters with good reliability, scalability, and operability, achieving optimal performance at low cost. The LVS architecture consists of a scheduling layer, a server cluster layer, and shared storage.

2. Basic Working Principle of LVS

1. When a user sends a request to the load‑balancing scheduler (Director Server), the request is forwarded to kernel space.

2. The PREROUTING chain receives the request, determines the destination IP is the local IP, and forwards the packet to the INPUT chain.

3. IPVS operates in the INPUT chain; it matches the request with defined cluster services. If matched, IPVS rewrites the packet’s destination IP and port, then sends the packet to the POSTROUTING chain.

4. The POSTROUTING chain sees the destination IP as a backend server and routes the packet to the selected real server.

3. Components of LVS

LVS consists of two programs: ipvs and ipvsadm .

1. ipvs (IP Virtual Server) runs in kernel space and implements the actual scheduling logic.

2. ipvsadm runs in user space, providing a tool to configure rules for ipvs, defining cluster services and real servers.

4. LVS Terminology

DS: Director Server (frontend load balancer node).

RS: Real Server (backend actual server).

VIP: Virtual IP address presented to clients.

DIP: Director Server IP used for internal communication.

RIP: Real Server IP.

CIP: Client IP.

The following sections summarize three working modes and their characteristics.

5. LVS/NAT Principle and Characteristics

1. Understand the NAT implementation and packet modifications.

(a) User request arrives at Director Server; packet enters PREROUTING with source CIP and destination VIP.

(b) PREROUTING forwards the packet to INPUT.

(c) IPVS matches the service, rewrites destination IP to the backend server’s IP, and sends the packet to POSTROUTING (source CIP, destination RIP).

(d) POSTROUTING routes the packet to the Real Server.

(e) Real Server processes the request and sends a response back to Director Server (source RIP, destination CIP).

(f) Director Server rewrites the source IP to its VIP before responding to the client (source VIP, destination CIP).

2. Characteristics of LVS‑NAT model

RS should use private addresses; RS gateway must point to DIP.

DIP and RIP must be in the same subnet.

All request and response packets pass through Director Server, which can become a performance bottleneck under high load.

Supports port mapping.

RS can run any operating system.

Drawback: Director Server bears significant load because both request and response traverse it.

6. LVS/DR Principle and Characteristics

1. Set the target MAC address of the request packet to the selected RS’s MAC address.

(a) User request arrives at Director Server; packet enters PREROUTING (source CIP, destination VIP).

(b) PREROUTING forwards to INPUT.

(c) IPVS matches the service, changes source MAC to DIP’s MAC and destination MAC to RIP’s MAC, then sends the packet to POSTROUTING (IP addresses unchanged, MAC addresses rewritten).

(d) Because DS and RS are on the same layer‑2 network, POSTROUTING forwards the packet to the Real Server based on the target MAC.

(e) RS receives the packet, processes it, and sends the response via lo to eth0, with source IP VIP and destination CIP.

(f) The response finally reaches the client.

2. Characteristics of LVS‑DR model

All client‑targeted packets (VIP) are forced to the Director Server.

RS can use private or public addresses; public RS can be accessed directly if needed.

RS and Director must reside in the same physical network.

All request packets pass through Director, but response packets bypass it.

No address translation or port mapping support.

RS can run most common operating systems.

RS gateway must never point to DIP.

RS’s lo interface must be configured with the VIP.

Drawback: RS and DS must be in the same data center.

3. Solution for characteristic 1

Configure static routing on the front‑end router to bind VIP only to the Director Server.

If the user lacks routing permissions (e.g., ISP‑provided), this method may be impractical.

Use arptables to filter RS ARP replies at the ARP layer.

Modify RS kernel parameters ( arp_ignore and arp_announce) to place VIP on the lo alias and prevent ARP responses for VIP.

7. LVS/Tun Principle and Characteristics

Encapsulate an additional IP header outside the original packet. Inner header (source CIP, destination VIP); outer header (source DIP, destination RIP).

(a) User request reaches Director Server; packet enters PREROUTING (source CIP, destination VIP).

(b) PREROUTING forwards to INPUT.

(c) IPVS matches the service and adds an outer IP header (source DIP, destination RIP) before sending to POSTROUTING.

(d) POSTROUTING routes the encapsulated packet to RS via the tunnel (source DIP, destination RIP).

(e) RS receives the packet, strips the outer header, discovers the inner destination is its lo VIP, processes the request, and sends the response out through lo to eth0 (source VIP, destination CIP).

(f) The response reaches the client.

Characteristics of LVS‑Tun:

RIP, VIP, DIP are all public addresses.

RS gateway cannot point to DIP.

All requests pass through Director; responses bypass it.

No port mapping support.

RS must support tunneling.

In practice, DR mode is most commonly used, while NAT is simpler to configure.

8. Eight LVS Scheduling Algorithms

1. Round‑Robin (rr)

Distributes requests sequentially across servers, assuming equal capacity.

2. Weighted Round‑Robin (wrr)

Introduces weights (0‑100) to favor more capable servers.

3. Least Connections (lc)

Routes request to the server with the fewest active connections.

4. Weighted Least Connections (wlc)

Combines weights with the least‑connections approach.

5. Locality‑Based Least Connections (lblc)

Prefers servers that have recently served the same destination IP.

6. Locality‑Based Least Connections with Hash (lblcr)

Maintains a mapping from destination IP to a server group to avoid overload.

7. Destination‑Hash (dh)

Hashes the destination IP to select a server; static mapping persists across failures.

8. Source‑Hash (sh)

Hashes the source IP for static server assignment.

9. Practical LVS NAT Mode

1. Experiment Environment

Three machines: one Director (external IP 172.16.254.200, internal IP 192.168.0.8) and two Real Servers (192.168.0.18 and 192.168.0.28). Real servers’ gateways point to the Director’s internal IP.

2. Installation

# yum install -y nginx # yum install -y ipvsadm

3. Director Configuration Script (lvs_nat.sh)

#!/bin/bash
# Enable IP forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward
# Disable ICMP redirects
echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
# Additional sysctl settings omitted for brevity
# Configure NAT firewall rule
iptables -t nat -F
iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -j MASQUERADE
# Add LVS rule using ipvsadm
IPVSADM=/sbin/ipvsadm
$IPVSADM -C
$IPVSADM -A -t 172.16.254.200:80 -s wrr
$IPVSADM -a -t 172.16.254.200:80 -r 192.168.0.18:80 -g -w 1
$IPVSADM -a -t 172.16.254.200:80 -r 192.168.0.28:80 -g -w 1

Running this script on the Director sets up the LVS/NAT configuration.

4. Testing

Modify the default Nginx page on each Real Server:

# echo "rs1rs1" > /usr/share/nginx/html/index.html # echo "rs2rs2" > /usr/share/nginx/html/index.html

Access http://172.16.254.200 in a browser; the response alternates between the two pages, confirming round‑robin distribution.

10. Practical LVS DR Mode

1. Experiment Environment

Director: eth0 192.168.0.8, VIP eth0:0 192.168.0.38

Real Server 1: eth0 192.168.0.18, VIP lo:0 192.168.0.38

Real Server 2: eth0 192.168.0.28, VIP lo:0 192.168.0.38

Installation steps are similar to NAT mode, installing nginx on the Real Servers and ipvsadm on the Director.

2. Director DR Script (lvs_dr.sh)

#!/bin/bash
# Enable IP forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward
# Set VIP
VIP=192.168.0.38
# Configure interface
ifconfig eth0:0 $VIP netmask 255.255.255.255 up
# Add route for VIP
route add -host $VIP dev eth0
# LVS configuration
IPVSADM=/sbin/ipvsadm
$IPVSADM -C
$IPVSADM -A -t $VIP:80 -s wrr
$IPVSADM -a -t $VIP:80 -r 192.168.0.18:80 -g -w 1
$IPVSADM -a -t $VIP:80 -r 192.168.0.28:80 -g -w 1

Execute the script on the Director to activate DR mode.

3. Real Server DR Script (lvs_dr_rs.sh)

#!/bin/bash
VIP=192.168.0.38
ifconfig lo:0 $VIP netmask 255.255.255.255 up
route add -host $VIP dev lo
# Disable ARP replies for VIP
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce

Run this script on each Real Server.

4. Testing

Access http://192.168.0.38 from a client. The response alternates between the two Real Servers, confirming DR operation.

11. LVS Combined with Keepalived

LVS lacks health checks; if a Real Server fails, LVS may still forward traffic to it. Keepalived provides health monitoring and high‑availability failover, making it ideal for LVS clusters.

1. Experiment Environment

Keepalived + LVS Director 1: 192.168.0.48

Keepalived + LVS Director 2: 192.168.0.58

Real Server 1: 192.168.0.18

Real Server 2: 192.168.0.28

Virtual IP: 192.168.0.38

2. Installation

# yum install ipvsadm keepalived -y # yum install epel-release -y # yum install nginx -y

3. Keepalived Configuration (Master)

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.0.38
    }
}
virtual_server 192.168.0.38 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 0
    protocol TCP
    real_server 192.168.0.18 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
    real_server 192.168.0.28 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}

Configure the backup node similarly, changing state to BACKUP and lowering priority to 90.

4. Enable IP Forwarding

# echo 1 > /proc/sys/net/ipv4/ip_forward

5. Start Keepalived

# service keepalived start

6. Verification

Stop Nginx on one Real Server; traffic continues to the other server, confirming health checks.

Restart the stopped Nginx; traffic returns to both servers according to the round‑robin algorithm.

Stop Keepalived on the master; the virtual IP migrates to the backup node, and client access remains uninterrupted, demonstrating HA capability.

References: LVS tutorial , LVS introduction .