Mastering LVS: How to Build High‑Performance Linux Load‑Balancing Clusters

What is a Cluster

LVS (Linux Virtual Server) is a Linux virtual server that aggregates multiple virtual hosts to meet a single demand. Developed by Zhang Wensong, LVS provides load balancing to achieve high‑performance, high‑availability server clusters at low cost.

Cluster Types

LB: Load Balancing – high‑availability cluster

HA: High Availability – high‑availability cluster

HP: High Performance – high‑performance cluster

LVS Common Cluster Modes and Details

1. LVS Architecture

LVS consists of user‑space commands and kernel‑level netfilter INPUT chain hooks, allowing user‑defined clustering for packet forwarding.

2. LVS Terminology

CIP: Client IP address

VIP: Virtual server IP address

DIP: Director IP address (communicates with real servers)

RIP: Real server IP address

RS: Real server

3. LVS Modes

lvs‑nat: Network Address Translation

lvs‑dr: Direct Routing

lvs‑tun: Tunneling

lvs‑fullnat: Enhanced NAT

lvs‑nat Features

Multiple source IPs are translated to the selected RS’s RIP and port.

RIP and DIP must be in the same network and use private addresses; RS gateway points to DIP.

Both request and response packets pass through the Director.

Supports port mapping.

VS must run Linux; RS can be any OS.

lvs‑dr Features

Requests are re‑encapsulated with the Director’s MAC address; destination MAC is the chosen RS’s MAC; IP header remains unchanged.

Front‑end router must send VIP‑destined packets to the Director.

RS RIP can be private or public.

RS and Director must share the same network.

Requests go through the Director; responses bypass it and come directly from the RS.

Port mapping not supported; RS can be any OS.

Solution to ensure the front‑end router forwards VIP packets to the Director:

Static bind VIP to Director’s MAC on the router and block ARP replies from RS.

Use arptables.

Adjust RS kernel parameters and bind VIP to the loopback alias (arp_ignore, arp_announce).

ARP Configuration

arp_announce levels: 0 – default (announce on all interfaces); 1 – avoid non‑local announcements; 2 – always avoid (preferred).

arp_ignore levels: 0 – default (respond using any local address); 1 – respond only if the target IP is configured on the incoming interface.

lvs‑tun Features

Encapsulates an additional IP header; source IP is DIP, destination IP is the selected RS’s VIP; original request IP (CIP) and VIP remain unchanged.

RIP, DIP, and VIP must be public.

RS gateway must not point to DIP.

Requests are forwarded by the Director; responses go directly to CIP.

No port mapping support.

RS must support tunneling.

LVS Scheduling Algorithms

Static Algorithms

RR – Round Robin

WRR – Weighted Round Robin

SH – Source Hashing

DH – Destination Hashing

Dynamic Algorithms

LC – Least Connections

WLC – Weighted Least Connections

SED – Shortest Expected Delay

NQ – Never Queue

LBLC – Local‑based Least Connections

LBLCR – Local‑based Least Connections with Replication

ipvsadm Commands

Manage cluster services (add, edit, delete, list):

ipvsadm -A|-E|-D|-L -t|-u|-f service-address [-s scheduler]

-A: add, -E: edit, -D: delete, -L: list, -t: TCP, -u: UDP, -f: firewall mark.

Example to clear all entries: ipvsadm -C Example to zero counters: ipvsadm -Z [-t|u|f service-address] Manage real servers:

ipvsadm -a|-e -t|-u|-f service-address -r server-address [-g|i|m] [-w weight] [-x upper] [-y lower]

-a: add, -e: edit, -r: real server IP, -g: gateway/direct routing, -m: masquerade/NAT, -i: IPIP/tunnel, -w: weight.

Other ipvsadm sub‑commands:

ipvsadm -L|l [options] – list

-n – numeric output

--stats – statistics

--rate – rate statistics

-c – connections

-exact – exact display

Save and restore configuration:

ipvsadm -S > /path/to/rule_file

ipvsadm-save > /path/to/rule_file

ipvsadm -R < /path/to/rule_file

ipvsadm-restore < /path/to/rule_file

Experimental Topology and Commands

Real Server 1 and 2 set default gateway to 192.168.184.128.

Virtual Server configuration:

ipvsadm -A -t 10.1.10.3:80 -s rr

ipvsadm -a -t 10.1.10.3:80 -r 192.168.184.129 -m -w 1

ipvsadm -a -t 10.1.10.3:80 -r 192.168.184.130 -m -w 2

Enable IP forwarding:

echo 1 > /proc/sys/net/ipv4/ip_forward

Test results show that the lvs‑dr configuration works; the front‑end scheduler uses the WRR algorithm to distribute traffic among backend servers, achieving load balancing. Understanding the underlying principles is essential for mastering LVS.