Mastering LVS: How Linux Virtual Server Load Balancing Works

Overview

LVS (Linux Virtual Server) is a kernel‑integrated load balancer that can handle millions of concurrent connections.

Key Terminology

VS : Virtual Server – the LVS node itself.

RS : Real Server that provides the actual service.

CIP : Client IP address.

VIP : Virtual IP advertised by LVS; clients connect to this address.

DIP : Director IP on the internal network, used by LVS to forward traffic.

RIP : Real Server IP.

Typical request flow: CIP → VIP → DIP → RIP.

Four LVS Operating Modes

1. NAT (lvs‑nat)

In NAT mode the director rewrites the destination IP/port of the client packet (DNAT) to point to a selected real server. When the real server replies, the source IP/port is rewritten back to the VIP (SNAT). All traffic – request and response – passes through the director.

Key points :

RS and DIP must reside in the same subnet.

Response packets traverse the director, increasing its load.

VS must run Linux; RS can be any OS.

2. Direct Routing (DR, lvs‑dr)

DR is the default mode. The director only changes the Ethernet MAC address; the IP header (source and destination) remains unchanged. The packet is delivered directly to the chosen real server.

Key points :

Each RS must be configured with the same VIP as the director.

Client packets must reach the director – typically achieved with static ARP entries or by disabling ARP replies on the RS.

RS can use private or public IPs; RIP and DIP share the same subnet; the RS gateway must not point to DIP.

Director and RS must be on the same physical network.

ARP handling for DR

Configure static ARP bindings on the upstream router so that the VIP resolves to the director’s MAC.

Disable ARP replies on the RS (e.g., bind VIP to the loopback interface and set arp_ignore / arp_announce to 0) so that the RS never answers ARP requests for the VIP.

# Verify ARP settings on the loopback interface
cat /proc/sys/net/ipv4/conf/lo/arp_ignore   # expected: 0
cat /proc/sys/net/ipv4/conf/lo/arp_announce # expected: 0

3. TUN (lvs‑tun)

TUN extends DR by encapsulating the original client packet inside an additional IP header. This allows the director (DIP) and the real server (RIP) to be on different physical networks.

Key points :

DIP and RIP do not need to share a subnet; RIP may be reachable via the Internet.

RS gateway typically must not point to DIP.

The RS operating system must support IP tunneling (e.g., GRE or IPIP).

4. FULLNAT (lvs‑fullnat)

FULLNAT rewrites both source and destination IP addresses (SNAT + DNAT). The client’s source IP is replaced by DIP, and the destination becomes RIP. The real server sees the request as coming from DIP and replies to DIP, which then forwards the response back to the client.

Key points :

VIP is usually a public address; RIP and DIP are private and often reside in different subnets.

Both request and response traverse the director, enabling port‑mapping and NAT for the entire flow.

Kernel support requires the FULLNAT patch set (not enabled in vanilla kernels).

Mode Comparison

NAT : DIP and RIP share a subnet; RIP gateway points to DIP so that responses flow through the director.

DR : DIP and RIP share a subnet; RIP gateway must NOT point to DIP, allowing direct replies from RS to client.

TUN : DIP and RIP can be in different networks; RIP may have Internet connectivity; encapsulation adds an extra IP header.

FULLNAT : DIP and RIP may be in different networks but must be reachable; both directions pass through the director.