Unveiling LVS: How Linux Virtual Server Handles NAT, DR, and FullNAT Load Balancing

Introduction

Linux Virtual Server (LVS) is a four‑layer (transport‑layer) load balancer that modifies IP addresses in packets to distribute traffic across multiple real servers (RS). The article uses a step‑by‑step diagrammatic approach to reveal how LVS works.

Why a Load Balancer Was Needed

Initially a single machine handled all traffic, but rapid growth in daily active users forced the deployment of multiple servers. Simple DNS‑based load balancing was considered, but it suffers from excessive public IP consumption and stale DNS cache causing traffic to hit failed servers.

LVS Architecture

LVS sits behind a virtual IP (VIP) that receives all client traffic. Internally it forwards requests to RSes that reside in the same private network. LVS only forwards packets; the RS processes the request and returns the response to LVS, which then rewrites the source IP back to the VIP before sending it to the client. This IP rewriting is essentially Network Address Translation (NAT).

Packet Transformation (NAT Mode)

When a client request arrives, the packet contains a TCP four‑tuple (source IP, destination IP, source port, destination port). LVS changes the destination IP to the RS’s IP while keeping the four‑tuple unchanged. After the RS processes the request, the response packet’s source IP is rewritten to the VIP so the client sees a consistent endpoint.

Direct Router (DR) Mode

To avoid the LVS becoming a bottleneck, DR mode lets the RS send response packets directly to the client, bypassing LVS. This requires:

LVS still receives all incoming requests and balances them to RSes.

Each RS must own the same VIP (bound to a virtual interface) and be in the same subnet as LVS, so both have two IPs: a VIP and a private IP.

Because a host can have multiple IP addresses on a single NIC or on virtual interfaces (e.g., eth0:0, lo:0), the VIP is bound to a virtual interface to ensure only LVS answers ARP requests for the VIP.

ARP Configuration

To prevent RSes from responding to ARP queries for the VIP, the following kernel parameters are set on each RS:

net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.lo.arp_ignore=1

These settings make the RS reply only when the target IP matches the address of the incoming interface (i.e., its physical NIC), not the VIP on the virtual interface.

When RS needs to resolve the gateway’s MAC address, it must use its physical IP as the ARP source to avoid contaminating the router’s ARP cache with the VIP. This is enforced by:

net.ipv4.conf.all.arp_announce=2
net.ipv4.conf.lo.arp_announce=2

Here arp_announce=2 tells the kernel to select the most appropriate local address for the outgoing ARP request, ensuring the VIP is not advertised.

FullNAT Mode

In public‑cloud environments, RSes may reside in different VLANs, making plain NAT unsuitable. FullNAT extends NAT by also rewriting the source IP of outgoing packets to the LVS’s internal IP, allowing LVS and RSes to be on different subnets while still preserving the client‑IP information via a TCP option inserted by the LVS. RSes equipped with the TOA module can extract the original client IP from this option.

Summary

LVS supports three primary modes: NAT (default), DR (direct routing), and FullNAT (cross‑VLAN NAT). NAT is simplest but creates a single‑point bottleneck; DR reduces load on LVS but requires complex ARP and interface configuration; FullNAT solves cross‑VLAN deployment issues at the cost of additional packet manipulation. In production, most deployments stick with NAT, while DR and FullNAT are used in specialized scenarios.