How LVS Balances Traffic: NAT, Direct Routing, and Tunneling Explained

LVS Overview

LVS (Linux Virtual Server) is a layer‑4 virtual router cluster that forwards user requests based on destination IP address and port.

How LVS Works

1. A user sends a request to the Director Server (load‑balancer). The request is first handled in the kernel's PREROUTING chain. 2. PREROUTING receives the packet, determines the target IP is the local machine, and forwards it to the INPUT chain. 3. IPVS, operating in the INPUT chain, matches the packet against defined cluster services. If it matches, IPVS rewrites the destination IP and port to point to a selected Real Server and sends the packet to the POSTROUTING chain. 4. POSTROUTING sees the destination IP now belongs to a backend server and routes the packet to that server.

LVS Components

LVS consists of two parts:

ipvs : kernel‑space code that actually performs packet scheduling.

ipvsadm : user‑space tool that defines rules for ipvs, specifying which services form a cluster and which machines are Real Servers.

Key Terminology

Director Server (DS)

Real Server (RS)

IP‑Related Terms

VIP – Virtual IP, the address presented to clients.

DIP – Director Server IP, used for internal communication.

RIP – Real Server IP.

CIP – Client IP.

LVS Deployment Models

There are three architectural models:

1. LVS‑NAT – suitable for small‑scale deployments.

2. LVS‑DR (Direct Routing) – recommended for large‑scale use and the most common today.

3. LVS‑Tun (Tunneling) – rarely used; mainly for cross‑datacenter disaster recovery.

LVS‑NAT Model

Key point: Destination IP is rewritten to the selected RS IP.

1. Request reaches Director Server → PREROUTING (source IP = CIP, destination IP = VIP). 2. PREROUTING forwards packet to INPUT. 3. IPVS matches service, rewrites destination IP to the Real Server IP, then sends to POSTROUTING (source IP = CIP, destination IP = RIP). 4. POSTROUTING routes packet to the Real Server. 5. Real Server processes request and sends response back to Director Server (source IP = RIP, destination IP = CIP). 6. Director Server rewrites source IP to VIP before replying to the client (source IP = VIP, destination IP = CIP).

Characteristics of NAT

1. Real Servers usually use private addresses; their gateway must point to DIP. 2. DIP and RIP must be in the same subnet. 3. Both request and response packets traverse the Director Server, which can become a performance bottleneck under heavy load. 4. Supports port mapping. 5. Real Servers can run any operating system.

Drawback: Director Server bears heavy load because all traffic passes through it.

LVS‑DR Model

Key point: Destination MAC address is changed to the selected RS MAC.

1. Request reaches Director Server → PREROUTING (source IP = CIP, destination IP = VIP). 2. PREROUTING forwards to INPUT. 3. IPVS matches service, rewrites source MAC to DIP's MAC and destination MAC to RS's MAC, then forwards to POSTROUTING (IP addresses unchanged). 4. Because DS and RS share the same L2 network, POSTROUTING sends the packet to the Real Server based on the MAC address. 5. Real Server receives the packet (its own MAC), processes it, and sends the response out via its loopback to eth0. 6. Response reaches the client.

Characteristics of Direct Routing

1. Ensure front‑end routing sends all VIP‑destined packets to the Director Server. • Solutions: static address routing on the front‑end router, ARP tables filtering, or adjusting kernel parameters (arp_ignore, arp_announce) and configuring VIP on the RS loopback interface. 2. Real Servers may use private or public addresses; public addresses allow direct Internet access to RIP. 3. RS and DS must reside in the same physical network. 4. All request packets pass through the Director Server, but responses bypass it. 5. No support for address translation or port mapping. 6. RS can run most common operating systems. 7. RS gateway must never point to DIP. 8. VIP is configured as an alias on the RS loopback interface.

Drawback: RS and DS must be in the same data center.

LVS‑Tun Model

Key point: The original IP packet is encapsulated with an additional outer IP header (outer source = DIP, outer destination = RIP).

1. Request reaches Director Server → PREROUTING (source IP = CIP, destination IP = VIP). 2. PREROUTING forwards to INPUT. 3. IPVS matches service, encapsulates the packet with a new outer IP header (source = DIP, destination = RIP) and sends to POSTROUTING. 4. POSTROUTING forwards the encapsulated packet to the Real Server. 5. Real Server strips the outer header, discovers the inner destination is its VIP on the loopback interface, processes the request, and sends the response out via the loopback to eth0. 6. Response reaches the client.

Characteristics of Tunneling

1. RIP, VIP, and DIP are all public IP addresses. 2. RS gateway cannot point to DIP. 3. All request packets traverse the Director Server, but responses must bypass it. 4. Port mapping is not supported. 5. RS operating system must support tunneling.

LVS Scheduling Algorithms

Scheduling algorithms are divided into static and dynamic methods.

Static Methods

These consider only the algorithm itself, ignoring server load.

rr – Round Robin: distributes requests sequentially regardless of server load.

WRR – Weighted Round Robin: assigns more requests to higher‑capacity servers based on configured weights.

sh – Source Hashing: directs all requests from the same client IP to the same Real Server, enabling session persistence.

dh – Destination Hashing: hashes the destination address; mainly useful when multiple firewalls share an internal network.

Dynamic Methods

These adjust distribution according to real‑time server load.

lc – Least Connections: selects the server with the fewest active connections (Overhead = Active*256 + Inactive).

wlc – Weighted Least Connections: similar to lc but incorporates server weight (Overhead = (Active*256 + Inactive) / weight).

sed – Shortest Expected Delay: based on wlc, calculates Overhead = ((Active+1)*256) / weight, favoring servers with higher weight.

nq – Never Queue: if any Real Server has zero connections, the request is sent there directly, bypassing further calculations.

lblc – Locality‑Based Least Connections: used in cache clusters; selects the nearest Real Server for a given target IP.

lblcr – Replicated lblc: an enhanced lblc that can add a server to the group when the current one is overloaded.

Overhead represents load; smaller values indicate lighter load.

Conclusion

This article covered LVS architecture, its three load‑balancing models (NAT, Direct Routing, Tunneling), and the scheduling algorithms (rr, wrr, sh, dh, lc, wlc, sed, nq, lblc, lblcr) used to distribute traffic efficiently.