Mastering Linux Virtual Server (LVS): Load Balancing Principles, Types, and Algorithms

Table of Contents

Preface

LVS

(Linux Virtual Server) is an open‑source load‑balancing cluster project initiated by Dr. Zhang Wensong and now part of the Linux kernel. It provides high‑performance, high‑availability server clusters with strong reliability, scalability, and operability at low cost.

1. Load Balancing Principles

System operations: availability → standardization → automation

1.1 Working Principle

The diagram below shows the basic LVS workflow:

(1) When a client sends a request to the Director Server, the scheduler forwards the request to kernel space.

(2) The PREROUTING chain receives the packet, determines the destination IP is the local IP, and forwards it to the INPUT chain.

(3) IPVS (running in the INPUT chain) matches the packet against defined cluster services. If it matches, IPVS rewrites the destination IP and port, then sends the packet to the POSTROUTING chain.

(4) The POSTROUTING chain forwards the packet to the selected real server.

1.2 Working Characteristics

LVS

operates at the TCP/IP layer (Layer 4) by modifying the

Netfilter

INPUT

chain based on the client packet's destination IP and port, then forwards it to a backend server according to the chosen scheduling algorithm.

Principle

When a client requests the LVS scheduler IP, the packet passes PREROUTING → INPUT. If a rule matches, the packet is sent to POSTROUTING.

Rules

Server port: the cluster works on TCP or UDP ports.

Server IP: the machines that can respond to client requests.

Architecture

Load‑balancing scheduler (Director Server)

Server cluster (Real Server)

Shared storage

Terminology

DS

: Front‑end load‑balancing node RS: Backend real server CIP: Client IP address VIP: Virtual IP presented to clients DIP: Director IP used for internal communication RIP: Real server IP

Tools

ipvs

: Works in the kernel

Netfilter

INPUT

chain, supports TCP, UDP, AH, ESP, etc. ipvsadm: User‑space utility built on ipvs for managing LVS clusters.

[root@localhost ~]# grep -i -A 10 'IPVS' /boot/config-2.6.32-573.26.1.el6.x86_64
# IPVS transmission protocol support load balancing
CONFIG_IP_VS_PROTO_TCP=y      # TCP load balancing
CONFIG_IP_VS_PROTO_UDP=y      # UDP load balancing
CONFIG_IP_VS_PROTO_AH_ESP=y   # Authentication Header and ESP
CONFIG_IP_VS_PROTO_ESP=y      # ESP
CONFIG_IP_VS_PROTO_AH=y       # AH
CONFIG_IP_VS_PROTO_SCTP=y
# IPVS schedulers
CONFIG_IP_VS_RR=m
CONFIG_IP_VS_WRR=m
CONFIG_IP_VS_LC=m
CONFIG_IP_VS_WLC=m
CONFIG_IP_VS_LBLC=m
CONFIG_IP_VS_LBLCR=m
CONFIG_IP_VS_DH=m
CONFIG_IP_VS_SH=m
CONFIG_IP_VS_SED=m
...

2. Load Balancing Types

CDN (regional cache) → LB (LVS/Nginx) → HA (LVS/Haproxy) → Cache (Nginx/Vanish/Mechcache) → HTTP (Nginx/Apache) → Database (master/worker)

LVS‑NAT: Multi‑target DNAT

LVS‑DR: Direct routing

LVS‑TUN: IP tunnel

LVS‑FULLNAT: Full NAT (Taobao custom implementation)

Enterprise environments most commonly use the DR mode because its configuration is simpler.

2.1 LVS/NAT

Key point: DNAT the request packet to the selected RS address.

NAT Model Implementation

(1) Client request reaches the Director Server's VIP, entering the kernel PREROUTING chain.

(2) PREROUTING forwards the packet to INPUT.

(3) IPVS rewrites the destination IP to the chosen RS and sends the packet to POSTROUTING.

(4) POSTROUTING routes the packet to the Real Server.

(5) Real Server processes the request and sends the response back to the Director Server.

(6) Director Server rewrites the source IP to its VIP and returns the response to the client.

NAT Model Features

Features

Supports port mapping.

Real Servers can run any OS, including Windows.

Drawbacks

Both request and response pass through the Director Server, which can become a performance bottleneck under high load.

Configuration

Director Server maintains a nat tracking table to forward client requests to real servers.

Requests are DNAT‑translated to the backend server IP.

Director Server uses two NICs: a public VIP and a private DIP, requiring DNAT and SNAT.

Real Server and DIP should be in the same private subnet; the Real Server gateway must point to DIP.

2.2 LVS/DR

Key point: Set the target MAC address of the request packet to the selected RS's MAC address.

DR Model Implementation

(1) Client request reaches Director Server's VIP, entering PREROUTING.

(2) PREROUTING forwards the packet to INPUT.

(3) IPVS changes the source MAC to DIP and destination MAC to the selected RS's MAC, then sends the packet to POSTROUTING.

(4) Because DS and RS are on the same L2 network, POSTROUTING forwards the packet to the Real Server.

(5) RS receives the packet, processes it, and sends the response (source IP = VIP, destination IP = client) back via the Director Server.

(6) Response reaches the client.

DR Model Features

Features

No address or port translation.

RS can run most OSes but not Windows.

Request packets pass through DS; response packets bypass DS.

Switches forward based on MAC addresses.

Drawbacks

RS and DS must reside in the same data center.

Configuration

DS configures two NICs: DIP on eth0 and VIP on eth0:0.

Each RS configures RIP on eth0 and VIP on lo:0.

RS gateway must not point to DIP because responses should not traverse DS.

2.3 LVS/TUN

Key point: Encapsulate the original IP packet with an additional outer IP header (inner: client → VIP, outer: director → RS).

TUN Model Implementation

(1) Client request reaches Director Server's VIP and enters PREROUTING.

(2) PREROUTING forwards to INPUT.

(3) IPVS encapsulates the packet with an outer IP header (source DIP, destination RIP) and sends it to POSTROUTING.

(4) POSTROUTING forwards the tunneled packet to the Real Server.

(5) RS removes the outer header, sees the inner VIP, processes the request, and sends the response (source VIP, destination client) back through the tunnel.

(6) Response reaches the client.

TUN Model Features

Features

Requests must pass through DS; responses must bypass DS.

RS can be located in different regions (e.g., Beijing and Shanghai).

Drawbacks

No port mapping support.

RS OS must support tunneling.

Additional encapsulation may exceed MTU, causing fragmentation.

Configuration

All IPs ( VIP, DIP, RIP) are public.

RS gateway must not point to DIP because responses should not traverse DS.

2.4 LVS/FULLNAT

Key point: DS modifies both destination and source addresses of the request packet.

FULLNAT Model Implementation

(1) Client sends request to Director Server's VIP (PREROUTING, source = CIP, dest = VIP).

(2) PREROUTING forwards to INPUT.

(3) IPVS rewrites source to DIP and destination to Real Server RIP, then sends to POSTROUTING.

(4) POSTROUTING routes packet to Real Server.

(5) Real Server builds response (source = RIP, dest = DIP) and sends to Director Server.

(6) Director Server rewrites source to its VIP and destination to client CIP, then returns the response.

https://github.com/alibaba/LVS

FULLNAT Model Features

Features

Supports port mapping.

Cross‑VLAN deployment across multiple locations.

Connection reuse similar to F5.

Syn‑proxy configurable per service.

Configuration

VIP

is a public address; RIP and DIP are private and need not be in the same network.

RS receives requests with source DIP, so it replies to DIP without pointing its gateway to DS.

Both request and response must pass through DS; high availability (HB) is recommended.

RS can run any OS, including Windows.

Challenges

RS cannot obtain the client’s real IP address.

Taobao solves this with the TOA method: the client IP is placed in a TCP option and retrieved by a kernel module.

3. Session Persistence

HTTP is stateless; to identify users, session persistence is required. Common methods include:

Session Binding

Source IP hash (SNAT): multiple users share a public IP; works with LVS, Nginx, HAProxy at IP level.

Cookie hash: custom implementation; supported by Nginx and HAProxy at process level.

Session Cluster

Requires extensive session synchronization; power loss can cause issues.

Session Server

Persist sessions in Redis (key‑value store) for durability.

Network latency is inevitable.

Single‑point failure leads to bottleneck → load balancer → high availability.

4. Ten Scheduling Algorithms

Static Methods (fair start)

RR – Round Robin

WRR – Weighted Round Robin

SH – Source Hash (session persistence)

DH – Destination Hash

Dynamic Methods (load‑aware)

LC – Least Connections

WLC – Weighted Least Connections

SED – Shortest Expected Delay

NQ – Never Queue

LBLC – Locality‑Based Least Connections (dynamic DH)

LBLCR – Locality‑Based LC with Replication

5. Load Balancing Comparison

LVS

Advantages

Strong load‑handling capability due to simple logic.

Highly stable as it runs in kernel space.

Zero user‑space traffic; only request forwarding.

Supports virtually all applications (HTTP, databases, chat, etc.).

Disadvantages

Configuration requires command‑line tools and has a learning curve.

Nginx

Advantages

Application‑layer (HTTP) routing based on domain, path, etc.

Lightweight deployment; only needs network connectivity.

Simple configuration via text files.

Handles high traffic; limited by server I/O and config.

Built‑in health checks (status codes, timeouts).

Supports asynchronous request handling to offload backend servers.

Disadvantages

Only supports HTTP and email protocols.