Designing Scalable, High‑Availability Network Services with Linux LVS

Table of Contents

Preface

Human society is entering an information age centered on networks, demanding faster, more reliable, and richer network services. The explosive growth of the World Wide Web has led to unprecedented traffic loads, raising concerns about how systems can be scaled to meet ever‑increasing performance demands while maintaining 7x24 availability. Future applications will require higher throughput, better interactivity, and stronger security, which in turn demand more powerful and highly available service platforms.

1. Definition of Scalable Network Services

Scalability

is a frequently used term in modern computing. For different people it has different meanings. Here we define a scalable network service.

A scalable network service can expand its performance as the number of users grows, e.g., by adding servers, memory, or disks. The system can be expanded without reconfiguring the whole system or interrupting service, making the expansion transparent to end users.

Scalable systems are usually highly available. When some hardware (e.g., disks, servers, sub‑networks) or software (e.g., OS, service processes) fails, the system continues to provide service, and users are unaware of the interruption except for the subset of requests that were being processed on the failed component.

Scalability is typically achieved through a one‑to‑many mapping mechanism, distributing request streams ( divide and conquer) across multiple nodes. Examples include DNS on the hostname level, TCP/IP at the network layer, and virtual memory in operating systems.

2. Requirements of Network Services

With the rapid development of the Internet, more enterprises move their interactions with customers and partners online, leading to massive traffic spikes that can overwhelm web sites and degrade user experience. High availability becomes critical for e‑commerce and other key applications.

Scalability

The system must expand to meet increasing load without degrading service quality.

High Availability

The service must remain 7*24 available despite hardware or software failures.

Manageability

Even large physical deployments should be easy to manage.

Cost‑effectiveness

The solution should be economical and affordable.

Single‑server upgrades have several drawbacks:

The upgrade process is cumbersome, causing temporary service interruption and resource waste.

Higher‑end servers become increasingly expensive.

A server or application failure can bring down the entire service.

Clustered servers connected by high‑performance networks provide an effective structure for achieving scalability and high availability. A loosely‑coupled architecture offers better scalability‑to‑price ratios than tightly‑coupled multiprocessor systems.

We propose load‑balancing and content‑based request distribution methods implemented in the Linux kernel, forming a Linux Virtual Server (LVS) cluster that appears as a single high‑performance, highly available server to clients.

3. LVS Cluster Architecture

Below we present the generic LVS cluster structure, discuss its design principles, and then apply it to scalable Web, Media, Cache, and Mail services.

3.1 Generic LVS Structure

LVS clusters use IP load‑balancing and content‑based request distribution. The scheduler provides high throughput and automatically masks server failures, presenting a virtual server to clients.

The architecture consists of three main components:

Load balancer (scheduler)

Server pool

Backend storage

The load balancer receives client requests and forwards them to a server pool; the pool executes services such as Web, Mail, FTP, and DNS. Backend storage provides a shared storage area, often implemented with distributed file systems (AFS, GFS, Coda, Intermezzo) and a distributed lock manager to ensure consistency.

High‑speed networks (100 Mbps, Myrinet, CompactNET, Gigabit Ethernet) interconnect these components, preventing the network from becoming a bottleneck as the system scales.

3.2 Scalable Web and Media Services

In a typical LVS‑based scalable Web/Media service, a load balancer (using IP load‑balancing) sits in front of a Web server pool, while a distributed file system supplies shared data to all nodes. This architecture enables rapid addition of new servers without copying content locally.

3.3 Scalable Cache Service

Cache servers reduce network traffic and latency. LVS clusters can provide scalable cache services by placing cache servers near backbone links and using ICP (Internet Cache Protocol) for inter‑cache communication.

3.4 Scalable Mail Service

Increasing Internet users strain ISP mail servers. An LVS‑based mail service places a load balancer in front of a server pool (including LDAP for user data) and uses distributed storage for mail. The load balancer distributes SMTP, POP3, IMAP4, and HTTP requests across the pool.

4. Geographically Distributed LVS Architecture

Deploying LVS clusters in multiple data centers worldwide allows users to connect to the nearest cluster, reducing latency and bandwidth costs while providing disaster resilience.

4.1 Architecture

Three LVS clusters are placed in different IDC locations (e.g., China, USA, Germany) and share a virtual IP address. Users are routed to the nearest cluster via BGP.

4.2 BGP‑Based Cluster Scheduling

BGP inserts routing information for the virtual IP address. Routers select the closest LVS cluster based on path evaluation. If a cluster fails, BGP withdraws its route, and traffic is automatically redirected to another cluster.

4.3 Inter‑Cluster Load Balancing

When a regional cluster becomes overloaded, IP tunnels can forward excess requests to a less‑loaded remote cluster, based on real‑time load metrics (e.g., local load > 1.1 and remote load < 0.7). This improves response times and avoids over‑loading any single site.

5. Conclusion

We analyzed current and future network service requirements and presented a three‑tier architecture—load balancer, server pool, and backend storage—implemented as a Linux Virtual Server (LVS) cluster. The design provides load balancing, scalability, and high availability for Web, Cache, Mail, and Media services. Geographic distribution with BGP routing and IP‑tunnel based inter‑cluster balancing further enhances performance, reduces bandwidth costs, and improves disaster resilience.