Understanding Load Balancing: From Single Tomcat to LVS + Nginx Architecture

Many interviewers ask candidates to describe the complete request flow from a keyword entered on Taobao to the final web page, which involves HTTP, TCP, gateways, and load balancers; mastering these concepts helps diagnose and optimize network performance.

Initially, the startup "Li Da Niu" ran a single Tomcat server that handled all client requests, which worked while traffic was low but soon became a bottleneck as the business grew.

To eliminate the single‑point‑of‑failure and improve capacity, three Tomcat instances were added and a load‑balancing layer (LB) was introduced, typically using Nginx to receive client traffic and forward it to the appropriate backend server.

Because routing all traffic directly to the servers posed security risks, a gateway layer was added in front of the LB to perform authentication, risk control, protocol conversion, and traffic shaping before forwarding requests to Tomcat.

Static resources (JS, CSS, images) were later offloaded to Nginx because Tomcat’s disk‑based file serving is inefficient; Nginx’s proxy cache stores static files in memory and disk, dramatically improving performance—a practice known as "static‑dynamic separation".

While this layered design worked, Nginx is a Layer‑7 (application‑layer) load balancer that must maintain separate TCP connections with both client and backend, consuming memory and limiting scalability under massive concurrent connections.

To overcome this limitation, a Layer‑4 load balancer (LVS) was introduced; LVS forwards packets without establishing connections, offering higher throughput and lower resource usage compared to Nginx.

LVS operates by intercepting the client’s initial SYN packet, selecting a backend server via a load‑balancing algorithm, rewriting the destination IP, and forwarding the packet; the TCP three‑way handshake then occurs directly between client and chosen server.

Combining LVS (for high‑performance packet distribution) with Nginx (for application‑level processing) yields a robust architecture: LVS handles the first‑level traffic split, while Nginx manages authentication, caching, and dynamic request routing. High availability is achieved with keepalived for Nginx and DNS‑based round‑robin for multiple LVS instances.

For moderate traffic, a single Nginx load balancer may suffice; as traffic scales, adding LVS + Nginx and eventually CDN for static assets provides the necessary elasticity and resilience.

In summary, architectural decisions must align with business growth: layered designs (gateway, LB, cache, CDN) decouple responsibilities, simplify scaling, and ensure reliability, echoing the classic TCP/IP layered model where each layer focuses on its own concerns.