From Single Server to Cloud‑Native: 14 Stages of Scaling a Large‑Scale Website

Overview

The article uses a Taobao‑style e‑commerce site as a conceptual example to illustrate how a web service can evolve from a single‑machine deployment serving a few hundred users to a multi‑data‑center architecture handling tens of millions of concurrent requests. Each evolution step addresses the current bottleneck and introduces specific technologies.

Basic Concepts

Distributed : Components run on separate physical or virtual machines (e.g., Tomcat web servers and database servers).

High Availability (HA) : The system continues to serve traffic when some nodes fail.

Cluster : Multiple instances of a service act as a single logical unit, providing failover and load distribution.

Load Balancing : Requests are evenly distributed across nodes to avoid overload.

Forward/Reverse Proxy : Forward proxy handles outbound traffic for internal services; reverse proxy receives inbound traffic and forwards it to internal servers.

Architecture Evolution

1. Single‑Machine Architecture

Tomcat and the database share one server; DNS resolves the domain to a single IP. Resource contention quickly becomes a bottleneck as traffic grows.

2. Separate Tomcat and Database

Deploy Tomcat and the database on different machines, isolating CPU, memory, and I/O resources. The new bottleneck shifts to database read/write throughput.

3. Local and Distributed Caching

Introduce in‑process cache (e.g., memcached) and an external cache such as Redis to store hot product data or rendered HTML. This offloads the majority of read traffic from the database, but the Tomcat layer becomes the next limiting factor.

4. Reverse‑Proxy Load Balancing

Deploy multiple Tomcat instances behind a layer‑7 reverse proxy (Nginx or HAProxy). Assuming each Tomcat handles ~100 concurrent connections and Nginx can handle ~50,000, the system can scale to tens of thousands of concurrent users. Database pressure resurfaces as more requests reach it.

5. Database Read/Write Separation

Introduce a middleware such as Mycat to split reads and writes across separate DB instances (one master for writes, multiple slaves for reads). Data synchronization and consistency mechanisms are required.

6. Business‑Based Sharding

Allocate different business domains (e.g., orders, user profiles) to separate databases, reducing cross‑business contention. Write‑heavy shards may still hit performance limits.

7. Large‑Table Partitioning

Apply hash‑based or time‑based partitioning (e.g., comment tables by product ID, payment tables by hour). Mycat can route queries to the appropriate partition. This approach leads to a distributed/Massively Parallel Processing (MPP) database layer. Popular open‑source MPP solutions include Greenplum , TiDB , PostgreSQL‑XC , and HAWQ . The next bottleneck often becomes the reverse‑proxy layer (Nginx).

8. Layer‑4 Load Balancing (LVS/F5)

Use Linux Virtual Server (LVS) or hardware load balancers (F5) to distribute traffic among many Nginx instances. High availability is achieved with keepalived and a virtual IP (VIP). A single LVS instance eventually reaches limits at several hundred thousand concurrent connections.

9. DNS Round‑Robin Across Data Centers

Configure DNS to return multiple IPs, each pointing to a virtual IP in a different data center. Clients are directed to different regions, enabling horizontal scaling to tens of millions of concurrent users.

10. Introduction of NoSQL and Search Engines

When relational databases can no longer satisfy diverse workloads, add specialized components:

File storage: HDFS

Key‑value stores: HBase , Redis

Full‑text search: Elasticsearch

OLAP/Multidimensional analysis: Kylin , Druid

These components increase system complexity and operational overhead.

11. Decompose Monolith into Small Applications

Split the system by business modules, each with its own codebase and deployment pipeline. Shared configuration can be managed with Zookeeper . Code duplication across modules becomes a maintenance challenge.

12. Extract Reusable Functions as Microservices

Common capabilities (user management, order processing, payment, authentication) are exposed as independent services accessed via HTTP, TCP, or RPC. Governance frameworks such as Dubbo or Spring Cloud provide service discovery, load balancing, circuit breaking, and rate limiting. Multiple communication protocols increase integration complexity.

13. Enterprise Service Bus (ESB)

An ESB abstracts protocol conversion and service interaction, reducing coupling between applications. This pattern aligns with SOA principles and can coexist with microservices.

14. Containerization

Package each service as a Docker image and orchestrate deployments with Kubernetes (K8s). Containers provide isolated runtime environments and enable rapid scaling, but underlying hardware still incurs cost when idle.

15. Cloud Platform Adoption

Move the entire stack to a public cloud (IaaS/PaaS/SaaS). Elastic compute, managed storage (e.g., managed Hadoop, managed MPP databases), and on‑demand scaling address peak traffic (e.g., flash sales). Remaining challenges include cross‑region data synchronization and distributed transaction management.

Design Summary

Evolution order is not fixed; address the most pressing bottleneck first.

Design depth should meet current performance goals while leaving room for future growth.

Key principles: N+1 redundancy, rollback capability, feature toggles, comprehensive monitoring, multi‑active data centers, mature technology adoption, resource isolation, horizontal scalability, purchasing non‑core components, using commercial‑grade hardware, rapid iteration, and stateless service design.