From Single Server to Cloud‑Native: 12 Steps of Scaling an E‑Commerce Backend

This article uses Taobao as a concrete example to illustrate how a backend architecture evolves from handling a few hundred concurrent users to supporting tens of millions. It first defines fundamental concepts such as distributed systems, high availability, clustering, load balancing, and the differences between forward and reverse proxies.

1. Overview

The goal is to give readers a holistic view of architectural evolution, summarizing the technologies encountered at each stage and the design principles that emerge.

2. Basic Concepts

Distributed system: Multiple modules deployed on different servers, e.g., Tomcat and database on separate machines.

High availability: The system continues to serve when some nodes fail.

Cluster: A group of servers providing a unified service, often with automatic failover.

Load balancing: Evenly distributing requests across nodes.

Forward vs. reverse proxy: Forward proxy handles outbound traffic for internal services; reverse proxy receives external requests and forwards them internally.

3. Architectural Evolution

3.1 Single‑machine Architecture

Initially, Tomcat and the database run on the same server. As traffic grows, resource contention becomes a bottleneck.

3.2 First Evolution – Separate Tomcat and Database

Deploy Tomcat and the database on separate machines, eliminating resource competition. The new bottleneck shifts to database read/write concurrency.

3.3 Second Evolution – Add Local and Distributed Caches

Introduce in‑process caches (e.g., memcached) and an external cache layer (e.g., Redis) to offload most reads from the database. Issues such as cache consistency, penetration, and avalanche are discussed.

3.4 Third Evolution – Reverse Proxy Load Balancing

Deploy multiple Tomcat instances behind Nginx or HAProxy. Assuming each Tomcat handles ~100 QPS and Nginx ~50,000 QPS, the system can scale to tens of thousands of concurrent users. The database becomes the next bottleneck.

3.5 Fourth Evolution – Database Read‑Write Splitting

Use middleware like Mycat to separate read and write traffic, allowing multiple read replicas. This reduces read contention but introduces challenges in data synchronization and consistency.

3.6 Fifth Evolution – Business‑Level Sharding

Split data by business domain into separate databases, reducing cross‑business contention. This complicates cross‑domain analytics.

3.7 Sixth Evolution – Table Partitioning

Further split large tables into smaller ones (hash‑based or time‑based) to enable horizontal scaling. Tools such as Mycat can manage routing. This turns the database into a distributed MPP system (e.g., TiDB, Greenplum).

3.8 Seventh Evolution – LVS/F5 for Multi‑Level Load Balancing

Introduce layer‑4 load balancers (LVS, hardware F5) to distribute traffic among multiple Nginx instances, achieving higher concurrency and providing high‑availability via keepalived.

3.9 Eighth Evolution – DNS Round‑Robin Across Data Centers

Configure DNS to return multiple IPs, each pointing to a different data‑center, achieving geographic load balancing.

3.10 Ninth Evolution – NoSQL & Search Engines

When relational databases can no longer handle complex queries or massive data volumes, integrate technologies such as HDFS, HBase, MongoDB, Elasticsearch, Kylin, or Druid for storage, analytics, and full‑text search.

3.11 Tenth Evolution – Split Monolith into Small Services

Extract common functionalities (user management, order, payment, authentication) into independent services using frameworks like Dubbo or Spring Cloud, enabling independent deployment and scaling.

3.12 Eleventh Evolution – Enterprise Service Bus (ESB)

Introduce an ESB to unify protocol conversion and hide interface differences, moving towards a SOA architecture that overlaps with micro‑services.

3.13 Twelfth Evolution – Containerization

Package services as Docker images and orchestrate them with Kubernetes, allowing rapid deployment, isolation, and scaling during peak events.

3.14 Thirteenth Evolution – Cloud Platform Adoption

Deploy the entire system on public cloud (IaaS/PaaS/SaaS). Leverage elastic resources, managed services, and pay‑as‑you‑go models to handle massive traffic spikes without over‑provisioning.

4. Design Summary & Principles

The article concludes with a checklist of architectural principles, including N+1 redundancy, rollback capability, feature toggles, built‑in monitoring, multi‑active data centers, mature technology selection, resource isolation, horizontal scalability, buying non‑core components, using commercial hardware, rapid iteration, and stateless service design.

It also notes that real‑world deployments must consider additional challenges such as cross‑data‑center synchronization and distributed transactions, which are left for future discussion.