How Taobao Scaled from Single Server to Billion-User Architecture

1. Overview

This article uses Taobao as an example to illustrate the evolution of server‑side architecture from a hundred concurrent users to tens of millions, listing the relevant technologies at each stage and summarizing design principles.

2. Basic Concepts

Distributed systems deploy multiple modules on different servers, e.g., Tomcat and databases on separate machines.

High Availability means the system continues to provide service when some nodes fail.

Cluster refers to a group of servers offering a unified service, with automatic failover.

Load Balancing distributes incoming requests evenly across nodes.

Forward and Reverse Proxy describe how internal services access external networks via a forward proxy, and how external requests reach internal services via a reverse proxy.

3. Architecture Evolution

3.1 Single‑Machine Architecture

Initially, Tomcat and the database are deployed on the same server. As user numbers grow, resource contention makes this architecture insufficient.

3.2 First Evolution: Separate Tomcat and Database

Tomcat and the database each occupy dedicated servers, significantly improving performance.

3.3 Second Evolution: Introduce Local and Distributed Caches

Local caches (e.g., memcached) and distributed caches (e.g., Redis) store hot product data or HTML pages, intercepting most requests before they hit the database and reducing database pressure.

3.4 Third Evolution: Reverse Proxy for Load Balancing

Deploy multiple Tomcat instances and use Nginx or HAProxy to distribute requests, greatly increasing the concurrent capacity of the application layer.

3.5 Fourth Evolution: Database Read‑Write Separation

Introduce separate read and write databases (e.g., using Mycat) to alleviate write‑heavy bottlenecks.

3.6 Fifth Evolution: Business‑Level Database Sharding

Store different business data in separate databases, reducing cross‑business contention.

3.7 Sixth Evolution: Split Large Tables into Small Tables

Hash‑based routing and time‑based partitioning enable horizontal scaling of databases, at the cost of higher DBA complexity.

3.8 Seventh Evolution: LVS/F5 for Multi‑Level Nginx Load Balancing

Use Layer‑4 load balancers (LVS software or F5 hardware) to distribute traffic among multiple Nginx instances, providing higher throughput and redundancy.

3.9 Eighth Evolution: DNS Round‑Robin Across Data Centers

Configure DNS to return multiple IPs, each pointing to a different data‑center, achieving global load balancing.

3.10 Ninth Evolution: Introduce NoSQL and Search Engines

Adopt HDFS, HBase, Redis, Elasticsearch, Kylin, Druid, etc., to handle massive data, full‑text search, and multi‑dimensional analytics.

3.11 Tenth Evolution: Split Monolith into Small Applications

Divide code by business domain, allowing independent deployment and scaling.

3.12 Eleventh Evolution: Extract Shared Functions as Microservices

Common functionalities (user management, order, payment, authentication) become independent services accessed via HTTP, TCP, or RPC, managed with Dubbo, Spring Cloud, etc.

3.13 Twelfth Evolution: Enterprise Service Bus (ESB) for Unified Access

ESB handles protocol conversion and decouples services, forming a SOA architecture that overlaps with microservices.

3.14 Thirteenth Evolution: Containerization (Docker & Kubernetes)

Package services as Docker images and orchestrate them with Kubernetes for dynamic scaling and isolation.

3.15 Fourteenth Evolution: Move to Cloud Platform

Deploy the system on public cloud (IaaS, PaaS, SaaS) to leverage elastic resources, reduce operational cost, and achieve on‑demand scaling.

4. Architecture Design Summary

4.1 Must the evolution follow this exact path? No; real projects may address multiple bottlenecks simultaneously or prioritize different concerns.

4.2 How far should the design go? For a one‑off system, meet current performance targets with room for growth; for continuously evolving platforms, design for the next stage and iterate.

4.3 Difference between backend and big‑data architecture Big‑data architecture focuses on data ingestion, storage, processing, and analytics, while backend architecture concerns application organization and often relies on big‑data components.

4.4 Design principles

N+1 design : No single point of failure.

Rollback design : Ensure forward compatibility and easy version rollback.

Feature toggle : Configurable enable/disable of functions.

Monitoring design : Plan observability from the start.

Multi‑active data centers : High availability across locations.

Use mature technology : Prefer proven solutions.

Resource isolation : Prevent one business from monopolizing resources.

Horizontal scalability : Architecture must scale out.

Buy non‑core components : Reduce development effort.

Commercial hardware : Increase reliability.

Rapid iteration : Deploy small features quickly for feedback.

Stateless design : Services should not depend on previous request state.