From a Simple MVP Monolith to a Complex Distributed Architecture: Taobao Case Study

Basic Concepts Distributed systems consist of multiple modules deployed on different servers, high availability ensures service continuity when nodes fail, clusters combine multiple servers into a single service, load balancing distributes requests evenly, and forward/reverse proxies handle external and internal traffic routing.

1. Single‑Machine Architecture Initially, Tomcat and the database run on the same server, handling low traffic.

2. First Evolution: Separate Tomcat and Database Deploy Tomcat and the database on different machines, improving resource isolation and performance, but database read/write becomes the new bottleneck.

3. Second Evolution: Add Local and Distributed Caches Introduce memcached as a local cache and Redis as a distributed cache to offload most read traffic from the database, addressing cache consistency, penetration, and avalanche issues.

4. Third Evolution: Reverse Proxy Load Balancing Deploy multiple Tomcat instances behind Nginx (or HAProxy) to distribute traffic, dramatically increasing concurrent request capacity, while the database becomes the next bottleneck.

5. Fourth Evolution: Database Read‑Write Separation Use MyCAT to split the database into a write master and multiple read replicas, synchronizing data and improving read scalability.

6. Fifth Evolution: Business‑Level Sharding Allocate different business data to separate databases, reducing contention and allowing independent scaling, at the cost of cross‑business joins.

7. Sixth Evolution: Split Large Tables Hash comments by product ID or partition hourly payment tables, enabling horizontal scaling of the database; this leads to MPP‑style distributed databases such as Greenplum, TiDB, etc.

8. Seventh Evolution: LVS/F5 Load Balancing Use layer‑4 load balancers (LVS software or F5 hardware) to distribute traffic among multiple Nginx instances, adding high‑availability via keepalived.

9. Eighth Evolution: DNS Round‑Robin Across Data Centers Configure DNS to return multiple IPs, each pointing to a different data‑center, achieving inter‑data‑center load balancing.

10. Ninth Evolution: Introduce NoSQL and Search Engines Add HDFS, HBase, Redis, Elasticsearch, Kylin, Druid, etc., to handle massive data, full‑text search, and multi‑dimensional analytics.

11. Tenth Evolution: Split Monolith into Small Applications Separate code bases by business domain, using ZooKeeper for shared configuration.

12. Eleventh Evolution: Extract Shared Functions as Microservices Pull common features (user management, payment, auth) into independent services, using Dubbo or Spring Cloud for service governance.

13. Twelfth Evolution: Enterprise Service Bus (ESB) Introduce an ESB to unify protocol conversion and reduce coupling, resembling SOA architecture.

14. Thirteenth Evolution: Containerization Package services as Docker images and orchestrate them with Kubernetes, enabling rapid scaling and isolated runtime environments.

15. Fourteenth Evolution: Cloud Platform Adoption Deploy the system on public cloud (IaaS, PaaS, SaaS), leveraging elastic resources, Docker/K8s, and managed services to achieve on‑demand scaling and lower operational costs.

Architecture Design Summary The article concludes with twelve practical design principles, including N+1 redundancy, rollback capability, feature toggles, built‑in monitoring, multi‑active data centers, mature technology selection, resource isolation, horizontal scalability, buying non‑core components, using commercial hardware, rapid iteration, and stateless service design.

Author: Shi Hua, a seasoned big‑data engineer with extensive experience in architecture, high concurrency, and distributed systems.