From Single Server to Cloud‑Native: 13 Steps to Scale High‑Concurrency Architecture

Basic Concepts

Distributed: multiple modules deployed on different servers.

High Availability: failed nodes are automatically taken over by others.

Cluster: a group of servers providing a unified service, with automatic failover.

Load Balancing: evenly distributing requests across multiple nodes.

Forward and Reverse Proxy: forward proxy accesses external networks on behalf of internal services; reverse proxy forwards external requests to internal servers.

Architecture Evolution

1. Single‑Machine Architecture

Initially Tomcat and the database run on the same server. As user traffic grows, the single machine cannot handle the load.

2. Separate Tomcat and Database

Tomcat and the database are deployed on separate servers, improving performance of each component but making the database a new bottleneck.

3. Add Local and Distributed Cache

Introduce memcached as a local cache and Redis as a distributed cache to intercept most read requests, reducing database pressure. Issues such as cache consistency, penetration, breakdown, and hot‑data eviction are considered.

4. Reverse Proxy Load Balancing (Nginx/HAProxy)

Deploy multiple Tomcat instances behind a reverse proxy (Nginx or HAProxy). The proxy distributes requests, increasing the overall concurrent capacity, but the database becomes the next bottleneck.

5. Database Read‑Write Separation

Use a middleware such as MyCAT to split the database into a write master and multiple read replicas, synchronizing data from the master to the slaves.

6. Business‑Based Database Sharding

Different business domains store data in separate databases, reducing cross‑business contention. This introduces logical distributed databases and requires careful data‑sync handling.

7. Split Large Tables into Small Tables

Hash‑based or time‑based partitioning creates many small tables, allowing horizontal scaling of the database. Tools like MyCAT support this, but operational complexity rises.

8. LVS or F5 Load Balancing (Layer‑4)

Layer‑4 load balancers such as LVS (software) or F5 (hardware) forward TCP/UDP traffic, supporting tens of thousands of concurrent connections. High availability is achieved with keepalived and virtual IPs.

9. DNS Round‑Robin Across Data Centers

Configure a domain to resolve to multiple IPs, each pointing to a different data‑center. DNS rotation distributes traffic globally, enabling horizontal scaling to millions of concurrent users.

10. Introduce NoSQL and Search Engines

When relational databases cannot handle massive data or complex queries, adopt HDFS, HBase, Redis, ElasticSearch, Kylin, Druid, etc., to meet storage, retrieval, and analytics needs.

11. Split Monolith into Small Applications

Divide the codebase by business modules, each becoming an independent application. Shared configuration can be managed with ZooKeeper.

12. Extract Shared Functions as Microservices

Common features such as user management, order processing, and authentication become independent services accessed via HTTP, TCP, or RPC. Frameworks like Dubbo or Spring Cloud provide service governance.

13. Enterprise Service Bus (ESB) for Unified Access

ESB abstracts protocol differences, allowing applications and services to communicate through a common bus, blending SOA and microservice ideas.

14. Containerization (Docker & Kubernetes)

Package services as Docker images and orchestrate them with Kubernetes for dynamic scaling, isolation, and simplified deployment.

15. Cloud Platform Deployment (IaaS / PaaS / SaaS)

Deploy the system on public cloud resources, leveraging on‑demand compute, storage, and managed services (e.g., Hadoop stack, MPP databases). This enables elastic scaling for peak traffic and reduces operational cost.

Architecture Design Summary

Key principles for building robust, scalable systems include:

N+1 design – no single point of failure.

Rollback capability – ensure forward compatibility and easy version rollback.

Feature toggle – ability to disable functions quickly during incidents.

Monitoring – design monitoring from the start.

Active‑active data‑center – multi‑site deployment for ultra‑high availability.

Use mature, battle‑tested technologies.

Resource isolation – prevent one business from monopolizing resources.

Horizontal scalability – the system must be able to scale out.

Buy non‑core components when appropriate.

Commercial‑grade hardware – reduce hardware failure rates.

Rapid iteration – develop small features quickly, validate early.

Stateless design – services should not rely on previous request state.