Understanding High Concurrency: TPS Thresholds and Architectural Solutions

High concurrency refers to a system's ability to process a large number of requests or tasks simultaneously, such as during massive sales events where thousands of users place orders at the same moment.

The primary metric for measuring concurrency is TPS (Transactions Per Second). TPS values between 1,000‑5,000 indicate moderate concurrency, above 5,000 is considered high, and exceeding 50,000 (e.g., Alibaba's 600,000 TPS during Double 11) represents ultra‑high concurrency.

To support such loads, several backend techniques are recommended:

Distributed Architecture – Spreads load across multiple servers, improving scalability and reliability.

Microservice Architecture – Breaks a monolithic application into independent services (e.g., Spring Cloud, Spring Cloud Alibaba) that can be scaled horizontally.

Caching – Uses distributed caches like Redis (and local caches such as Guava) to reduce database pressure by storing hot data in memory.

Load Balancing – Distributes requests across servers using tools like Nginx, HAProxy, LVS, or F5, employing strategies such as round‑robin, weighted, or IP‑hash.

Traffic Shaping (Peak‑Smoothing) – Queues incoming requests in message queues (Kafka, RocketMQ) to smooth bursts and prevent overload.

Rate Limiting and Circuit Breaking – Protects the system during spikes using algorithms like token‑bucket or leaky‑bucket (implemented via Nginx, Guava RateLimiter, Sentinel) and circuit‑breaker mechanisms to isolate failing services.

These combined techniques significantly improve performance and stability under high‑concurrency scenarios.

The article also promotes the author’s extensive resources, including a 300,000‑word Alibaba architecture collection and a comprehensive Java interview question set, which readers can obtain by following the provided links or contacting the author.