Understanding High Concurrency and Strategies to Boost System Throughput

What is High Concurrency

High concurrency (High Concurrency) is a crucial factor in designing distributed internet systems; it refers to the ability of a system to process many requests in parallel. Common metrics include response time, throughput, queries per second (QPS), transactions per second (TPS), and the number of concurrent users.

Improving System Concurrency

There are two main approaches: vertical scaling (scale‑up) and horizontal scaling (scale‑out).

Vertical Scaling : Enhancing a single machine’s capability by upgrading hardware (e.g., more CPU cores, 10‑GbE NICs, SSDs, larger disks, more memory) or improving its architecture (using cache to reduce I/O, asynchronous processing to increase throughput, lock‑free data structures to lower latency).

Horizontal Scaling : Adding more servers to linearly increase capacity, which requires architectural designs that support scaling at each layer.

Key techniques for horizontal scaling include:

System clustering and load balancing : Deploy a load‑balancer to distribute requests evenly and use active‑active cluster deployments to absorb initial concurrency pressure.

Database strategies : Apply sharding (horizontal or vertical), read‑write separation (primary for writes, replicas for reads), and adopt distributed databases such as TiDB (HTAP, MySQL‑compatible, horizontal scaling, distributed transactions) or distributed storage solutions like Elasticsearch, ClickHouse, Druid.

Caching : Utilize local caches (disk or memory), distributed cache clusters for high read volumes, and implement pre‑caching or multi‑level caching.

Message middleware : Decouple systems and synchronize data, enabling asynchronous request handling to smooth traffic spikes.

Application splitting (micro‑services) : Separate business domains to reduce coupling, enable tiered deployment, support scaling and resource isolation.

Concurrency mechanisms : Use multithreading with thread pools and parallel task processing.

Content Delivery Network (CDN) : Bypass internet bottlenecks by routing user requests to the nearest edge node based on network traffic, node load and latency.

The article also provides additional recommended readings on architecture diagrams, data warehouses, high‑traffic rate limiting, messaging patterns, and drawing effective architecture diagrams.