What Really Powers High‑Concurrency Systems? Practical Solutions Explained

Interview Motivation

Open‑ended questions like “Describe a high‑concurrency solution” are popular in interviews because they are easy for interviewers, generate diverse answers, and allow deep assessment of a candidate’s real abilities.

Horizontal Scaling (Stacking Machines)

The simplest way to handle increased QPS is to add more application servers or database replicas. For example, 1,000 QPS can be served by three app servers plus one DB; 2,000 QPS may need six app servers and a master‑slave DB pair. This approach solves over 50% of read‑heavy scenarios but introduces replication lag, so critical read‑after‑write operations should still hit the master.

Introducing Cache

For read‑heavy workloads, adding a cache dramatically reduces load. Choose between local caches (e.g., HashMap, Guava Cache) and centralized caches like Redis. In most cases (≈95%) Redis is the safe default. Local cache is only advisable when dealing with large, hot keys where network latency becomes the bottleneck and the data rarely changes (e.g., JSON course catalogs).

Cache‑DB consistency can be handled with a simple pattern: delete the cache first, then write to the DB, and optionally perform a delayed double‑delete to mitigate race conditions.

Introducing Elasticsearch (ES)

When queries become multi‑dimensional and cannot be satisfied by simple key‑value caches, Elasticsearch’s inverted index and analyzer provide superior performance over MySQL’s B‑tree for complex search scenarios.

Data consistency between MySQL and ES is typically achieved via binlog‑based tools like Canal or DataBus rather than risky dual‑writes.

Sharding (Database Partitioning)

Sharding addresses mixed read/write bottlenecks. Database sharding is used when hardware limits are reached (CPU, IOPS, network). Table sharding is applied when lock contention becomes the bottleneck.

Table sharding can be vertical (splitting large, infrequently used columns into separate tables) or horizontal (splitting rows across multiple tables to keep B‑tree depth low). This reduces disk I/O and improves concurrency.

Message Queue (MQ) Smoothing

Push every request directly into a queue; consumers process at a controlled rate.

Perform fast, essential steps first, then enqueue the remaining heavy steps for asynchronous processing.

MQ helps absorb traffic spikes, but if sustained high traffic persists, the underlying system capacity must be increased.

Cellization (Unit‑Based Architecture)

Also known as Cell (Alibaba) or Set (Meituan), this approach routes traffic to isolated units that contain a full business loop, achieving routing, isolation, and closed‑loop processing. The key is a well‑chosen sharding key (e.g., city ID for food delivery). While powerful, it incurs high cost and risk, making it suitable only for very large scale services.

Conclusion

These techniques—horizontal scaling, caching, Elasticsearch, sharding, MQ smoothing, and cellization—form a toolbox for tackling high‑concurrency challenges. Selecting the right combination depends on workload characteristics, latency requirements, and operational scale.