Designing Scalable High‑Concurrency Architecture: Practical Strategies and Patterns

Introduction

High‑concurrency scenarios such as flash‑sale events or timed red‑packet distribution generate a large number of simultaneous requests. To keep the service responsive, the expected concurrency must be estimated and a suitable architecture designed.

Server Architecture

A production‑grade high‑concurrency service typically includes:

Load balancers to distribute traffic across multiple application nodes.

Master‑slave (or primary‑replica) database clusters for read‑write separation.

Redis (or other NoSQL) cache clusters with master‑slave replication.

Content Delivery Network (CDN) for static assets.

Concurrency Testing

Before go‑live, perform load testing to determine the maximum sustainable request rate. Common approaches:

Third‑party services (e.g., Alibaba Cloud performance testing).

Self‑hosted test servers with tools such as Apache JMeter, Visual Studio Load Test, or Microsoft Web Application Stress Tool.

Collect latency, error rate, and resource utilization metrics to set a safe concurrency ceiling.

General Caching Scheme

For workloads with large daily traffic but occasional spikes (e.g., sign‑in, user profile, order list), prioritize cache reads and fall back to the database only when the cache misses.

Redis Hash Partitioning

Distribute user‑related keys across multiple Redis hash slots to keep each shard small. Example key pattern: user:hash:{shard_id}:{user_id} where {shard_id} is derived from user_id % N (N = number of cache nodes).

Typical Workflows

User sign‑in

Compute hash_key = user:hash:{shard_id}:{user_id}.

Attempt HGET hash_key sign_in_today.

If present, return the cached record.

If absent, query the relational DB, store the result with HSET hash_key sign_in_today {record}, and return it.

When a new sign‑in occurs, execute the DB transaction first, then update the cache.

User orders

Cache only the first page (e.g., 40 items) using a key like orders:{user_id}:page:1.

Subsequent pages are read directly from the DB.

User profile (center)

Cache profile fields under profile:{user_id}.

On cache miss, read from DB, populate the cache, and return.

For shared (public) cache data, use administrative tools or DB‑level locking to avoid a thundering‑herd of DB reads under heavy load.

Message Queue for Write‑Intensive Bursts

When a burst of write operations (e.g., flash‑sale participation, timed red‑packet distribution) would overwhelm the DB, introduce a queue to decouple request intake from persistence.

Redis List Queue

# Producer (web layer)
redis.rpush('red_packet_queue', json.dumps(user_action))

# Consumer thread (worker)
while True:
    item = redis.blpop('red_packet_queue', timeout=5)
    if item:
        action = json.loads(item[1])
        process_red_packet(action)

Multiple consumer threads can run in parallel, each popping items and executing the business logic. This smooths the write rate and protects the DB.

The same pattern can implement scheduled SMS delivery using a sorted set (zset) where the score is the scheduled timestamp.

First‑Level (In‑Process) Cache

If the number of connections to the external Redis cluster becomes a bottleneck, add a local in‑process cache on each web server for the hottest data. Characteristics:

Store only a small subset (e.g., product list for the home page).

TTL measured in seconds, tuned per business need.

Fallback to the external cache on miss.

Static Data Offloading

For data that changes rarely, generate static files (JSON, XML, HTML) and serve them via CDN. Workflow:

Backend updates trigger a script that regenerates the static file.

Upload the file to the CDN; the CDN propagates it to edge nodes.

Clients request the CDN URL first; on a CDN miss, fall back to the cache or DB.

This reduces origin server load and bandwidth consumption during traffic spikes.

Version‑Based Client Caching (Other Scheme)

When data updates are infrequent, let the client cache the payload locally and send a version identifier with each request (e.g., If-None-Match: v123). The server compares the supplied version with the current one:

If they match, respond with a 304‑like status indicating no change.

If they differ, return the new payload together with the updated version tag.

This approach minimizes unnecessary data transfer and reduces load on the backend.