Why Redis Handles Millions of Concurrent Requests: Event Loop and Cluster Design

Problem Overview

Many teams treat Redis as a "faster key‑value store" and attribute its ability to handle million‑level concurrency to the myth that "Redis is single‑threaded, therefore fast". The real answer lies in a chain of engineering mechanisms that together enable production‑grade performance.

Core Capabilities Required for Million‑Level Concurrency

Stable maintenance of millions of long‑lived connections and event distribution.

Steady processing of hundreds of thousands to a million QPS across a cluster.

No catastrophic latency spikes even under hot keys, batch requests, replication, persistence and failover.

Correct semantics for cache, distributed lock, counter, session, leaderboard and stream use‑cases.

If any link in the following chain breaks, the "high‑concurrency myth" collapses:

Connection Acceptance
  → Event Dispatch
  → Command Parsing
  → Memory Access
  → Data‑Structure Operation
  → Result Write‑Back
  → Replication / Persistence / Cluster Forwarding
  → Client Retry & Business Fallback

1. Event‑Driven Architecture – The Real Heart of Redis

1.1 Request Execution Path

A client sends GET user:1. Internally Redis processes the request as:

Client Socket
  → TCP receive buffer
  → epoll/kqueue readable event
  → Redis event loop reads data
  → Protocol parsing
  → Key object lookup
  → Command logic execution
  → Write response to output buffer
  → Event loop writes when socket is writable

The three factors that set the performance ceiling are:

Non‑blocking network I/O – a single slow connection never blocks the whole process.

Memory‑resident data access – no random disk I/O.

Serialized command execution – eliminates lock contention on shared structures.

1.2 Event Loop vs. Thread‑Per‑Connection

Two common models:

One connection per thread (or thread‑pool).

Multiplexing + event loop.

Redis chooses the latter because most commands are short, the cost of a thread switch far exceeds command execution time, data structures are heavily shared (locking would be expensive), and the workload is I/O‑bound rather than CPU‑bound.

Typical trade‑offs (illustrated as a list instead of the original table):

One‑thread‑per‑connection : intuitive programming, but massive thread count and heavy context switches.

Thread‑pool + blocking I/O : easy to combine with business threads, but suffers from idle spinning, queueing and obvious lock contention.

epoll + event loop : high connection utilization, few context switches, but requires fine‑grained control of non‑blocking sockets and buffers.

2. Selective Multithreading Since Redis 6

Redis 6 introduced optional I/O threads. The execution model remains single‑threaded for command logic and data‑structure mutation; only network read/write can be parallelized. This compromise keeps the benefits of a lock‑free core while alleviating the network bottleneck.

Network read/write is the real performance limiter in LAN environments.

Full multithreading would require fine‑grained locks on dict, skiplist, expiration tables – a massive engineering cost.

3. Memory Layout and Object Encoding – The Real Speed Drivers

3.1 Redis Object Model

Redis objects are not plain hash entries. Each object carries a type (String, List, Hash, Set, ZSet, Stream) and an encoding that adapts to the actual data shape:

String → int for numeric values, embstr for short strings (single allocation), raw for long strings.

3.2 Simple Dynamic Strings (SDS)

Redis strings use SDS instead of C char* to avoid O(n) strlen, support binary data, and enable O(1) length retrieval and pre‑allocation.

struct sdshdr64 {
    uint64_t len;
    uint64_t alloc;
    unsigned char flags;
    char buf[];
};

Benefits: len gives O(1) length. alloc enables pre‑allocation, reducing realloc overhead.

Binary safety lets strings hold JSON, serialized objects, compressed blobs, or media indexes.

3.3 Dict – Fast KV Operations

Redis uses a hash table ( dict) with incremental rehashing. Instead of moving all buckets at once (which would block the main thread), Redis keeps the old table ( ht[0]) and the new one ( ht[1]) and migrates a small portion on each operation, turning a large pause into many cheap steps.

4. Composite Data Structures – ZSet Example

ZSet combines a hash table (O(1) member lookup) and a skiplist (ordered by score, supporting range queries). This dual‑structure design satisfies two typical use‑cases:

"What is user 10001's current points?" – hash lookup.

"Show top 100 leaderboard entries" – ordered skiplist range.

Using only a hash would make sorting heavy; using only a balanced tree would make point lookups expensive. The hybrid approach trades a modest amount of memory for stable performance.

5. Network Overhead – The Real Performance Killer

5.1 Full Request Latency Breakdown

Client request encoding.

Network transmission.

Server parsing & execution.

Response write‑back.

Client deserialization.

In LAN scenarios the server’s actual command execution often occupies a tiny fraction of total latency. Common bottlenecks are too‑small requests (RTT amplification), undersized connection pools, and improper batch strategies.

5.2 Pipeline – Reducing RTT Tax

Writing 10 000 keys can be done in three ways:

Loop 10 000 individual SET commands (3 × RTT).

One pipeline of 10 000 SET commands (single RTT).

Bulk MSET or a custom multi‑key structure (single RTT, often the fastest).

Performance differences can span an order of magnitude.

5.3 Pipeline Pitfalls

Too large a batch inflates client/server memory.

Response back‑pressure can stall slow clients.

Errors do not roll back subsequent commands.

In Cluster mode, keys crossing slots cannot be pipelined together.

Best practice: batch 100‑1000 commands, keep TTL randomised, and avoid mixing key slots.

6. Lua Scripting – Atomic Business Logic

For scenarios requiring "check‑then‑modify" semantics, Lua scripts run atomically inside Redis, providing both idempotence and state encapsulation.

-- KEYS[1]: stock key
-- KEYS[2]: reserved set key
-- ARGV[1]: order id
-- ARGV[2]: quantity
local stockKey = KEYS[1]
local reservedKey = KEYS[2]
local orderId = ARGV[1]
local quantity = tonumber(ARGV[2])
if redis.call("SISMEMBER", reservedKey, orderId) == 1 then
    return {"DUPLICATE", redis.call("GET", stockKey) or "0"}
end
local stock = tonumber(redis.call("GET", stockKey) or "0")
if stock < quantity then
    return {"INSUFFICIENT", tostring(stock)}
end
redis.call("DECRBY", stockKey, quantity)
redis.call("SADD", reservedKey, orderId)
redis.call("EXPIRE", reservedKey, 1800)
return {"OK", tostring(stock - quantity)}

This script guarantees that duplicate submissions are ignored and that stock deduction and reservation happen in a single atomic step.

7. Client‑Side Patterns

7.1 Java Connection Pool Example (Lettuce)

@Bean
public LettuceConnectionFactory redisConnectionFactory(RedisProperties properties) {
    RedisStandaloneConfiguration server = new RedisStandaloneConfiguration();
    server.setHostName(properties.getHost());
    server.setPort(properties.getPort());
    server.setPassword(RedisPassword.of(properties.getPassword()));

    GenericObjectPoolConfig<?> poolConfig = new GenericObjectPoolConfig<>();
    poolConfig.setMaxTotal(128);
    poolConfig.setMaxIdle(32);
    poolConfig.setMinIdle(8);
    poolConfig.setMaxWait(Duration.ofMillis(200));

    LettuceClientConfiguration clientConfig = LettucePoolingClientConfiguration.builder()
            .commandTimeout(Duration.ofMillis(150))
            .shutdownTimeout(Duration.ZERO)
            .poolConfig(poolConfig)
            .build();
    return new LettuceConnectionFactory(server, clientConfig);
}

Key takeaways:

Avoid unbounded waiting for a Redis connection.

Include Redis latency in the overall request timeout budget.

Fail fast or degrade gracefully when Redis is unavailable.

7.2 Rate‑Limiter Lua Script

-- KEYS[1]: rate limiter key
-- ARGV[1]: now (ms)
-- ARGV[2]: window (ms)
-- ARGV[3]: threshold
local key = KEYS[1]
local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local threshold = tonumber(ARGV[3])
redis.call("ZREMRANGEBYSCORE", key, 0, now - window)
local current = redis.call("ZCARD", key)
if current >= threshold then
    return 0
end
redis.call("ZADD", key, now, tostring(now))
redis.call("PEXPIRE", key, window)
return 1

Typical use‑cases: API gateway throttling, flash‑sale entry protection, captcha send limits, and system self‑protection.

8. Observability & Governance

Key metrics to monitor:

QPS, command distribution, network traffic.

Connection count, rejected connections, slow‑log entries.

Memory usage, fragmentation, eviction counters.

Replication lag, AOF rewrite time, RDB fork time.

Hot keys, big keys, output‑buffer back‑pressure.

Critical alarm signals: used_memory approaching maxmemory.

High mem_fragmentation_ratio.

Rising blocked_clients.

Spike in instantaneous_ops_per_sec with dropping hit‑rate.

Abnormal latest_fork_usec indicating heavy fork overhead.

9. Common Production Incidents

9.1 Big Key

Large collections (e.g., a set with hundreds of thousands of message IDs) cause long delete times, replication delays, network payload bloat, and higher persistence cost. Mitigation: split data, use incremental delete, and scan proactively.

9.2 Hot Key

Single keys that receive disproportionate traffic saturate a node or slot. Countermeasures: local cache, short‑lived replicas, key‑sharding (bucket keys), or gateway fallback.

9.3 Slow Client & Output Buffer Bloat

When a client reads slowly, Redis buffers responses, inflating memory without adding keys. This can trigger OOM. Monitoring client-output-buffer-limit and blocked_clients helps detect the issue.

9.4 Fork Overhead

RDB/AOF rewrite forks the process; large memory leads to high copy‑on‑write cost and latency spikes. Best practice: keep instance memory moderate, avoid rewrites during traffic peaks, and tune persistence policies per workload.

9.5 Expiration & Memory Fragmentation

Redis performs lazy and periodic expiration. Concentrated TTLs cause mass expirations, CPU spikes, and DB load. Randomise TTLs, avoid bulk writes with identical expiry, and monitor mem_fragmentation_ratio.

10. Evolution Roadmap from Single‑Node to Multi‑Cluster

Stage 1 – Single Node : focus on key design, TTL, cache‑penetration protection, connection pool tuning, and basic monitoring.

Stage 2 – Master‑Slave + Sentinel : adds availability, read scaling, but remember async replication and brief failover hiccups.

Stage 3 – Redis Cluster : horizontal scaling via 16 384 hash slots; requires slot‑aware key design and careful multi‑key operations.

Stage 4 – Multi‑Cluster Segmentation : isolates tenant or business domains to avoid hot‑slot or big‑key interference.

Stage 5 – Unified State Platform : provides SDKs, naming conventions, unified rate‑limit, monitoring, and change‑management pipelines.

11. Production Checklist Before Going Live

11.1 Data & Semantics

Consistent key naming, auditability, and extensibility.

Clear distinction between cache, lock, counter, and idempotency keys.

Defined TTL and jitter strategies.

Explicit mapping of which keys carry business semantics.

11.2 High‑Concurrency Path

Hot‑key local caching or multi‑level protection.

Back‑fill mechanisms (single‑flight, mutex) for cache misses.

Connection‑pool sizing and timeout budgets validated by load tests.

Verified pipeline/Lua/multi‑key behavior under Cluster slot constraints.

11.3 Consistency & Compensation

Idempotent stock, lock, and reservation states with compensation flows.

Rollback or repair paths when Redis succeeds but downstream DB fails.

Message‑driven retry and timeout handling.

11.4 Stability & Governance

Slow‑log, hit‑rate, fragmentation, replication lag, fork time monitoring.

Periodic big‑key / hot‑key scans.

Client output‑buffer limits enforced.

Failover, node removal, and scaling rehearsals.

11.5 Change Management & Drills

Backward‑compatible key schema migrations.

Impact analysis for eviction or persistence policy changes.

Capacity planning and stress testing before major promotions.

Runbooks and on‑call drills for node failures and data loss scenarios.

12. When Redis Is Not the Right Choice

Strongly consistent financial ledgers.

Complex multi‑table transactional guarantees.

Long‑term massive raw data storage.

Unbounded collections without TTL.

In such cases Redis should serve as a fast‑path cache, coordination layer, or state engine, not as the sole source of truth.

13. Final Takeaway

Redis’s ability to survive million‑level concurrency stems from a holistic design:

Event‑loop‑driven non‑blocking I/O.

Compact, adaptive in‑memory encodings.

Serialized command execution that removes lock contention.

Pipeline, batch, and Lua techniques that cut network RTT.

Replication, sharding, persistence, observability, rate‑limiting, and compensation that elevate a single node into a production‑grade system.

When Redis is positioned as a high‑frequency state engine, entry‑point decision maker, multi‑level cache core, or distributed coordination component – and when its surrounding ecosystem supplies idempotence, compensation, monitoring, and operational discipline – its value far exceeds the simplistic claim "Redis is fast because it is single‑threaded".