Understanding Redis Blocking: Commands, Persistence, and Big Key Pitfalls

Command‑level Blocking

Using commands that require full scans can block the client because Redis processes each request in a single thread. Typical offenders are: KEYS * – scans all keys. HGETALL – returns every field in a hash. SMEMBERS – returns all members of a set.

All have O(N) time complexity and may trigger a complete table scan, causing the server to become unresponsive as N grows.

SAVE Command Blocking

During an RDB snapshot Redis forks a child process to write the snapshot file. The BGSAVE command triggers this non‑blocking fork, while the manual SAVE command runs in the main thread and blocks it until the snapshot is finished.

Synchronous AOF Persistence

When appendfsync always is configured, every write is flushed to disk immediately. Disk I/O is relatively slow, so each write can stall the main thread, degrading performance.

AOF Rewrite Blocking

Redis forks a child process to create a new AOF file using the BGREWRITEAOF command. While the child writes the new file, the parent continues to serve requests and buffers all write commands in an AOF rewrite buffer.

When the child finishes, the parent appends the buffered commands to the new AOF file, ensuring the new file reflects the current database state.

Finally, the new AOF replaces the old one.

The blocking point occurs in step 2, when the buffered data is flushed to the new file, because this operation runs in the main thread.

AOF Logging Characteristics

AOF logs are written after command execution, without a prior syntax check, to avoid extra overhead. The logging itself runs in the main thread; if the disk is under heavy write pressure, the logging can become a bottleneck and block subsequent operations.

Big‑Key Issues

A “big key” refers to a key whose value size is large, not the key name itself. Big keys cause several problems:

Client timeout blocking: Processing a big key takes longer, so the client sees a long pause.

Network blocking: Transferring a 1 MB key at 1 000 requests per second generates 1 GB/s traffic, overwhelming typical 1 Gbps network interfaces.

Worker‑thread blocking: Deleting a big key with DEL blocks the thread that would handle subsequent commands.

Finding Big Keys

Use the --bigkeys option of the Redis CLI on a replica to avoid blocking the master. Alternatives include:

Running SCAN with appropriate patterns and measuring value sizes.

Analyzing RDB files with tools such as redis‑rdb‑tools (Python) or rdb_bigkeys (Go) when the instance uses RDB persistence.

Deleting Big Keys Safely

Removing a large key frees memory, but the operating system must insert the freed block into its free‑list, which takes time and can block the main thread. If many large keys are deleted at once, the server may become unresponsive, leading to connection exhaustion.

It is recommended to delete big keys in batches or use asynchronous deletion mechanisms.

Flushing the Database

Commands like FLUSHDB and FLUSHALL delete all keys and free their memory, presenting the same blocking characteristics as big‑key deletions.

Cluster Scaling and Migration

Redis Cluster supports dynamic node scaling, but the process is only semi‑automatic and requires manual intervention. Data migration is performed synchronously; both source and target nodes block while transferring data. For small keys the pause is negligible, but large keys can cause prolonged blocking and may even trigger a failover.

Reference: https://mp.weixin.qq.com/s/Q4Etl5xJZtOhXgy8UK_69g