How to Prevent Redis Blocking: Strategies for Long‑Running Commands and Fork‑Based Operations

Redis processes events in a single‑threaded event loop, so keeping command latency short is essential; otherwise later tasks get blocked, especially when the dataset grows to tens of gigabytes.

Long‑Running Commands Cause Blocking

keys, sort and similar commands

The keys command scans all keys (O(N)) and can block the thread for seconds when the database contains millions of keys. The same applies to sort and set‑union operations.

Solution: Separate slow and fast requests by routing expensive commands to a dedicated Redis slave that does not serve online traffic, leaving the primary instance for low‑latency operations.

smembers command

smembers

returns the entire set (O(N)). When a set holds millions of members, the operation blocks the event thread.

Solution: Keep sets small (e.g., under 500 elements) by sharding data across multiple keys (monthly or daily keys) or, if a large set is unavoidable, use SRANDMEMBER key [count] to fetch a sample instead of iterating the whole set.

save command

The save command performs persistence in the event thread; with large datasets it can block the server for many seconds.

Solution: Prefer bgsave (background save) for persistence, avoiding blocking the main thread.

Blocking Caused by Fork

Redis forks a child process for heavy operations such as RDB snapshots, AOF rewrites, and master‑to‑slave synchronization. Although copy‑on‑write shares pages, the page‑table copy can be costly (e.g., a 40 GB instance may copy an 80 MB page table, causing >200 ms pause).

Initial master‑to‑slave sync creates a child that dumps memory to disk.

AOF rewrite also forks a child to rewrite the log.

Solutions:

Limit each Redis instance’s max memory (recommended ≤20 GB, preferably ≤10 GB on virtual machines) to reduce fork latency.

Use large memory pages (e.g., 4 MB) to shrink the page‑table size, though this increases memory usage during copy‑on‑write.

Prefer physical servers over virtualized environments for lower fork overhead.

Avoid forking altogether: run a single node without persistence for cache‑only use, or isolate persistence to a dedicated master that does not serve reads, while slaves handle queries.

Persistence‑Induced Blocking

Both AOF and RDB persistence can heavily impact performance, especially on nodes already handling other I/O.

Child process write vs. parent fsync

When appendfsync is set to everysec or always, the parent process calls fsync() each second. Heavy disk I/O can block this call, stalling the entire server.

Solution: Set no-appendfsync-on-rewrite yes so the parent skips fsync() during AOF rewrite, accepting up to 30 seconds of potential data loss.

System sync causing write block

Large amounts of dirty kernel buffers trigger long sync operations, blocking Redis writes.

Solution: Tune kernel parameters ( vm.dirty_background_ratio or vm.dirty_bytes) to limit the amount of data synced at once (e.g., 32 MB). Redis 2.6.12+ also calls fdatasync after 32 MB during AOF rewrite.

AOF rewrite merge block

After a background AOF rewrite, the parent merges buffered writes into the new AOF file, which can block for seconds if the buffer is large.

Solution: Increase disk capacity, raise the AOF rewrite threshold, and schedule rewrites during idle periods.

Source: 大CC (http://www.cnblogs.com/me115/p/5032177.html)