Ensuring Safe Redis Queues with RPOPLPUSH and Blocking Commands

Redis list data structures make it easy to implement a message queue, e.g., using LPUSH/BLPUSH to enqueue and RPOP/BRPOP to dequeue.

However, when LPOP (or RPOP) returns an element, it is removed from the list; if the client crashes before processing, the message is lost.

Redis provides the RPOPLPUSH command (and its blocking variant BRPOPLPUSH) which atomically returns and removes the last element from a source list and pushes it to the head of a destination list.

Using RPOPLPUSH for message retrieval ensures safety: the message is delivered to the client while simultaneously being stored in a backup list, and the operation is atomic. After successful processing, the message can be removed from the backup list.

Redis commands often have both blocking and non‑blocking forms. For example, LPUSH is non‑blocking while BLPUSH is blocking. The only difference is that blocking commands (e.g., BRPOP) wait indefinitely (or for a specified timeout) when the list is empty, whereas non‑blocking commands (e.g., RPOP) return nil immediately.

In practice, a non‑blocking loop might repeatedly poll the queue:

# Infinite loop reading tasks
loop
$task = RPOR queue
if $task
    # Execute if task exists
    execute($task)
else
    # Wait 1 second if no task
    wait 1 second

This approach wastes resources because it queries the queue every second even when empty. A blocking approach lets the server notify the consumer when a new task arrives:

# Blocking loop
loop
    # BRPOP blocks until a task appears (0 = wait forever)
    $task = BRPOP queue, 0
    # Execute the returned task
    execute($task[1])