Is Redis a Viable Message Queue? List, Pub/Sub, and Stream Compared

Introduction

Many developers wonder whether Redis is suitable as a message queue. This article walks through the pros and cons of using Redis List, Pub/Sub, and the newer Stream data type for queuing, and discusses the scenarios where Redis can or cannot replace a dedicated message‑queue middleware.

List as a Simple Queue

When the workload is simple, the Redis List type can be used as a queue because it is implemented as a linked list, giving O(1) operations at both ends.

Typical producer code uses LPUSH to push messages, while the consumer uses RPOP to retrieve them:

<code>127.0.0.1:6379> LPUSH queue msg1
(integer) 1
127.0.0.1:6379> LPUSH queue msg2
(integer) 2</code>

<code>127.0.0.1:6379> RPOP queue
"msg1"
127.0.0.1:6379> RPOP queue
"msg2"</code>

If the list is empty, RPOP returns nil . A naïve consumer loop that continuously calls RPOP will waste CPU cycles (busy‑spinning) and put pressure on Redis.

One simple fix is to add a sleep when no message is returned:

<code>while true:
    msg = redis.rpop("queue")
    if msg == null:
        sleep(2)
        continue
    handle(msg)
</code>

This eliminates CPU waste but introduces latency: a newly published message may wait up to the sleep interval before being processed.

Pub/Sub Model

Redis also provides a publish/subscribe mechanism ( PUBLISH / SUBSCRIBE ) that solves the duplicate‑consumer problem of Lists. Multiple consumers can subscribe to the same channel and receive each message.

Example:

<code># Two consumers subscribe to the same channel
127.0.0.1:6379> SUBSCRIBE queue
# Producer publishes a message
127.0.0.1:6379> PUBLISH queue msg1
# Both consumers receive the message
1) "message"
2) "queue"
3) "msg1"
</code>

Pub/Sub also supports pattern subscriptions ( PSUBSCRIBE ) to receive messages from multiple matching channels.

However, Pub/Sub has serious drawbacks: messages are not persisted, so if a consumer is offline or Redis restarts, the messages are lost. Additionally, if a consumer crashes after receiving a message but before processing it, the message is gone.

Stream – A More Mature Queue

Redis 5.0 introduced the Stream data type, which combines the advantages of Lists and Pub/Sub while adding durability and consumer‑group semantics.

Basic production and consumption use XADD and XREAD :

<code># Producer adds two entries
127.0.0.1:6379> XADD queue * name zhangsan
"1618469123380-0"
127.0.0.1:6379> XADD queue * name lisi
"1618469127777-0"
# Consumer reads from the beginning
127.0.0.1:6379> XREAD COUNT 5 STREAMS queue 0-0
1) 1) "queue"
   2) 1) 1) "1618469123380-0"
          2) 1) "name"
               2) "zhangsan"
      2) 1) "1618469127777-0"
          2) 1) "name"
               2) "lisi"
</code>

Blocking reads are possible with the BLOCK option, so the consumer can wait indefinitely for new entries:

<code>127.0.0.1:6379> XREAD COUNT 5 BLOCK 0 STREAMS queue 1618469127777-0</code>

Consumer groups enable multiple independent consumer groups to read the same stream without interfering with each other. Creating groups:

<code>127.0.0.1:6379> XGROUP CREATE queue group1 0-0
127.0.0.1:6379> XGROUP CREATE queue group2 0-0
</code>

Each group can have several consumers. A consumer reads with XREADGROUP and acknowledges processed messages with XACK . If a consumer crashes before acknowledging, the message remains pending and can be re‑delivered when the consumer restarts:

<code># Consumer in group1 reads
127.0.0.1:6379> XREADGROUP GROUP group1 consumer COUNT 5 STREAMS queue >
# After processing, acknowledge
127.0.0.1:6379> XACK queue group1 1618472043089-0
</code>

All Stream write operations are persisted to Redis RDB/AOF, so data survives server restarts. To avoid unbounded memory growth, a maximum length can be set with MAXLEN :

<code>127.0.0.1:6379> XADD queue MAXLEN 10000 * name zhangsan
"1618473015018-0"
</code>

When the length limit is reached, older entries are trimmed, which may still cause data loss if the limit is too low for the workload.

Comparison with Professional Message Queues

Professional queues (e.g., Kafka, RabbitMQ) guarantee no data loss through replication and durable storage, and they handle large backlogs by writing to disk. Redis, being an in‑memory store, can lose data in two main situations:

AOF configured to write every second is asynchronous; a crash may lose recent writes.

Master‑replica replication is asynchronous; a failover can lose writes that have not yet been replicated.

Furthermore, when a backlog grows, Redis memory may be exhausted, whereas disk‑based queues can continue to accept messages.

When to Use Redis as a Queue

If the use case is simple, data loss is acceptable, and the expected traffic is modest, Redis List or Stream can provide a lightweight, easy‑to‑deploy solution. For high‑throughput, loss‑intolerant scenarios, or when strict ordering and durability are required, a dedicated message‑queue system is recommended.

Conclusion

The article has compared Redis List, Pub/Sub, and Stream as queue mechanisms, highlighted their strengths and weaknesses, and positioned Redis as a viable option for simple or low‑risk workloads while acknowledging its limitations compared to specialized queue middleware.