Is Redis a Viable Message Queue? List, Pub/Sub, and Stream Compared to Kafka & RabbitMQ

Redis List Queue

Redis List is a double‑ended linked list where both head and tail operations run in O(1) time, making it suitable for simple FIFO queues.

Typical usage:

127.0.0.1:6379> LPUSH queue msg1
(integer) 1
127.0.0.1:6379> LPUSH queue msg2
(integer) 2

Consumers pull messages with RPOP:

127.0.0.1:6379> RPOP queue
"msg1"
127.0.0.1:6379> RPOP queue
"msg2"

Problem 1: CPU spin‑loop – continuously calling RPOP when the list is empty wastes CPU cycles. A common mitigation is to sleep when no message is found, but this adds latency.

Problem 2: Lack of blocking reads – Redis provides BRPOP / BLPOP for blocking reads, which pause the consumer until a new item arrives:

while true:
    msg = redis.rpop("queue")
    if msg == null:
        continue
    handle(msg)

Using the blocking variant eliminates empty‑loop waste:

while true:
    // block indefinitely (0 = no timeout)
    msg = redis.brpop("queue", 0)
    if msg == null:
        continue
    handle(msg)

Limitations of List:

No multi‑consumer support : a message can be consumed by only one consumer.

Message loss on consumer failure : once RPOP removes the item, a processing failure discards it.

Publish/Subscribe Model (Pub/Sub)

Redis Pub/Sub is suited for broadcast scenarios where multiple consumers receive the same message.

// Two consumers subscribe the same channel
127.0.0.1:6379> SUBSCRIBE queue
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "queue"
(integer) 1

When a producer publishes a message, all subscribed consumers are unblocked:

127.0.0.1:6379> PUBLISH queue msg1
(integer) 1

127.0.0.1:6379> SUBSCRIBE queue
// received new message
1) "message"
2) "queue"
3) "msg1"

Drawbacks:

If a consumer disconnects, any messages sent during its absence are lost.

Redis does not guarantee persistence; a crash can discard in‑flight messages.

Message backlog can cause buffer overflow and force consumers offline.

Therefore Pub/Sub is best for low‑reliability real‑time broadcasts (e.g., chat, notifications), not for critical business pipelines.

Redis Stream – A More Mature Queue

Streams introduce persistent, ID‑based messaging with built‑in consumer groups.

XADD – add a message ("*" generates a unique ID).

XREAD – read messages from a stream.

127.0.0.1:6379> XADD queue * name zhangsan
"1618469123380-0"
127.0.0.1:6379> XADD queue * name lisi
"1618469127777-0"

Reading the first five entries:

127.0.0.1:6379> XREAD COUNT 5 STREAMS queue 0-0
1) 1) "queue"
   2) 1) "1618469123380-0"
       "name" "zhangsan"
   2) 1) "1618469127777-0"
       "name" "lisi"

If no new messages exist, Redis returns (nil):

127.0.0.1:6379> XREAD COUNT 5 STREAMS queue 1618469127777-0
(nil)

Blocking reads are possible with the BLOCK option:

127.0.0.1:6379> XREAD COUNT 5 BLOCK 0 STREAMS queue 1618470740565-0

Consumer groups enable multiple independent consumers to process the same stream without duplication:

127.0.0.1:6379> XGROUP CREATE queue group1 0-0
OK
127.0.0.1:6379> XGROUP CREATE queue group2 0-0
OK

Each group can have its own consumer:

127.0.0.1:6379> XREADGROUP GROUP group1 consumer COUNT 5 STREAMS queue >
...messages for group1...

127.0.0.1:6379> XREADGROUP GROUP group2 consumer COUNT 5 STREAMS queue >
...messages for group2...

After processing, a consumer acknowledges the message with XACK to prevent loss:

127.0.0.1:6379> XACK queue group1 1618472043089-0

If a consumer crashes before acknowledging, the message remains pending and can be re‑read after the consumer restarts:

127.0.0.1:6379> XREADGROUP GROUP group1 consumer1 COUNT 5 STREAMS queue 0-0
...re‑delivered pending messages...

Persistence: Stream entries are written to Redis' RDB and AOF files like any other data type, so configuring standard persistence policies ensures durability across restarts.

Comparison with Professional Message Queues

Professional systems such as Kafka and RabbitMQ guarantee two core properties:

No message loss – they replicate data across multiple nodes.

Back‑pressure handling – they store messages on disk, allowing unlimited backlog without exhausting RAM.

Key questions for any queue:

Can the producer lose messages? Network failures may cause publish failures; retries or idempotent producers are required.

Can the consumer lose messages? Consumers must acknowledge receipt; otherwise the broker may redeliver.

Can the broker itself lose messages? Redis AOF is asynchronous and master‑slave replication is also asynchronous, so crashes can lead to data loss, whereas Kafka/RabbitMQ use synchronous replication.

How to handle backlog? Redis stores data in memory, so a large backlog can cause OOM; Streams allow a max‑length cap, but disk‑based brokers handle large backlogs more gracefully.

Conclusion

Redis can function as a lightweight queue for simple use‑cases, especially when ease of deployment outweighs strict durability requirements. However, it has two major drawbacks: potential data loss under certain failure scenarios and limited scalability due to in‑memory storage. For mission‑critical pipelines with high reliability and large backlogs, dedicated message‑queue systems like Kafka or RabbitMQ remain the preferred choice.

Technical selection should consider not only functional fit but also the organization’s operational resources and expertise.