Master Redis Streams: From Simple Adds to Powerful Consumer Groups

1. Introduction

Redis 5 introduces a major new feature: STREAM, a log‑structured storage that appends data with timestamp IDs and provides convenient read models, making it ideal for message queues and time‑series storage.

2. Environment Setup

Run a Redis 5 container:

docker run --name redis5 -p 6379:6379 -d redis:5.0-rc3

Start a Redis client:

docker run -it --link redis5:redis --rm redis-cli -p 6379

3. Practice

3.1 Adding Elements to a Stream

Add a record with sensor data:

redis:6379> XADD mystream * sensor-id 1234 temperature 19.8

The returned ID is a timestamp plus a sequence number (e.g., 1531989605376-0). You can get the stream length with:

redis:6379> XLEN mystream

3.2 Range Queries

Query a time range by specifying start and end IDs:

redis:6379> XRANGE mystream 1531989605376 1531989605377

Use - and + for minimum and maximum IDs, and COUNT to limit results.

3.3 Listening for New Elements

Read new entries with blocking: redis:6379> XREAD COUNT 2 STREAMS mystream 0 Block indefinitely until new data arrives:

redis:6379> XREAD BLOCK 0 STREAMS mystream $

3.4 Consumer Groups

Create a consumer group for a stream: redis:6379> XGROUP CREATE mystream mygroup01 $ Add test messages:

redis:6379> XADD mystream * message apple
redis:6379> XADD mystream * message orange
redis:6379> XADD mystream * message strawberry
redis:6379> XADD mystream * message banana

Read from the group:

redis:6379> XREADGROUP GROUP mygroup01 Alice COUNT 1 STREAMS mystream >

Acknowledge a processed message:

redis:6379> XACK mystream mygroup01 1531999977149-0

3.4.5 Failure Handling

List pending (delivered but not acknowledged) messages: redis:6379> XPENDING mystream mygroup01 - + 10 Transfer ownership of idle messages to a new consumer:

redis:6379> XCLAIM mystream mygroup01 Gates 3600 1531999980272-0 1531999983493-0

3.5 Stream Information

Get basic stream info: redis:6379> XINFO STREAM mystream Get group info: redis:6379> XINFO GROUPS mystream Get consumer info:

redis:6379> XINFO CONSUMERS mystream mygroup01

3.7 Deleting Messages

Delete a specific entry:

redis:6379> XDEL mystream 1531989605376-0

Note: XDEL only marks the entry as deleted; memory is not reclaimed immediately.

3.8 Limiting Stream Length

Append entries while keeping only the latest two:

redis:6379> XADD mystream MAXLEN 2 * value 1
redis:6379> XADD mystream MAXLEN 2 * value 2
redis:6379> XADD mystream MAXLEN 2 * value 3

Check length:

redis:6379> XLEN mystream

4. Summary

The article covers the fundamentals of Redis Streams, including creation, data insertion, range queries, blocking reads, consumer groups, pending message handling, ownership claiming, stream inspection, deletion, and length trimming, providing a solid foundation for using streams as a reliable message queue.