Why Redis Added Streams and How to Use Them Effectively

1. Why Add Streams?

The use of stream data is growing, and Redis creator antirez is actively considering how to support stream scenarios.

He believes existing Redis data structures cannot handle streams well:

(1) Sorted sets – elements change by score and cannot naturally model continuously arriving messages, nor support client blocking for new messages.

(2) Lists – blocking is one element per client and list elements lack a fixed identifier.

(3) Pub/Sub – suitable for one‑to‑many, but messages disappear quickly, lacking history, reconnection replay, and range queries needed for time‑series use cases.

Antirez tried to extend these structures (e.g., adding history to Pub/Sub or flexible access to lists) but found them unsatisfactory, so he decided to create a new data structure.

2. Implementation Process

Inspired by a log‑type Redis module, antirez chose to implement Streams using a log.

A log is a simple append‑only file, but ordinary log offsets have no logical meaning and are hard to garbage‑collect; ideal logs allow specifying a number to delete older entries.

The radix tree used in Redis Cluster provides an efficient foundation for Streams, offering both space and access‑time efficiency.

Antirez also studied Kafka’s approach (log‑based) and adopted concepts such as consumer groups.

He envisions Redis Streams playing a key role in event‑driven and messaging applications, especially for time‑series scenarios.

3. Command Examples

Key Features of Streams

Elements are objects composed of multiple field‑value pairs, not just simple strings.

Range queries are convenient and efficient.

Different clients can block waiting for new elements and can specify the starting ID.

Example

> XADD mystream * sensor-id 1234 temperature 10.5
1506871964177.0

The XADD command adds a new element to the specified stream.

In this example, mystream is the target stream, and the new element contains two fields: sensor-id and temperature.

Fields can differ between elements in the same stream, but using the same field names improves memory efficiency.

XADD returns the ID of the newly inserted element; the asterisk (*) lets Redis generate the ID automatically, though a custom ID can also be provided.

The ID consists of two parts: a millisecond timestamp and a sequence number, separated by a dot. The sequence distinguishes elements added at the same millisecond.

The timestamp is derived from the greater of the server’s local time and the maximum timestamp already present in the stream, ensuring monotonically increasing IDs even if the server clock is adjusted backward.

> MULTI
OK
> XADD mystream * foo 10
QUEUED
> XADD mystream * bar 20
QUEUED
> EXEC
1) 1506872463535.0
2) 1506872463535.1

This demonstrates fast batch insertion and shows that fields can differ between elements.

> XRANGE mystream 1506871964177.0 1506871964177.0
1) 1) 1506871964177.0
   2) 1) "sensor-id"
      2) "1234"
      3) "temperature"
      4) "10.5"

The XRANGE command retrieves elements from a stream within a given range, inclusive of the start and end IDs.

4. Summary

Redis Streams will extend Redis to cover more use cases, especially time‑series, and is slated for release in the 4.0 series later this year; the core functionality is already complete, and interested users can try the streams branch on GitHub.