How to Use Redis Sorted Sets and Hashes for Real‑Time Live‑Stream Rankings and IM Sync

Background

Live‑stream rooms are the presentation layer of a streaming system and contain video, online users, gifts, comments, likes, leaderboards, etc. Because these messages are highly time‑sensitive, the system requires sub‑second latency, making Redis an appropriate caching and data‑structure layer.

Live‑Stream Information

Real‑time Ranking (SortedSet)

Ranking data such as online‑user lists, gift leaderboards, and bullet‑screen (danmu) messages can be stored in a Redis SortedSet. Each member has a numeric score, which enables fast ordering by time or other criteria.

Example – add five danmu messages for user55 using a Unix timestamp (ms) as the score:

redis> ZADD user55:_danmu 1523959031601166 message111111111111
(redis) (integer) 1
redis> ZADD user55:_danmu 1523959031601266 message222222222222
(integer) 1
redis> ZADD user55:_danmu 1523959088894232 message33333
(integer) 1
redis> ZADD user55:_danmu 1523959090390160 message444444
(integer) 1
redis> ZADD user55:_danmu 1523959092951218 message5555
(integer) 1

Retrieve the latest three messages (highest score first):

redis> ZREVRANGEBYSCORE user55:_danmu +inf -inf LIMIT 0 3
1) "message5555"
2) "message444444"
3) "message33333"

Retrieve three messages within a specific time range (score ≥ 1523959088894232):

redis> ZREVRANGEBYSCORE user55:_danmu 1523959088894232 -inf LIMIT 0 3
1) "message33333"
2) "message222222222222"
3) "message111111111111"

Count‑type Information (Hash)

Metrics that are simple counters—e.g., unread messages, follower count, fan count, experience points—fit naturally into a Redis Hash. Each field stores a numeric value that can be incremented atomically.

Example – store and increment a follower count:

redis> HSET user:55 follower 5
(integer) 1
redis> HINCRBY user:55 follower 1   # increment by 1
(integer) 6
redis> HGETALL user:55
1) "follower"
2) "6"

Timeline Information (List / SortedSet)

Time‑ordered streams such as broadcaster activities or new posts can be stored with a List (push‑new‑at‑head) or a SortedSet (score = timestamp). The following shows a List‑based timeline:

redis> LPUSH user:55_recent_activity '{datetime:201804131910,type:publish,title:请假,content:抱歉，今天有事鸽一天}'
(integer) 1
redis> LPUSH user:55_recent_activity '{datetime:201804112010,type:publish,title:开播啦,content:加油}'
(integer) 2
redis> LRANGE user:55_recent_activity 0 10
1) "{datetime:201804131910,type:publish,title:请假,content:抱歉，今天有事鸽一天}"
2) "{datetime:201804112010,type:publish,title:开播啦,content:加油}"

Live‑Room IM – Message Synchronization

Unified Message Structure

struct message {
  int   type;   // business type identifier
  string data;  // JSON or other serialized payload
}

Storage Requirements

Each user has a message_inbox that holds recent messages.

Every new message receives a monotonically increasing sync_id to preserve global order.

Each client device stores the last sync_id it has processed; this per‑device cursor enables selective replay.

Redis Implementation

Use a Redis SortedSet (key = user’s inbox) to map sync_id → serialized message. Use a Redis Hash (key = user’s inbox) to map device_id → last processed sync_id. The INCR command provides an atomic counter for sync_id.

Scenario Example

When a new message arrives, the system:

Generates a new sync_id with INCR.

Stores the message in the user’s SortedSet using ZADD.

Fetches all device cursors from the Hash.

For each device, reads the range of messages whose sync_id is greater than the stored cursor (using ZRANGEBYSCORE or ZREVRANGEBYSCORE).

Pushes the missing messages to the device.

Updates the device’s cursor in the Hash with HSET.

Key command snippets:

# 1. Insert new message
sync_id = INCR bob
ZADD bob $sync_id "message:{type:new_message, data:'{\"msgid\":991,\"cid\":123,\"text\":\"hello\"}'}"

# 2. Retrieve a range of messages
ZRANGEBYSCORE bob 100103 100310

# 3. Get per‑device sync points
HGETALL bob

# 4. Add or update a device cursor
HSET bob dev_1001 100103
HSET bob dev_1002 100202

The following diagram illustrates the flow from message insertion to per‑device push: