Redis‑Backed Timeline Implementation with MongoDB for a Subscription Feed

In the previous article a user broadcast mechanism was designed with pseudocode; this follow‑up implements the timeline using MongoDB and Redis, addressing details omitted before.

Because broadcast rules are complex, the timeline is best stored in a queue rather than retrieved by database queries each time. The status collection contains two types, Status::Topic and Status::Reply , sharing fields such as user_id and topic_id while having their own attributes.

The timeline rules include hiding a user’s own status, excluding direct replies marked as targeted , showing statuses from followed users, from favorite tags, from subscribed topics, and ordering by creation time.

Implementing these rules with a MongoDB $in query would produce very large $in lists, so a Redis list is used as a cache. Redis supports fast insertion at the head and trimming to a fixed length, which matches the required operations:

def push_status(status)
  $redis.lpush store_key, status.id
  $redis.ltrim store_key, 0, Stream.status_limit - 1
end

def status_ids(start = 0, stop = -1)
  $redis.lrange store_key, start, stop
end

When the timeline is read, the IDs are fetched from Redis and the corresponding documents are retrieved from MongoDB. Rebuilding the timeline is needed after a service crash or when a user adds a new subscription; this can be performed as a background job using the same MongoDB query logic.

Data integrity issues such as stale IDs left after deletions are considered acceptable for most web applications because read‑write traffic outweighs delete operations; stale IDs are eventually removed during a rebuild or when they fall off the list.

In summary, the article demonstrates a practical Redis‑backed timeline cache combined with MongoDB storage, providing a scalable solution for subscription‑based feeds while acknowledging trade‑offs in completeness and future extensions like user blocking and topic muting.