Design and Architecture of Feed Stream Systems

This article provides a comprehensive overview of Feed Stream systems, covering their evolution, core concepts, classification, and architectural design.

1. Understanding Feed Streams A Feed Stream is a continuously updated information flow that aggregates content from multiple sources based on user subscriptions or behavior. It is personalized ("千人千面") and appears in many mobile apps as "recommended" or "friend" feeds.

2. Why Feed Streams Exist Traditional media (TV, newspapers) require users to actively seek information. Feed Streams aggregate relevant content automatically, improving user experience by delivering personalized items such as "You may like" or friend updates.

3. Classification of Feed Streams - By aggregation logic: no dependency (e.g., Douyin recommendation), single‑direction follow (e.g., Weibo follow page), bidirectional friendship (e.g., WeChat Moments). - By presentation: weight‑based ranking vs. time‑ordered timeline.

4. Historical Background Feed Streams originated from RSS aggregators and evolved into social‑media News Feeds (e.g., Facebook) that combine content creation and social relationships.

5. Terminology Key terms include Feed, Feed Stream, Timeline, Inbox/Outbox, Write‑diffusion, Read‑diffusion, etc.

6. Architecture Design The system consists of message publishing, inbox (Redis ZSET), message tables, publishing configuration, and follow‑relationship tables. The overall flow is: a user publishes a message → it enters a queue → the system writes the message to the author’s outbox → synchronizes the message ID to followers’ inboxes (write‑diffusion) → followers read from their inboxes (read‑diffusion when needed).

7. Core Challenges and Solutions - Real‑time performance: high write/read throughput. - Massive data volume: use Redis ZSET for sorted inboxes. - Consistency: soft‑delete + lazy‑delete to avoid missing IDs during pagination. - Large‑v (big‑V) handling: hybrid write/read diffusion with hot‑cold fan‑out.

8. Pagination Strategy Instead of page number/size, use last_id to fetch the next page, ensuring stable ordering even when items are added or removed.

9. Data Model Tables include Message (msg_id, title, content, type, status, channel, timestamps), Inbox (Redis ZSET keyed by user+channel), PublishConfig (send_type, crontab, channel), and FollowRelation (main_uid, follower_uid, status, hot_follower).

10. Publishing Process 1) Message enters queue. 2) Retrieve follower list and determine if author is a big‑V. 3) Write to author’s timeline. 4) Write message ID to followers’ inboxes (write‑diffusion). 5) End.

11. Reading Process 1) Determine if user is active; if not, fetch big‑V list. 2) Read from inbox using last_id . 3) For each big‑V, pull their outbox if needed. 4) Merge, sort by timestamp, and return.

12. Summary The article outlines a reliable, high‑performance Feed Stream design that balances write‑diffusion for active users and read‑diffusion for large‑V scenarios, with soft‑delete handling and efficient pagination.