Push vs Pull: Designing a Scalable Feed Timeline with Redis and Cassandra

Feed timelines aggregate posts from creators a user follows and display them in reverse chronological order. Two classic architectures exist: a pull model that reads followers' posts on demand, and a push model that writes each new post to every follower’s timeline.

Pull Model

When a user opens the timeline, the system first queries the followings table to obtain all followed creator IDs, then selects their articles and orders them by create_time. The SQL statement is:

SELECT *
FROM articles
WHERE author_uid IN (
    SELECT following_uid
    FROM followings
    WHERE uid = ?
)
ORDER BY create_time DESC
LIMIT 20;

This approach requires reading N authors’ articles for each page load (read amplification) and performing a large temporary‑table sort, which can become a CPU bottleneck when the follow list is long. It needs no extra storage and is simple to implement, but latency grows with the number of followees.

Push Model

When a creator publishes an article, the service writes a copy of that article into the timeline of each of the creator’s M followers (write amplification). The write cost spikes for popular creators with hundreds of thousands of followers, potentially overwhelming the database. To mitigate latency, the write is often performed asynchronously via a message queue, and additional logic is required for follow/unfollow events.

Online Push, Offline Pull (Hybrid)

Because only a small fraction of followers are active at any moment, the system can push new posts only to active users while inactive users rebuild their timelines on demand using the pull model. The timeline thus acts as a cache that stores recent, hot data and expires after a configurable period.

Implementation uses Redis SortedSet where the member is article_id and the score is the publish timestamp. Example:

ZADD user:timeline:123 1666850112.11 "article_11"

Idempotency is guaranteed because duplicate pushes of the same article_id do not create duplicate members. Cache‑penetration is handled by inserting a sentinel member NoMore to indicate that the underlying database has no more older entries: ZADD user:timeline:456 0 "NoMore" If a timeline is missing in Redis, the service falls back to the pull query to reconstruct it. The presence of the NoMore marker also distinguishes an empty timeline from a cache miss.

Multi‑Level Caching

When Redis memory is insufficient, a second‑level cache such as Cassandra can store the same sorted data. A sample Cassandra table definition:

-- Cassandra stores a map of PartitionKey to SortedMap
CREATE TABLE taojin.following_timelines (
    uid bigint,
    publish_time timestamp,
    article_id bigint,
    PRIMARY KEY (uid, publish_time, article_id)
) WITH default_time_to_live = 60*24*60*60;

Each additional cache layer introduces consistency considerations that must be weighed against the performance gains.

Pagination Strategies

Traditional LIMIT + OFFSET pagination suffers from duplicate or missing entries when new posts appear between page requests. A cursor‑based approach uses the last item’s identifier or timestamp as the next page’s starting point. To avoid collisions when timestamps are identical, the feed ID is appended as a fractional part of the score (e.g., 1666850112.11 for article 11).

Large‑Scale Push Processing

Pushing a feed to millions of followers is split into sub‑tasks dispatched via a message queue to multiple worker instances. If a sub‑task fails, the queue automatically retries it, ensuring reliability without manual intervention.

Key Takeaways

Pull model offers simplicity and storage efficiency but suffers from high read latency under large follow lists.

Push model provides fast reads but incurs massive write amplification for popular creators.

Hybrid "online push, offline pull" limits push traffic to active users and treats timelines as a cache with expiration.

Redis SortedSet is ideal for ordered, deduplicated timelines; a sentinel NoMore flag prevents cache‑penetration issues.

Cassandra can serve as a secondary cache when Redis memory is constrained.

Cursor‑based pagination using timestamps plus feed‑ID fractions avoids duplicate entries.

Sharding large push jobs across multiple workers via a message queue improves scalability and fault tolerance.