Design and Implementation of an Enterprise Feed Stream System

Background

The requirement is to display the activity feed of people a user follows, which involves designing a feed stream system suitable for typical enterprises.

Related Concepts

What is a Feed?

Feed : Each status or message in a feed stream, e.g., a Weibo post.

Feed Stream : Continuously updated information flow presented to users, such as a personal timeline or a follow page.

Feed Stream Classification

Timeline : Ordered by creation time; each feed is important and should be shown (e.g., WeChat Moments, Weibo).

Rank : Ordered by non‑time factors like user preferences, typical for news or product recommendation.

Design

Designing a feed stream system involves two key steps: initializing the feed stream and pushing updates. Persistence is usually handled with MySQL, though further optimizations are possible.

Feed Stream Initialization

When a user's feed does not exist, create a personal feed by iterating the follow list and storing each followed user's feed IDs in a Redis sorted set (ZSet). Key points include loading initial data from the database, using the user ID as the ZSet key, and setting the score based on timestamp for Timeline or weight for Rank.

Push Updates

After initialization, the feed resides in Redis. Updates are required in four scenarios: a followed user publishes a new feed, deletes a feed, a user adds a follow, or a user removes a follow.

Publish/Delete Process

When handling large‑scale users (e.g., celebrities with millions of followers), two strategies are considered: push (actively sending the new feed to all followers) and pull (followers fetch the feed when they access the timeline). A hybrid push‑pull model is described.

Push‑Pull Combined Mode

For a new feed, the system reads the fan list, writes the feed to the author's timeline, and, if the author is not a high‑profile user, writes the feed to each follower's feed. When refreshing a feed, the system reads the list of followed high‑profile users, fetches their timelines concurrently, merges results, and sorts by time.

Push‑Only Mode

In pure push mode, publishing a feed involves a single step for all users, while reading a feed requires only one network call, dramatically reducing latency.

Summary of Two Modes

Hybrid mode can overload high‑profile users' timelines during refresh.

Solutions: use push for active fans and pull for inactive fans, or adopt full push at the cost of higher storage and longer fan‑out time.

Implementation

The author implements a pure push mode suitable for ordinary enterprises. The implementation consists of three parts: initializing the feed stream, updating followers' feeds when a user publishes or deletes a feed, and handling follow/unfollow actions.

Initialize Feed Stream

/**
 * 获取关注的人的信息流
 */
public List
listFocusFeed(Long userId, Integer page, Integer size) {
    String focusFeedKey = "focusFeedKey" + userId;
    // 如果 zset 为空，先初始化
    if (!zSetRedisTemplate.exists(focusFeedKey)) {
        initFocusIdeaSet(userId);
    }
    // 如果 zset 存在，但是存在 0 值
    Double zscore = zSetRedisTemplate.zscore(focusFeedKey, "0");
    if (zscore != null && zscore > 0) {
        return null;
    }
    //分页
    int offset = (page - 1) * size;
    long score = System.currentTimeMillis();
    // 按 score 值从大到小从 zSet 中取出 FeedId 集合
    List
list = zSetRedisTemplate.zrevrangeByScore(focusFeedKey, score, 0, offset, size);
    List
result = new ArrayList<>();
    if (QlchatUtil.isNotEmpty(list)) {
        for (String s : list) {
            // 根据 feedId 从缓存中 load 出 feed
            FeedDto feedDto = this.loadCache(Long.valueOf(s));
            if (feedDto != null) {
                result.add(feedDto);
            }
        }
    }
    return result;
}

/**
 * 初始化关注的人的信息流 zSet
 */
private void initFocusFeedSet(Long userId) {
    String focusFeedKey = "focusFeedKey" + userId;
    zSetRedisTemplate.del(focusIdeaKey);
    // 从数据库中加载当前用户关注的人发布过的 Feed
    List
list = this.feedMapper.listFocusFeed(userId);
    if (QlchatUtil.isEmpty(list)) {
        //保存0，避免空数据频繁查库
        zSetRedisTemplate.zadd(focusFeedKey, 1, "0");
        zSetRedisTemplate.expire(focusFeedKey, RedisKeyConstants.ONE_MINUTE * 5);
        return;
    }
    // 遍历 FeedList，把 FeedId 存到 zSet 中
    for (Feed feed : list) {
        zSetRedisTemplate.zadd(focusFeedKey, feed.getCreateTime().getTime(), feed.getId().toString());
    }
    zSetRedisTemplate.expire(focusFeedKey, 60 * 60 * 60);
}

Handle Followed User Publish/Delete

/**
 * 新增/删除 feed 时，处理粉丝 feed 流
 * @param userId 新增/删除 feed 的用户 id
 * @param feedId 新增/删除 的 feedId
 * @param type   feed_add = 新增 feed, feed_sub = 删除 feed
 */
public void handleFeed(Long userId, Long feedId, String type) {
    Integer currentPage = 1;
    Integer size = 1000;
    List
fansDtos;
    while (true) {
        Page page = new Page();
        page.setSize(size);
        page.setPage(currentPage);
        fansDtos = this.fansService.listFans(userId, page);
        for (FansDto fansDto : fansDtos) {
            String focusFeedKey = "focusFeedKey" + userId;
            // 如果粉丝 zSet 不存在，退出
            if (!this.zSetRedisTemplate.exists(focusFeedKey)) {
                continue;
            }
            // 新增 Feed
            if ("feed_add".equals(type)) {
                this.removeOldestZset(focusFeedKey);
                zSetRedisTemplate.zadd(focusFeedKey, System.currentTimeMillis(), feedId);
            } else if ("feed_sub".equals(type)) {
                // 删除 Feed
                zSetRedisTemplate.zrem(focusFeedKey, feedId);
            }
        }
        if (fansDtos.size() < size) {
            break;
        }
        currentPage++;
    }
}

/**
 * 删除 zSet 中最旧的数据
 */
private void removeOldestZset(String focusFeedKey) {
    // 如果当前 zSet 大于 1000，删除最旧的数据
    if (this.zSetRedisTemplate.zcard(focusFeedKey) >= 1000) {
        // 获取当前 zSet 中 score 值最小的
        List
zrevrange = this.zSetRedisTemplate.zrevrange(focusFeedKey, -1, -1, String.class);
        if (QlchatUtil.isNotEmpty(zrevrange)) {
            this.zSetRedisTemplate.zrem(focusFeedKey, zrevrange.get(0));
        }
    }
}

Handle Follow/Unfollow

/**
 * 关注/取关 时，处理 followerId 的 zSet
 * @param followId   被关注的人
 * @param followerId 当前用户
 * @param type       focus = 关注, unfocus = 取关
 */
public void handleFocus(Long followId, Long followerId, String type) {
    String focusFeedKey = "focusFeedKey" + userId;
    // 如果粉丝 zSet 不存在，退出
    if (!this.zSetRedisTemplate.exists(focusFeedKey)) {
        return;
    }
    List
FeedDtos = this.listFeedByFollowId(source, followId);
    for (FeedDto feedDto : FeedDtos) {
        // 关注
        if ("focus".equals(type)) {
            this.removeOldestZset(focusFeedKey);
            this.zSetRedisTemplate.zadd(focusFeedKey, feedDto.getCreateTime().getTime(), feedDto.getId());
        } else if ("unfocus".equals(type)) {
            // 取关
            this.zSetRedisTemplate.zrem(focusFeedKey, feedDto.getId());
        }
    }
}

The code snippets illustrate the core logic; a production system would require additional classes and error handling.

Conclusion

The article presents a simple, enterprise‑ready feed stream solution and invites feedback for further improvement.