Evolution of Bilibili Dynamic Feed Architecture

Author : Jin Binwu, Senior Development Engineer at Bilibili.

Business Overview

Bilibili Dynamic is a social feature on the Bilibili platform that allows users to share text, images, videos, and other content, as well as their watch history, comments, and collections. Creators (UP hosts) also use dynamics to interact with fans, posting video previews, behind‑the‑scenes clips, live‑room links, and answering questions.

Dynamic video enables users to upload a short (≤15 s) video to their feed, providing a more vivid way to express status or ideas.

The overall goal is to enrich user expression and interaction across the platform.

Business Characteristics

Large traffic pool : Daily page views exceed one billion.

Complex logic : Nearly 30 types of dynamic cards, requiring deep understanding of business logic.

Broad coverage : Over 20 non‑dynamic business integrations, with frequent inter‑service coupling.

Strong traffic diversion : Dynamics receive close to 100 million clicks daily, driving article, image, and live‑stream traffic.

Long iteration history : Launched in September 2017, now over five years old.

These characteristics raise questions about architecture evolution, which the following sections address.

Architecture Overview

Figure 2.1 shows the content workflow from production to indexing, covering content ingestion, preprocessing, persistence, and indexing.

Figure 2.2 depicts the four‑layer micro‑service architecture. The Dynamic Gateway (BFF layer) abstracts rendering, layout, and format control, separating presentation logic from the original services.

Motivation : The legacy C/S architecture could not meet the diverse needs of mobile, web/H5, PC, and OTT clients. The dynamic service’s complex data requirements led to high development costs.

Figure 2.3 illustrates the internal module layering of the dynamic service, including a data abstraction layer that groups related business domains.

Storage Design – Cache List

The user‑list cache stores only dynamic IDs (dynID) with sentinel values:

dyn_max: upper bound sentinel to prevent cache penetration.

dyn_zero: lower bound sentinel indicating that all database records are cached.

Feed streams are highly time‑sensitive; most users consume only the latest page (≈400 items). The cache therefore holds the newest entries, achieving hot‑cold data separation and cost reduction.

Persistence Layer – Database

Accounts and user behavior are stored centrally.

Index and content are separated.

Dynamics store only content data; relational DB with sharding.

High‑write scenarios use KV stores.

Reasons for this design include service isolation, uneven QPS across dimensions, and the need for hot‑cold separation.

Architecture Evolution

Feed Flow Architecture

Key feed characteristics:

Massive data volume (potentially 100 TB–PB) due to write‑push amplification.

Simple data format.

High reliability requirements.

Monotonically increasing IDs per user to enable range reads.

Cost‑sensitive storage.

Push vs. Pull Comparison

Push (real‑time) suits write‑heavy scenarios; pull (on‑demand) suits read‑heavy scenarios. Each has pros and cons regarding latency, bandwidth, and server load.

Evolution Stages

Stage 1 – Incremental Pull with Inbox Snapshots : Pulling N followees, storing a snapshot in the inbox to avoid repeated outbox reads. Problems: inbox write bottleneck, gaps when users are offline.

Stage 2 – Pure Pull : Users fetch pages based on a global max ID. Read amplification grows with follower count; mitigated by red‑dot cache isolation.

Stage 3 – Push‑Pull Combination : After obtaining a full fan list, introduce push for small UPs while large UPs remain pull‑based. This reduces outbox pressure and mitigates long‑tail issues.

Design details include:

List merging of large‑UP (pull) and small‑UP (push) results.

Inbox storing only visible user‑generated content.

Push module that writes content IDs to fan inboxes upon publication.

Relationship‑chain compensation to handle follow/unfollow events.

Figure : Feed publishing flow – after a user posts, the push job reads the binlog, determines UP size, fetches fan lists, and writes the dynamic ID to each fan’s inbox.

Additional optimizations:

Outbox local cache to handle hotspots and Redis scaling limits.

Automatic downgrade from push‑pull to pure pull when push failures occur.

Future Development

From an operations perspective, growing traffic, higher relationship limits, and inactive accounts will increase push‑mode storage costs. Interest‑based recommendation may become a new direction.

From a user perspective, challenges include:

Large UPs often push low‑quality promotional content, polluting the feed.

Interest‑based consumption may suffer when a user follows an UP for a specific topic but receives unrelated content.

Content Platformization

Current efforts aim to unify image‑text content handling (display, creation, storage) into a platform. Future work includes completing this platform, deprecating legacy services, and extending the platform to more scenarios.

--- End of summary ---