How Baidu Scales Live‑Stream Messaging to Millions of Users

Background

Live streaming requires two core functions: real‑time audio/video streaming and a bi‑directional message flow inside the live‑room. Unlike simple group chat, live‑room messages include gifts, entrance notifications, likes, purchases, host recommendations, and stream switches, all of which depend on a reliable, low‑latency message channel.

1. Live‑Room Messaging 1.0 – Foundations

Key observations:

Participant scale : Group chats handle thousands, while popular live rooms can host millions of users simultaneously.

User‑room relationship : Users join and leave a live‑room extremely frequently, generating tens of thousands of join/leave events per second.

Session duration : Live‑rooms typically last only a few hours, unlike group chats that may persist for days.

From these differences, two core problems emerge:

Problem 1 – User management : Supporting up to 20 k concurrent joins/leaves per second and maintaining millions of online users.

Problem 2 – Message delivery : Delivering a high volume of messages to millions of online users with strict real‑time requirements.

2. Design Goals for Live‑Room Messaging

Achieve sub‑second end‑to‑end latency.

Support over one million concurrent online users per room.

Allow controlled message dropping for excess peak traffic.

Cap per‑room outbound messages to a small N (e.g., N ≤ 20) within a short interval S (S ≤ 2 seconds).

3. Pressure Analysis of Ordinary Group Chat

Re‑using a classic group‑chat pipeline for live‑room messaging creates six million‑scale challenges:

Splitting a user list of millions per second.

Fetching device information for millions of users.

Resolving dynamic routing for millions of connections.

Sending notifications to millions of long‑connection instances.

Handling millions of fetch‑message requests from clients.

Updating read‑status for millions of messages.

Optimizations such as merging user‑list and device queries, eliminating pull‑based fetch in favor of push‑only delivery, and simplifying read‑status handling reduce three of these pressures but still leave user‑list splitting, routing, and push as bottlenecks.

4. Multicast (mcast) Architecture

To break the bottleneck, Baidu introduced a long‑connection multicast mechanism:

Each multicast group has a globally unique mcastID.

Clients join/leave groups via SDK calls ( mcastJoin, mcastLeave).

The multicast router maintains a list of long‑connection service instances ( mcastRoute) that host the group’s connections.

When a message arrives, the backend hashes the mcastID to a router instance, which then forwards the message to all relevant long‑connection instances, which finally push it to each client SDK.

Key operations:

Client SDK sends mcastJoin over an existing long connection.

Business layer validates the request and creates a multicast route.

The route records which connection instances belong to the group.

On leave, the SDK issues mcastLeave, and the route is updated accordingly.

5. Performance Evaluation

Pressure points shift mainly to mcastJoin/Leave (peak ≈ 20 k QPS) and to the long‑connection layer for downstream delivery. Benchmarks show a single long‑connection instance can sustain 5‑8 × 10⁴ QPS with 25 k concurrent connections; therefore, 20 instances comfortably handle 1 × 10⁶ QPS, and 125 instances can support 1 × 10⁷ QPS.

Capacity Estimation (instances needed)

Online connections | 1 M | 5 M | 10 M
--- | --- | --- | ---
1 × 10⁴ QPS per instance | 100 | 500 | 1000
5 × 10⁴ QPS per instance | 20 | 100 | 200
8 × 10⁴ QPS per instance | 12.5 | 62.5 | 125

Bandwidth analysis reveals that raw downstream traffic can exceed 1.5 Tbps for 5 M connections. Applying a 6.7:1 compression ratio reduces the peak to ~230 Gbps, which fits within modern data‑center uplinks.

6. Client‑Side Optimizations

Rate limiting : Prioritize high‑importance messages (e.g., gifts) and cap per‑user outbound rate to avoid UI overload.

Priority‑based delivery : High‑priority messages bypass aggregation, while normal messages are batch‑aggregated with a modest 500 ms latency increase.

Connection stability : Deploy access points across major ISPs and a Hong Kong node for overseas users; use HTTPDNS to mitigate DNS hijacking; shorten heartbeat intervals for faster failure detection.

Reconnection handling : On disconnection, the server removes the client from the multicast group; the client re‑joins after rebuilding the connection.

7. Extensions – Historical and Gift Messages

Historical messages are stored in a dedicated multicast mailbox ( mcast‑mbox ) that supports range queries by time or count. Gift messages, which require high reliability and low latency, are sent through a separate high‑priority multicast channel, with fallback pull via short‑connection if the long‑connection fails.

8. Ongoing Challenges and Future Work

Remaining topics include multi‑platform support (Android, iOS, H5, mini‑programs), cross‑app message synchronization, handling anonymous users, graceful degradation paths, moderation pipelines, cross‑room broadcasting, and new interactive features such as live quizzes and e‑commerce integration.

Conclusion

The multicast mcast mechanism shifts most pressure to the long‑connection layer, enables horizontal scaling, and, combined with aggregation, compression, and rate‑limiting, meets the real‑time, high‑throughput demands of modern live‑streaming platforms.