Why WebSocket Broadcast Fails at 1M Viewers: Solving Massive Fan-Out in Live Chat

Bandwidth Math and the Interview Trap

A single danmaku message, including protocol overhead, is roughly 100 bytes. Pushing it to 1 million viewers generates 100 MB of egress traffic. A 1 Gbps access node offers a theoretical 125 MB/s output, meaning one full broadcast consumes 5 MB; at 25 broadcasts per second the link saturates. Yet a popular live room can produce tens of thousands of messages per second.

The author recounts a candidate who answered "WebSocket long connections, broadcast to all connections in the room." The interviewer immediately asked: "Have you calculated the egress bandwidth for pushing one message 1 million times?" The candidate suggested adding machines. The interviewer followed up: "A hot room sees 100k messages per second — push them all? Can the user's phone even handle that?" The candidate was stuck.

Why "WebSocket Broadcast" Is a Naive Answer

WebSocket itself is fine; the mistake is treating 1 million viewers as 1 million individual recipients. This is classic fan-out amplification : write once, read 1 million times. Write cost is constant, read cost grows linearly with online users, and there is no buffer between them.

Three Fatal Flaws

Flaw 1: Linear bandwidth amplification. Audience grows 10x, egress bandwidth grows 10x. Adding machines solves cost, not architecture.

Flaw 2: The client cannot consume it. A phone rendering 200 messages/sec freezes the UI thread; human reading speed is ~10-20/sec. 99% is pure waste — you spend 100 MB bandwidth pushing messages users never see.

Flaw 3: Access nodes are stateful. Connections are pinned to specific machines. When a node fails, its 50k users reconnect simultaneously, hammering the remaining nodes — that's how avalanches start.

The core insight for a high-score answer: Don't treat 1 million as 1 million recipients; treat them as N access nodes doing local fan-out, and deliberately drop what can be dropped.

Core Architecture: Three Solutions (From Bronze to King)

Solution 1: Full Push via MQ (The Interviewer's Trap)

Many candidates reflexively say "Add Kafka for decoupling and peak shaving!" Stop there at your peril. MQ solves decoupling and absorbs write spikes, but does not solve fan-out at all : one message enters the queue and still must be consumed 1 million times.

Worse, the consumption model breaks: Kafka partitions by roomId create a super-hot partition for a popular room, forcing a single consumer to bear the load; partitioning by uid requires replicating one message into 1 million partitions, exploding the write side. Conclusion: MQ is a transport pipe, not a fan-out engine. It can be part of the architecture but must be paired with the next layer.

Solution 2: Room Routing + Access Layer Local Fan-Out (Standard Senior Solution)

This is the industry mainstream. The core idea: turn "1 million remote pushes" into "dozens of remote pushes + dozens of local in-memory traversals."

Architecture splits into three layers:

Access Layer (Comet): Maintains long connections, locally indexes roomId → connection list on this node. Only handles I/O, no business logic.

Routing Layer (Router): Maintains roomId → list of access nodes. Dynamically registers and deregisters nodes as they come online/offline.

Logic Layer (Logic/Job): Consumes danmaku, queries router, RPC pushes to relevant nodes (typically 30-50). Each node locally iterates its own connections for that room (e.g., 20k out of 50k) and delivers.

One message's full path: client sends → logic layer audits + assigns seq → router returns 30 nodes → parallel RPC to those 30 nodes → each node locally traverses its room connections → batch deliver. Remote pushes drop from 1 million to ~30 ; the rest are local memory operations.

Bilibili's open-source goim follows exactly this layering: comet for connections, job consumes Kafka and routes by room, router maintains room-to-node mapping. Mentioning it shows you've studied production architectures.

Three Mandatory Optimizations

Protocol slimming. Replace JSON with Protobuf; 100 bytes shrinks to ~30 bytes. Be cautious with WebSocket's permessage-deflate — it trades CPU for bandwidth and often loses on small packets like danmaku.

Write merging. In Netty, separate write and flush: call write for all channels, then a single flush, merging dozens of syscalls into one.

Batch packing. See Solution 3.

Solution 3: Merge Window + Tiered Degradation (Architect-Level Solution)

Routing solves "push to whom," but not "push how many." A hot room generating tens of thousands per second still overwhelms clients even if only 30 nodes receive them.

A. Merge Window (Batching)

Viewers don't need every message instantly; they need a smooth danmaku stream . The access layer opens a 100 ms merge window: messages arriving within the window are batched into one packet. 10k messages/sec → 10 packets/sec, packet count drops three orders of magnitude, protocol overhead nearly vanishes. 100 ms latency is imperceptible in danmaku scenarios.

B. Tiered Degradation (Sampling)

If batching still yields too many, actively drop by priority:

Gifts, paid messages: Full delivery + ACK (money involved, loss = incident).

VIP / noble danmaku: High priority, higher rate limits (paid experience).

Normal danmaku: Sampled delivery, drop when threshold exceeded (inherently excessive).

Entrance prompts: Aggregate into "UserX and 100 others entered" (pure noise, aggregation suffices).

This step is the litmus test of architectural thinking : admitting "most messages don't need to arrive" marks someone who has run real live systems.

C. Edge Sink

Deploy access nodes regionally; complete fan-out at the edge, avoiding backhaul to central data centers. This layer often reuses CDN vendors' live edge nodes.

Advanced Pitfall: How to Preserve Ordering Merging and sampling introduce reordering. The fix: Maintain a per-room monotonically increasing seq (not timestamps — timestamps collide under concurrency). Client sorts locally by seq ; on detecting gaps, proactively fetches missing pieces. seq generation must not hit Redis per message. A hot room doing tens of thousands of INCR per second creates a Redis hot key. Use named segment pre-allocation (like Leaf-segment): fetch 1000 numbers at once, increment locally via AtomicLong , refill when exhausted — Redis QPS drops by three orders of magnitude. If asked "Isn't single-room serial seq a bottleneck?" Answer: split by message type — danmaku, gifts, entrance each have independent seq chains, ordering only within each chain.

Code-Level Implementation

1) Access Layer Local Fan-Out: Write/Flush Separation

// Iterate local connections for the room, batch write then unified flush
public void broadcast(long roomId, ByteBuf payload) {
    List<Channel> channels = roomLocalIndex.get(roomId);
    if (channels == null || channels.isEmpty()) return;

    int n = 0;
    for (Channel ch : channels) {
        if (!ch.isActive()) continue;
        ch.write(payload.retainedDuplicate()); // reuse same buffer
        n++;
        if (n % 1024 == 0) ch.flush(); // segmented flush to prevent starvation
    }
    for (Channel ch : channels) {
        if (ch.isActive()) ch.flush();
    }
}

Key point: retainedDuplicate() lets 20k connections share the same ByteBuf, so serialization happens only once — the fundamental reason local fan-out holds up.

2) Merge Window: 100 ms Batch Accumulation

// Room-scoped buffer, timed flush to avoid lock contention
private final Map<Long, List<Danmaku>> buffer = new ConcurrentHashMap<>();

public void offer(long roomId, Danmaku d) {
    buffer.computeIfAbsent(roomId, k -> new ArrayList<>()).add(d);
}

@Scheduled(fixedDelay = 100)
public void flushAll() {
    for (Map.Entry<Long, List<Danmaku>> e : buffer.entrySet()) {
        List<Danmaku> list = e.getValue();
        if (list.isEmpty()) continue;
        broadcast(e.getKey(), BatchDanmaku.encode(list)); // single encode, single deliver
        list.clear();
    }
}

3) Segment Seq: Don't Use Redis as a Counter

// Fetch 1000 numbers at a time, local increment, Redis QPS reduced to 1/1000
public class SegmentSeqGenerator {
    private final AtomicLong cursor = new AtomicLong(0);
    private volatile long max = 0;
    private final RoomSeqRepository repo;

    public synchronized long next(long roomId) {
        if (cursor.get() >= max) {
            Segment seg = repo.fetchNextSegment(roomId, 1000); // INCRBY 1000
            cursor.set(seg.start());
            max = seg.end();
        }
        return cursor.incrementAndGet();
    }
}

Anti-Objection Guide (Clearing Blind Spots)

Q1: Access node crashes, 50k users reconnect simultaneously — how to prevent avalanche?

Four defense lines: client-side exponential backoff + random jitter (avoid thundering herd at same second); access layer token bucket rate limiting per data center; nodes stateless — connections hold no business data, reconnect via service discovery to another node; on reconnect, incremental replay using locally stored max seq to fetch only missing messages, not full replay.

Q2: What if messages are lost?

Tiered handling. Normal danmaku already sampled — loss is acceptable. Gifts, paid messages, system notifications must not lose : write to offline storage + client ACK + replay on reconnect. Defining what can be lost is more valuable than guaranteeing zero loss.

Q3: Super-hot room makes the room service itself a hotspot?

Shard room logic by roomId hash across multiple logic nodes; seq segments are local (as above); sensitive-word review cannot be synchronous — fast-pass via allowlist, async review with recall command if violated. In danmaku scenarios, "send first, review later, recall if needed" is standard practice.

Q4: What if all middleware collapses?

Fallback philosophy: protect the core path . Danmaku is an enhancement; the live stream is the core. When danmaku service trips, access layer returns empty lists — never let danmaku drag down the live stream. True high availability isn't "never fail"; it's core functionality stays alive when failures happen .

Interview Standard Answer Template (Memorize This)

This is a classic fan-out amplification problem: write once, read millions. Naive WebSocket full broadcast fails on both bandwidth and client rendering. My approach: turn remote fan-out into local fan-out and actively sacrifice partial delivery . Architecture layering: Access layer holds connections and local room-to-connection index; routing layer maps room to access nodes; logic layer consumes danmaku, queries router, RPC pushes only to the dozens of nodes carrying that room. Remote pushes drop from millions to dozens. Volume reduction: Protobuf for protocol slimming; access layer uses 100 ms merge window to batch packets, cutting packet count by three orders of magnitude; tiered by message type — gifts guaranteed, normal danmaku sampled and dropped. Reliability: Per-room monotonic seq for ordering; seq via segment pre-allocation to avoid Redis hotspot; node failure triggers client exponential backoff reconnect, then incremental replay by seq. Fallback: Danmaku is non-core; circuit-break to empty on failure, ensuring live stream stays up.

Closing Thoughts

Back to that number: 25 messages per second . A 1 Gbps node under naive broadcast caps at 25 msg/sec. This reminds us: in live streaming, "full delivery" was never the requirement; "user-perceived smoothness" is .

If merging solves it, don't push per message; if local fan-out works, don't broadcast remotely; if active dropping works, don't pretend reliability. The combo of routing layer + local fan-out + merge degradation + tiered ordering solves not just danmaku but group chat, sports live, online meeting notifications, stock quote push, game AOI broadcast — all the same fan-out problem.

Honestly, I've never seen a solution that's simultaneously fast, complete, and cheap. Merging adds latency, sampling loses messages, edge deployment costs money. Architecture is picking the least painful trade-off.

So next time an interviewer asks "How to push danmaku?", don't blurt "WebSocket." Counter with:

Does this scenario require every message delivered, or just that the user perceives a smooth stream?

Ask that, and you've already won half the battle.