Can Your System Survive a Sudden 500K Live Viewers? 4 Proven Traffic‑Splitting Techniques

When a live‑streaming room suddenly receives half a million concurrent viewers, the backend must handle massive comment bursts, 100,000‑plus QPS likes, and reward orders that update inventory in real time. The article presents a four‑step traffic‑splitting architecture that has been validated in production.

1. Room‑Based Sharding – “Put the Elephant in the Fridge”

The idea is to create logical rooms first and then admit users. A request hits a scheduling layer that, based on load‑test results (e.g., 5,000 users per server), assigns the user a token pointing to a specific regional server cluster. Subsequent connections use the token to stay within that cluster. This keeps per‑machine load predictable, makes capacity planning easy, and allows scaling simply by adding more machines—similar to a restaurant reserving seats to avoid queues.

2. Queue Buffering – “Let the Danmaku Fly for a Moment”

Because nationwide live‑comment volume is unpredictable, the system first captures messages in a distributed queue. It performs real‑time aggregation and compression to filter duplicate spam. Processed results are placed into a content‑distribution server group; clients receive a “update available” notification via a long‑living connection and then pull batched data for replay. During low traffic the system pushes updates instantly; under high load it switches to batch‑push mode, sacrificing a small amount of timeliness to prevent crashes.

3. Multi‑Layer Write Cache – The Secret Behind 100K‑QPS Likes

Likes are a classic “write‑heavy, read‑light, temporarily inconsistent” workload. Writing directly to a database at >100K QPS would fail. The solution uses two cache layers. Incoming like requests are randomly routed to different cache nodes; the first layer aggregates likes locally (e.g., merges likes occurring within a 3‑second window). Periodically the first layer flushes aggregated counts to a second‑level cache, which then merges into a central cache that synchronises with read‑only query nodes. Clients can also coalesce rapid repeated likes (e.g., 10 likes within 3 seconds become a single report). The article notes that Weibo later replaced Redis counters with a custom liking service, saving substantial memory.

4. Consistent Hashing – No‑Loss Reward and Inventory Deduction

Reward transactions involve real money, requiring strong consistency. The design hashes each user ID to one of 256 buckets (hash(uid) % 256). The gateway routes the request to the corresponding shard, where the balance and inventory are updated atomically. Scaling uses a “tree‑hot‑migration slice” method: the 256 buckets are evenly distributed across 16 machines (16 buckets per machine). If a machine becomes overloaded, two replica buckets are first added, then the original 16 buckets are split into two groups of eight and reassigned to a new node; the old node is taken offline. Throughout this process a proxy forwards traffic, making the migration invisible to clients.

A single diagram summarises the logic: predictable load → room‑based throttling; delay‑tolerant load → queue buffering; merge‑able load → multi‑layer cache aggregation; must‑be‑consistent load → hash‑sharded strong writes.

The underlying infrastructure consists of three core components: a message queue, real‑time computation, and distributed storage. Operations rely on Kubernetes for dynamic scaling, while traffic routing uses HttpDNS for intelligent dispatch.

In essence, high concurrency is not about “hard‑pushing” resources but about “divide and conquer”: split a traffic surge into four manageable streams, each handled by a dedicated pipeline.