Traffic Shaping Strategies for High‑Concurrency Events: Staggering, Rate Limiting, and Peak Cutting

During last year’s Double‑11 event, Youku’s live streaming caused a massive surge in user traffic, exposing a huge gap between server resources and user demand. This article, written by a senior wireless development engineer at Youku, explains how to handle such extreme traffic and concurrency through three core strategies: staggering (错峰), rate limiting (限流), and peak cutting (削峰), all aimed at reducing the maximum QPS (queries per second).

QPS is the standard metric for server request handling capacity. When client traffic exceeds the server’s peak QPS, the server may crash or return rate‑limit errors, both of which degrade user experience and damage the brand.

1. Staggering Strategy (错峰)

The goal of staggering is to spread request peaks over time, turning a single large QPS spike into multiple smaller spikes, thereby lowering the overall peak.

1.1 Server‑side Staggering

If the client request is triggered by a server‑initiated push (e.g., PowerMsg), the server can batch the pushes and send them sequentially, assuming uniform round‑trip time (RT) to all clients.

1.2 Client‑side Staggering

Clients can apply a random delay within a predefined interval T . Each client picks a random time t ∈ [0, T] before sending the request, ensuring a uniform distribution of requests. The theoretical new maximum QPS becomes the original peak divided by T .

2. Rate Limiting Strategy (限流)

Servers typically enforce rate limits (e.g., Mtop returns a specific error code). The article focuses on client‑side rate limiting, whose objectives are to eliminate unnecessary or overly frequent requests.

Stop sending requests once the user reaches the maximum allowed request count.

If the user is currently rate‑limited, delay the next request.

Cease further requests after receiving the desired response.

3. Peak Cutting Strategies (削峰)

3.1 Minimum Effective Interval Between Requests

Define a minimum interval t . Any request occurring within t seconds of the previous one is considered invalid and ignored. This is effective for high‑frequency activities such as “red‑packet rain”.

3.2 Probabilistic Request Strategy

Introduce a probability P for each request. Only with probability P does the client actually send the request. Combined with the minimum interval t , the maximum QPS can be expressed as:

Adjusting t and P provides fine‑grained control over the QPS ceiling.

3.3 Fairness Strategy

To avoid extreme request bursts caused by pure probability, a fairness strategy combines a random algorithm with an interpolation algorithm to generate a controlled request sequence. Each user’s effective request probability P (e.g., 0.2) determines how many opportunities they have within a activity cycle.

By tuning the granularity of this algorithm, the system can ensure fairness while still reducing peak QPS.

Conclusion and Best Practices

Combining staggering, rate limiting, and peak cutting effectively lowers QPS during massive promotional events. Practical recommendations include:

Adjust parameters (e.g., T , t , P ) based on real‑world data.

Account for network latency, especially for server‑push scenarios.

Use pre‑loading and CDN backup to offload traffic.

Conduct thorough rehearsals to validate all edge cases.

Obtain realistic stress‑test data to verify server capacity.

Always have a backup plan for critical events.