High Concurrency Traffic Control and Rate Limiting Techniques

In real projects the author has experienced peak traffic of over 50,000 QPS and stress‑tested up to 100,000 QPS, prompting a discussion on handling high‑concurrency traffic.

High traffic is defined not by a fixed number but by any load that stresses the system and degrades performance.

Common mitigation techniques include caching (bringing data closer to the application to reduce DB hits), degradation (downgrading non‑core services during spikes), and rate limiting (controlling request volume within a time window to protect system stability).

Rate‑limiting methods covered are:

Counter : a simple algorithm that counts requests in a fixed interval and resets the counter at the interval boundary.

Sliding Window : divides time into small slices that move forward, avoiding the counter’s boundary‑spike problem.

Leaky Bucket : models a bucket with constant outflow; excess requests overflow when the bucket is full.

Token Bucket : improves on the leaky bucket by generating tokens at a steady rate, allowing bursts while still limiting overall throughput.

Illustrative images for each algorithm are included:

Guava’s RateLimiter implements the token‑bucket algorithm; by specifying a desired QPS, it automatically generates tokens and applications acquire permits via tryAcquire().

While the article focuses on single‑machine rate limiting, it notes that distributed scenarios often combine technologies such as Nginx+Lua or Redis+Lua, but these are not detailed here.

In summary, effective traffic control combines caching, degradation, and appropriate rate‑limiting algorithms to protect system stability under massive request loads.