Mastering Rate Limiting: 5 Algorithms to Prevent Service Avalanches

Rate limiting is a common concern in high‑concurrency, high‑traffic scenarios. When a service (e.g., Service 8) receives excessive requests, its response may slow down or crash, causing a cascade of failures across dependent services.

After a certain point (T2), the entire application becomes unusable. To prevent such avalanche effects, adding rate‑limiting protection at both the gateway and service layers is essential.

1. Counter Method

The core idea is to count the number of requests within a fixed time window; if the count exceeds a threshold, rate limiting is triggered.

This simple algorithm suffers from a critical flaw: it can miss bursts that span two adjacent windows.

2. Sliding Window

To overcome the fixed‑window limitation, a sliding time window is used, continuously evaluating the request count over the most recent interval.

3. Leaky Bucket Algorithm

A fixed‑size bucket stores incoming requests; if the bucket is full, excess requests are dropped. Requests are processed at a constant rate, smoothing bursts.

4. Token Bucket Algorithm

A bucket receives tokens at a fixed rate; a request can proceed only if it can acquire a token. This approach handles bursty traffic well while enforcing an average rate.

5. Redis‑Based Implementations

5.1 Sliding Window with ZSET

Use a Redis sorted set where the score is the timestamp. Query the range [now‑window, now] to count recent requests and decide whether to limit.

Core code:

# Create ZSET queue
ZSet<String> zset = redisTemplate.opsForZSet().create("userLimiter");

# Add user request
long currentTimeMillis = System.currentTimeMillis();
String userId = "12345789";
redisTemplate.opsForZSet().add("userLimiter", userId, currentTimeMillis);

# Check threshold
int limitCount = 100;
long size = redisTemplate.opsForZSet().size("userLimiter");
if (size >= limitCount) {
    // Exceeds limit, reject request
} else {
    // Allow request, execute business logic
}

5.2 Leaky Bucket with List

Maintain a fixed‑size List; record requests while the list is not full. Once the capacity is reached, further requests are rejected.

5.3 Token Bucket with Hash

A scheduled task periodically adds a fixed number of tokens to a Redis hash. Requests consume a token from the hash; if none are available, the request is degraded.

Summary

Common rate‑limiting algorithms include Counter, Sliding Window, Leaky Bucket, and Token Bucket, each suited to different scenarios. Redis data structures (ZSET, List, Hash) enable practical implementations of these algorithms, allowing developers to protect services from traffic spikes and prevent cascading failures.