Mastering Rate Limiting: 4 Proven Strategies to Protect Your Services

Introduction

Last summer a financial platform reported a payment‑interface error rate soaring to 35%. The root cause was a lack of rate‑limiting protection, which exhausted the database connection pool and caused massive request backlogs.

Rate limiting is not about denying service; it sacrifices controllable traffic to safeguard core pathways. For example, an e‑commerce flash‑sale limited its spike interface to 50 k QPS with a token‑bucket algorithm, losing 20% of burst traffic but preserving 99% of core transaction success.

1 Common Rate‑Limiting Schemes

1.1 Fixed Window Counter

Core principle: Count requests within a fixed time window (e.g., 1 second) and reject any that exceed the threshold.

<code>public class FixedWindowCounter {
    private final AtomicLong counter = new AtomicLong(0);
    private volatile long windowStart = System.currentTimeMillis();
    private final int maxRequests;
    private final long windowMillis;
    public boolean tryAcquire() {
        long now = System.currentTimeMillis();
        if (now - windowStart > windowMillis) {
            if (counter.compareAndSet(counter.get(), 0)) {
                windowStart = now;
            }
        }
        return counter.incrementAndGet() <= maxRequests;
    }
}</code>

Fatal flaw: If a burst of 100 requests occurs at 0.9 s, the next second may allow another 100, resulting in 200 requests over two seconds.

Applicable scenarios: Log collection, non‑critical APIs that can tolerate coarse‑grained throttling.

1.2 Sliding Window

Core principle: Divide the time window into smaller slices (e.g., 10 s) and count requests in the most recent N slices.

Redis Lua implementation:

<code>String lua = "\n    local now = tonumber(ARGV[1])\n    local window = tonumber(ARGV[2])\n    local key = KEYS[1]\n    redis.call('ZREMRANGEBYSCORE', key, '-inf', now - window)\n    local count = redis.call('ZCARD', key)\n    if count < tonumber(ARGV[3]) then\n        redis.call('ZADD', key, now, now)\n        redis.call('EXPIRE', key, window/1000)\n        return 1\n    end\n    return 0\n";"</code>

Technical highlight: A securities trading system reduced API error rate from 5% to 0.3% after adopting sliding windows, achieving ±10 ms precision using Redis ZSET.

1.3 Leaky Bucket

Core principle: Requests enter a bucket like water; the system processes them at a fixed rate, discarding excess when the bucket is full.

<code>public class LeakyBucket {
    private final Semaphore permits;
    private final ScheduledExecutorService scheduler;
    public LeakyBucket(int rate) {
        this.permits = new Semaphore(rate);
        this.scheduler = Executors.newScheduledThreadPool(1);
        scheduler.scheduleAtFixedRate(() -> permits.release(rate), 1, 1, TimeUnit.SECONDS);
    }
    public boolean tryAcquire() {
        return permits.tryAcquire();
    }
}</code>

Technical pain point: An IoT platform kept 100 k devices reporting data stable at 500 req/s, but burst traffic could still cause queue buildup.

Applicable scenarios: IoT command dispatch, payment‑channel rate limits that require a steady processing rate.

1.4 Token Bucket

Core principle: Tokens are generated at a fixed rate; a request must acquire a token before execution. Bursts consume stored tokens.

Implementation using Guava RateLimiter:

<code>// Guava RateLimiter advanced usage
RateLimiter limiter = RateLimiter.create(10.0, 1, TimeUnit.SECONDS);
limiter.acquire(5); // warm‑up, acquire 5 tokens
// Dynamically adjust rate via reflection (example)
Field field = RateLimiter.class.getDeclaredField("tokens");
field.setAccessible(true);
AtomicDouble tokens = (AtomicDouble) field.get(limiter);
tokens.set(20); // inject 20 tokens during burst</code>

Real‑world case: A video platform limited normal traffic to 100 k QPS, but allowed a 50% over‑limit for three seconds during hot events, preventing avalanche while preserving user experience.

Dynamic features:

Normal QPS limit

Burst allowance via token reserve

Token depletion on sustained spikes

2 Production‑Level Practices

2.1 Distributed Rate Limiting at the Gateway

An e‑commerce Double‑11 solution combined Redis + Lua counters with Nginx local cache, blocking 83% of malicious requests at the gateway layer.

2.2 Adaptive Circuit‑Breaker Mechanism

A social platform automatically lowered its rate‑limit threshold from 50 k to 30 k during traffic spikes, then gradually restored it after recovery.

3 Pitfalls and Performance Optimizations

3.1 Fatal Mistake

Applying rate limiting before the database connection pool can cause connection leaks and overload the database.

Correct approach follows the three principles of circuit breaking:

Fast failure: reject invalid requests at the entry point.

Dynamic downgrade: keep core services with minimal resources.

Automatic recovery: gradually increase traffic after a break.

3.2 Performance Tuning

A financial system’s JMH benchmark showed that replacing

AtomicLong

with

LongAdder

increased rate‑limiting throughput by 220%.

Optimization techniques include reducing CAS contention and using segmented locks.

Conclusion

The article listed the four most commonly used rate‑limiting schemes and emphasized selecting the appropriate algorithm based on business scenarios. The “golden rule” is that a good rate‑limiting solution should ensure high throughput while protecting system stability, much like a high‑speed rail gate that balances efficiency and safety.