Distributed Rate Limiting in Java: Fixed, Sliding, Leaky & Token Bucket

Distributed Rate Limiting Overview

In high‑traffic scenarios such as year‑end promotions, a surge of requests can overload a distributed system. To protect the service, two common defensive strategies are rate limiting (applied before traffic enters) and circuit breaking (applied after a failure is detected).

Why Rate Limiting?

When traffic exceeds the system’s capacity, it can cause crashes. By measuring the peak QPS before a promotion and configuring a limit, excess requests are either rejected or delayed, preventing the system from going down.

General Rate‑Limiting Process

Collect request statistics : count requests using counters, sliding windows, etc.

Check against the limit : compare current traffic with the configured threshold.

Apply limiting strategy : reject, delay, or return an error for excess requests.

Update statistics : adjust counters or window data.

Repeat : continuously monitor and enforce the limit.

Implementation details (e.g., token‑bucket, leaky‑bucket) may vary; common algorithms include token bucket, leaky bucket, fixed window, sliding window, etc.

Single‑Machine vs Distributed Limiting

For a single instance, statistics can be stored locally. In a clustered environment, a shared store such as Redis or Tair is required so that all nodes see the same counters.

Four Rate‑Limiting Algorithms with Redisson

All examples use Redis as the distributed store and Redisson as the client.

Fixed Window Algorithm

Principle

The time axis is divided into fixed windows; each window has a request quota. If the count exceeds the quota, further requests are blocked until the window resets.

Implementation

public class FixedWindowRateLimiter {
    public static final String KEY = "fixedWindowRateLimiter:";
    private Long limit;
    private Long windowSize;
    public FixedWindowRateLimiter(Long limit, Long windowSize) {
        this.limit = limit;
        this.windowSize = windowSize;
    }
    public boolean triggerLimit(String path) {
        RedissonClient redissonClient = RedissonConfig.getInstance();
        RLock rLock = redissonClient.getLock(KEY + "LOCK:" + path);
        try {
            rLock.lock(100, TimeUnit.MILLISECONDS);
            String redisKey = KEY + path;
            RAtomicLong counter = redissonClient.getAtomicLong(redisKey);
            long count = counter.incrementAndGet();
            if (count == 1) {
                counter.expire(windowSize, TimeUnit.SECONDS);
            }
            if (count > limit) {
                counter.decrementAndGet();
                return true; // limit triggered
            }
            return false;
        } finally {
            rLock.unlock();
        }
    }
}

The algorithm is simple and low‑memory but suffers from a “boundary problem” when a burst arrives exactly at window transition.

Sliding Window Algorithm

Principle

The large time window is split into multiple small sub‑windows, each with its own counter. Requests are counted across all sub‑windows, providing a smoother limit.

Implementation

public class SlidingWindowRateLimiter {
    public static final String KEY = "slidingWindowRateLimiter:";
    private Long limit;
    private Long windowSize;
    public SlidingWindowRateLimiter(Long limit, Long windowSize) {
        this.limit = limit;
        this.windowSize = windowSize;
    }
    public boolean triggerLimit(String path) {
        RedissonClient redissonClient = RedissonConfig.getInstance();
        RScoredSortedSet<Long> counter = redissonClient.getScoredSortedSet(KEY + path);
        RLock rLock = redissonClient.getLock(KEY + "LOCK:" + path);
        try {
            rLock.lock(200, TimeUnit.MILLISECONDS);
            long now = System.currentTimeMillis();
            long windowStart = now - windowSize * 1000;
            counter.removeRangeByScore(0, true, windowStart, false);
            counter.add(now, now);
            long count = counter.size();
            if (count > limit) {
                System.out.println("[triggerLimit] path:" + path + " count:" + count + " over limit:" + limit);
                return true;
            }
            return false;
        } finally {
            rLock.unlock();
        }
    }
}

This approach eliminates the boundary issue but stores every request timestamp, which may increase memory usage under heavy load.

Leaky Bucket Algorithm

Principle

Requests pour into a bucket at any rate; the bucket leaks at a fixed rate. When the bucket is full, incoming requests are dropped.

Implementation

public class LeakyBucketRateLimiter {
    private RedissonClient redissonClient = RedissonConfig.getInstance();
    private static final String KEY_PREFIX = "LeakyBucket:";
    private Long bucketSize;
    private Long leakRate;
    public LeakyBucketRateLimiter(Long bucketSize, Long leakRate) {
        this.bucketSize = bucketSize;
        this.leakRate = leakRate;
        ScheduledExecutorService executor = Executors.newScheduledThreadPool(1);
        executor.scheduleAtFixedRate(this::leakWater, 0, 1, TimeUnit.SECONDS);
    }
    public void leakWater() {
        RSet<String> pathSet = redissonClient.getSet(KEY_PREFIX + ":pathSet");
        for (String path : pathSet) {
            String redisKey = KEY_PREFIX + path;
            RScoredSortedSet<Long> bucket = redissonClient.getScoredSortedSet(redisKey);
            long now = System.currentTimeMillis();
            bucket.removeRangeByScore(0, true, now - 1000 * leakRate, true);
        }
    }
    public boolean triggerLimit(String path) {
        RLock rLock = redissonClient.getLock(KEY_PREFIX + "LOCK:" + path);
        try {
            rLock.lock(100, TimeUnit.MILLISECONDS);
            RScoredSortedSet<Long> bucket = redissonClient.getScoredSortedSet(KEY_PREFIX + path);
            RSet<String> pathSet = redissonClient.getSet(KEY_PREFIX + ":pathSet");
            pathSet.add(path);
            long now = System.currentTimeMillis();
            if (bucket.size() < bucketSize) {
                bucket.add(now, now);
                return false; // not limited
            }
            System.out.println("[triggerLimit] path:" + path + " bucket size:" + bucket.size());
            return true; // limited
        } finally {
            rLock.unlock();
        }
    }
}

The fixed leak rate can waste capacity when a sudden burst occurs, because excess requests are simply dropped.

Token Bucket Algorithm

Principle

Tokens are added to a bucket at a constant rate; each request consumes a token. If no token is available, the request is rejected. This allows bursts as long as tokens exist.

Implementation

public class TokenBucketRateLimiter {
    public static final String KEY = "TokenBucketRateLimiter:";
    private Long limit;
    private Long tokenRate;
    public TokenBucketRateLimiter(Long limit, Long tokenRate) {
        this.limit = limit;
        this.tokenRate = tokenRate;
    }
    public boolean triggerLimit(String path) {
        RedissonClient redissonClient = RedissonConfig.getInstance();
        RRateLimiter rateLimiter = redissonClient.getRateLimiter(KEY + path);
        rateLimiter.trySetRate(RateType.OVERALL, limit, tokenRate, RateIntervalUnit.SECONDS);
        return rateLimiter.tryAcquire();
    }
}

Redisson’s built‑in token bucket implementation is concise and reliable.

Summary and Further Reading

The demo demonstrates three (actually four) rate‑limiting algorithms using Redisson. Improvements could include replacing distributed locks with Lua scripts for better performance, providing annotation‑based AOP integration, and adding configurable rejection strategies (exceptions, fallback queues, etc.).

Other popular open‑source rate‑limiting tools include:

Guava RateLimiter (single‑node token bucket)

Sentinel (sliding window, supports clustering)

Gateway solutions such as Spring Cloud Gateway and Nginx