Understanding Guava RateLimiter and the Token Bucket Algorithm for Java Rate Limiting

Guava, a Google open‑source Java library, offers the RateLimiter utility for controlling request flow in high‑concurrency scenarios. By creating a limiter with a specified rate (e.g., 2 permits per second) and invoking acquire() before submitting tasks to a thread pool, the system ensures a stable interval (approximately 500 ms) between executions.

Example usage:

RateLimiter limiter = RateLimiter.create(2.0);
ExecutorService es = Executors.newFixedThreadPool(1);
long prev = System.nanoTime();
for (int i = 0; i < 20; i++) {
    limiter.acquire();
    es.execute(() -> {
        long cur = System.nanoTime();
        System.out.println((cur - prev) / 1000_000);
        prev = cur;
    });
}

The limiter is based on the token bucket algorithm, where tokens are added to a bucket at a fixed rate (r tokens per second). A request can proceed only if a token is available; otherwise it must wait until the next token is generated. The bucket also has a capacity (b), representing the maximum burst of requests that can be handled instantly.

Guava implements this algorithm without a dedicated timer. It records the timestamp of the next token release and updates it dynamically. A simplified implementation is shown below:

class SimpleLimiter {
    long next = System.nanoTime();
    long interval = 1_000_000_000L; // 1 second in nanoseconds

    synchronized long reserve(long now) {
        if (now > next) {
            next = now;
        }
        long at = next;
        next += interval;
        return Math.max(at, 0L);
    }

    void acquire() {
        long now = System.nanoTime();
        long at = reserve(now);
        long waitTime = Math.max(at - now, 0);
        if (waitTime > 0) {
            try { TimeUnit.NANOSECONDS.sleep(waitTime); } catch (InterruptedException e) { e.printStackTrace(); }
        }
    }
}

When the bucket capacity exceeds one, the implementation must track the number of stored permits and recalculate them based on elapsed time. The following code adds a resync method to refresh the permit count and adjusts reserve accordingly:

class SimpleLimiter {
    long storedPermits = 0;
    long maxPermits = 3;
    long next = System.nanoTime();
    long interval = 1_000_000_000L;

    void resync(long now) {
        if (now > next) {
            long newPermits = (now - next) / interval;
            storedPermits = Math.min(maxPermits, storedPermits + newPermits);
            next = now;
        }
    }

    synchronized long reserve(long now) {
        resync(now);
        long at = next;
        long fresh = Math.min(1, storedPermits);
        long needed = 1 - fresh;
        next += needed * interval;
        storedPermits -= fresh;
        return at;
    }

    void acquire() {
        long now = System.nanoTime();
        long at = reserve(now);
        long waitTime = Math.max(at - now, 0);
        if (waitTime > 0) {
            try { TimeUnit.NANOSECONDS.sleep(waitTime); } catch (InterruptedException e) { e.printStackTrace(); }
        }
    }
}

The article also contrasts the token bucket with the leaky bucket algorithm, noting that both serve as complementary approaches to rate limiting. Finally, it mentions that Guava's RateLimiter extends the basic token bucket with features such as warm‑up periods, which are useful for handling sudden traffic spikes.