Mastering Rate Limiting in Spring Boot: Token Bucket, Leaky Bucket, and Counter Techniques

Environment: Spring Boot 2.6.12 with Guava 30.1.1‑jre.

Rate limiting algorithms

Common algorithms include the counter method, the leaky‑bucket algorithm and the token‑bucket algorithm.

Counter

The counter limits the number of requests per second by incrementing an AtomicLong for each request and rejecting further requests once a threshold (e.g., 100 QPS) is reached. This can cause a “burst” problem when many requests arrive at the beginning of the interval.

Leaky Bucket

The leaky‑bucket algorithm smooths burst traffic by queuing requests in a bucket and releasing them at a constant rate; excess requests are dropped when the bucket overflows.

Token Bucket

The token‑bucket algorithm adds tokens to a bucket at a fixed rate; a request proceeds only if a token can be taken, otherwise it is rejected. It is the opposite of the leaky‑bucket approach.

Using Guava RateLimiter

Guava’s RateLimiter implements a token‑bucket algorithm. It does not support distributed environments.

<code>&lt;dependency&gt;
  &lt;groupId&gt;org.springframework.boot&lt;/groupId&gt;
  &lt;artifactId&gt;spring-boot-starter-web&lt;/artifactId&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
  &lt;groupId&gt;com.google.guava&lt;/groupId&gt;
  &lt;artifactId&gt;guava&lt;/artifactId&gt;
  &lt;version&gt;30.1.1-jre&lt;/version&gt;
&lt;/dependency&gt;
</code>

Example controller limiting to 3 requests per second:

<code>@RestController
@RequestMapping("/products")
public class ProductController {

    private final RateLimiter rateLimiter = RateLimiter.create(5.0);

    @GetMapping("/{id}")
    public ResponseEntity<R> queryProducts(@PathVariable("id") String id) throws Exception {
        if (rateLimiter.tryAcquire(1)) {
            TimeUnit.MILLISECONDS.sleep(200);
            return new ResponseEntity<>(R.success("Query product [" + id + "] succeeded"), HttpStatus.OK);
        }
        return new ResponseEntity<>(R.failure("Too many requests"), HttpStatus.INTERNAL_SERVER_ERROR);
    }
}
</code>

Testing with JMeter (100 concurrent threads, 2 loops) shows the rate limiter works as expected.

Baidu ratelimiter‑spring‑boot‑starter

This starter provides node‑level, method‑level and service‑level rate limiting based on token‑bucket algorithm.

<code>&lt;dependency&gt;
  &lt;groupId&gt;com.baidubce.formula&lt;/groupId&gt;
  &lt;artifactId&gt;ratelimiter-spring-boot-starter&lt;/artifactId&gt;
  &lt;version&gt;2.1.1.1&lt;/version&gt;
&lt;/dependency&gt;
</code>

<code>spring:
  application:
    name: ratelimiter
---
formula:
  ratelimiter:
    enabled: true
    ratelimiters:
      - effectiveLocation: /products/q/**
        effectiveType: 1
        enabled: true
        httpMethod: GET
        limiterType: 1
        threshold: 5
</code>

Customizing the response for blocked requests requires overriding HttpUtil.isBlockException :

<code>public static boolean isBlockException(HttpServletResponse response, Exception e) throws IOException {
    if (e instanceof BlockException) {
        response.setStatus(429);
        response.setContentType("application/json; charset=utf-8");
        response.setCharacterEncoding("UTF-8");
        response.getWriter().print("{\"code\": -1, \"message\": \"Your request is too fast\"}");
        response.flushBuffer();
        return true;
    } else {
        return false;
    }
}
</code>

After configuration and code changes, the service returns HTTP 429 for throttled requests.