Mastering Rate Limiting: Counter, Sliding Window, Leaky & Token Buckets + Redis Lua

Hello, I'm Su San!

In high‑concurrency e‑commerce scenarios, we often use methods such as rate limiting, circuit breaking, and degradation; today we discuss rate limiting.

Rate limiting restricts the number of concurrent requests reaching the system, ensuring normal responses for some users while rejecting excess traffic to maintain overall availability.

Based on scope, rate limiting can be single‑machine or distributed; based on method, it includes counter, sliding window, leaky bucket, and token bucket. Below we explain each in detail.

Most content references TPaper's "Go实现各类限流算法".

Common Rate Limiting Methods

Counter

The counter is the simplest algorithm: count requests within a time interval and compare to a threshold; reset the counter at the interval boundary.

Analogy: like a bus with limited seats; exceeding capacity leads to overload.

Implementation logic:

Set a variable

count

; increment on each request and record request time.

When the next request arrives, check if

count

exceeds the limit and if the request time is within one minute of the first request.

If within one minute and over the limit, reject the request.

If the interval has passed and

count

is still within the limit, reset

count

.

Example: an endpoint

/query

allows 200 requests per minute. If a user sends 200 requests in the last milliseconds of the 59th second, the counter resets at 60 seconds, allowing another 200 requests in the next second, effectively doubling the rate—a design flaw.

This method, while simple, does not handle time boundaries well and can be abused.

In high‑concurrency, the counter may use locks; it's better to use atomic counters to avoid contention.

Code details at: https://github.com/lml200701158/go_demo/blob/master/current_limit/count.go

Sliding Window

Sliding window addresses the boundary issue of the counter. It divides a fixed time slice into multiple slots that move forward with time, counting requests in each slot.

Illustration: a one‑minute window divided into six 10‑second slots; each slot has its own counter. When a request arrives at 0:45, the fifth slot counter increments. To decide rate limiting, sum all slot counters and compare to the threshold.

When a user sends 200 requests at 0:59, the sixth slot records +200; after the window slides, those 200 remain counted, triggering limit on subsequent requests.

Increasing the number of slots improves precision, but the algorithm still cannot fully solve boundary problems.

Code details at: https://github.com/RussellLuo/slidingwindow

Leaky Bucket

The leaky bucket algorithm models a bucket with fixed capacity that leaks at a constant rate.

Incoming requests fill the bucket; the bucket empties at a steady rate, shaping traffic and preventing bursts.

Characteristics:

Fixed capacity, constant outflow rate.

No outflow when bucket is empty.

Arbitrary inflow rate.

Excess inflow beyond capacity is dropped (requests rejected).

The maximum outflow rate limits the traffic, preventing bursts.

Code details at: https://github.com/lml200701158/go_demo/blob/master/current_limit/leaky_bucket.go

Token Bucket

Token bucket is widely used for traffic shaping and rate limiting, allowing bursts as long as tokens are available.

A bucket holds tokens; tokens are added at a fixed rate up to a maximum. When a request arrives, a token is removed; if none are available, the request is rejected or blocked.

Features:

Tokens added at a fixed rate.

Bucket capacity B; excess tokens are discarded.

If tokens < N, request is limited.

The algorithm limits average inflow rate while permitting bursts.

Code details at: https://github.com/lml200701158/go_demo/blob/master/current_limit/token_bucket.go

Redis + Lua Distributed Rate Limiting

Single‑machine limiting protects only the local node; distributed limiting controls the whole cluster, protecting downstream services.

The key is atomicity; using Redis counters with Lua scripts achieves this. Example Lua script:

local key = "rate.limit:" .. KEYS[1] -- limit key
local limit = tonumber(ARGV[1])        -- limit size
local current = tonumber(redis.call('get', key) or "0")
if current + 1 > limit then
  return 0
else
  redis.call("INCRBY", key, "1")
  redis.call("expire", key, "1")
  return current + 1
end

Java rate‑limiting logic using Jedis to evaluate the script:

public static boolean accquire() throws IOException, URISyntaxException {
    Jedis jedis = new Jedis("127.0.0.1");
    File luaFile = new File(RedisLimitRateWithLUA.class.getResource("/").toURI().getPath() + "limit.lua");
    String luaScript = FileUtils.readFileToString(luaFile);
    String key = "ip:" + System.currentTimeMillis()/1000; // current second
    String limit = "5";
    List<String> keys = new ArrayList<>();
    keys.add(key);
    List<String> args = new ArrayList<>();
    args.add(limit);
    Long result = (Long)(jedis.eval(luaScript, keys, args));
    return result == 1;
}

Other Considerations

Beyond server‑side limiting, containers like Tomcat and Nginx can also be limited; Tomcat via maxThreads, Nginx via rate or concurrent connection limits.

In Java, RateLimiter implements token bucket; in Go, channels or third‑party libraries can be used.

Rate limiting can also be applied per IP, city, channel, device ID, user ID, or per appkey in open platforms.

Rate Limiting Comparison

Summary of pros and cons:

Counter:

Pros: Simple, fixed interval counting, suitable for low‑precision needs.

Cons: Poor boundary handling, imprecise control.

Sliding Window:

Pros: Divides interval into slots for finer granularity.

Cons: Slightly more complex, still cannot fully solve boundary issues.

Leaky Bucket:

Pros: Controls consumption rate well.

Cons: Cannot handle bursts, less flexible.

Token Bucket:

Pros: Allows bursts while preventing over‑consumption; highly recommended.

Cons: Slightly more complex, other drawbacks minimal.

Redis + Lua Distributed:

Pros: Supports distributed limiting, protects downstream services.

Cons: Depends on Redis, still has boundary limitations.

Personal disclaimer about possible errors.