Mastering Rate Limiting in Nginx: Leaky & Token Bucket Algorithms Explained

First, an overview of leaky bucket and token bucket algorithms. Nginx does not implement them directly, but they are useful for traffic shaping in network programming, API services, load balancing, and similar scenarios.

Leaky Bucket Algorithm :

* The leaky bucket algorithm limits data transmission rate. Requests are treated as water flow, and the bucket's outflow rate is the processing speed.
* Incoming requests are placed into the bucket; if the bucket is full, new requests are rejected or dropped.
* A drawback is poor handling of burst traffic; even when the bucket is not full, bursts are still limited.

Token Bucket Algorithm :

* The token bucket algorithm controls transmission rate while allowing bursts.
* Tokens are generated at a constant rate and stored in a bucket; requests must consume tokens to be processed.
* If enough tokens exist, bursts are handled; otherwise requests are rejected or queued.
* Advantages include better burst handling while maintaining average rate.

In Nginx, you can simulate these algorithms using the ngx_http_limit_req_module and ngx_http_limit_conn_module directives.

ngx_http_limit_req_module (request limiting) Example

The module limits request processing rate for a specific location, protecting backend servers.

Example configuration limiting requests to the /api/ path:

http {
    # ... other config ...

    limit_req_zone $binary_remote_addr zone=mylimit:10m rate=10r/s;

    server {
        # ... other config ...

        location /api/ {
            limit_req zone=mylimit burst=5 nodelay;

            # ... other location config ...

            proxy_pass http://backend_servers;
        }
    }
}

In this configuration: limit_req_zone defines a shared memory zone mylimit to store request state, using the client IP as the key; the zone size is 10 MB and the rate is 10 requests per second. limit_req applies the limit in the location block, referencing the zone, with a burst of 5 additional requests and nodelay to return a 503 error immediately when the limit is exceeded.

ngx_http_limit_conn_module (connection limiting) Example

The module limits the number of concurrent connections per IP address.

Example configuration limiting concurrent connections to the root path:

http {
    # ... other config ...

    limit_conn_zone $binary_remote_addr zone=perip:10m;

    server {
        # ... other config ...

        location / {
            limit_conn perip 10;

            # ... other location config ...

            proxy_pass http://backend_servers;
        }
    }
}

In this configuration: limit_conn_zone defines a shared memory zone perip to store connection state, keyed by client IP, with a size of 10 MB. limit_conn applies the limit in the location block, allowing up to 10 concurrent connections per IP.

Both modules help protect Nginx and backend servers from excessive requests or connections, improving system stability and performance.