Implementing Dynamic IP Blocking in Nginx with Lua and Redis

To block malicious crawlers or abusive users, a dynamic IP blacklist is required that can deny service for listed IPs and set expiration times.

Design Options

OS level (iptables) : Simple and direct, but requires manual updates on each server and lacks flexibility.

Nginx level (deny or Lua plugin) : Supports dynamic blocking via Redis and can set block durations, but introduces a learning curve for Lua and Nginx configuration.

Application level : Implemented in code, easy to maintain but may become verbose and impact performance under high concurrency.

The chosen solution combines nginx‑openresty, Lua scripts, and a Redis instance to achieve a lightweight, distributed blacklist.

Environment Preparation

Linux (CentOS 7 / Ubuntu, etc.)

Redis 5.0.5

Nginx‑OpenResty

nginx.conf Configuration

location / {
    access_by_lua_file /usr/local/lua/access_limit.lua;
    alias /usr/local/web/;
    index index.html index.htm;
}

Lua Script (access_limit.lua)

The script defines connection‑pool parameters, Redis connection details, and block/limit settings:

local pool_max_idle_time = 10000
local pool_size = 100
local redis_connection_timeout = 100
local redis_host = "your redis host ip"
local redis_port = "your redis port"
local redis_auth = "your redis authpassword"
local ip_block_time = 120          -- block duration (seconds)
local ip_time_out = 1              -- rate‑limit window (seconds)
local ip_max_count = 3             -- max requests per window

Key functions: errlog(msg, ex) – logs errors. close_redis(red) – returns the Redis connection to the pool. getIp() – extracts the client IP from X‑Real‑IP, x_forwarded_for, or remote_addr.

Processing flow:

Connect to Redis; authenticate if the connection is new.

Construct keys limit:count:<IP> and limit:block:<IP>.

Check if the IP is already blocked; if so, return 403 Forbidden.

Increment the request counter; set an expiration for the counter on the first hit.

If the counter exceeds ip_max_count, set the block key with expiration ip_block_time.

Close the Redis connection.

Advantages

Simple, lightweight configuration with minimal performance impact.

Multiple servers can share the same Redis instance, enabling a unified blacklist.

Dynamic updates allow manual or automated modifications of the Redis blacklist.

Extensions

Typical use cases include preventing malicious access, throttling crawlers, mitigating DDoS attacks, and limiting request frequency. Advanced features that can be added are anomaly detection with automatic bans, whitelist mechanisms, captcha challenges for suspicious IPs, and statistical analysis of blacklist activity.