Build a Dynamic IP Blacklist with Nginx, Lua, and Redis

Requirement

To block certain crawlers or malicious users, we need a dynamic IP blacklist that denies service to listed IPs and can set an expiration time for each block.

Environment Preparation

Linux (CentOS7 / Ubuntu etc.)

Redis 5.0.5

Nginx (OpenResty)

Design Options

Three ways to implement an IP blacklist:

OS level using iptables – simple but requires manual updates.

Nginx level using deny or Lua plugins – dynamic and can set expiration, but needs Lua knowledge.

Application level checking the IP before processing – easy to code but may affect performance under high concurrency.

We choose the Nginx + Lua + Redis approach for its dynamic configurability and ability to share the blacklist across multiple servers.

Configure nginx.conf

location / {
    # optional static file check
    # if ($request_uri ~ .*\\.(html|htm|jpg|js|css)) {
    #     access_by_lua_file /usr/local/lua/access_limit.lua;
    # }
    access_by_lua_file /usr/local/lua/access_limit.lua;
    alias /usr/local/web/;
    index index.html index.htm;
}

Configure Lua Script

local pool_max_idle_time = 10000
local pool_size = 100
local redis_connection_timeout = 100
local redis_host = "your redis host ip"
local redis_port = "your redis port"
local redis_auth = "your redis authpassword"
local ip_block_time = 120
local ip_time_out = 1
local ip_max_count = 3

function errlog(msg, ex)
    ngx.log(ngx.ERR, msg, ex)
end

function close_redis(red)
    if not red then return end
    local ok, err = red:set_keepalive(pool_max_idle_time, pool_size)
    if not ok then
        ngx.say("redis connct err:", err)
        return red:close()
    end
end

local redis = require "resty.redis"
local client = redis:new()
local ok, err = client:connect(redis_host, redis_port)
if not ok then
    close_redis(client)
    ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
end
client:set_timeout(redis_connection_timeout)
local connCount, err = client:get_reused_times()
if connCount == 0 then
    local ok, err = client:auth(redis_auth)
    if not ok then errlog("failed to auth: ", err) return end
end

local function getIp()
    local ip = ngx.req.get_headers()["X-Real-IP"]
    if not ip then ip = ngx.req.get_headers()["x_forwarded_for"] end
    if not ip then ip = ngx.var.remote_addr end
    return ip
end

local clientIP = getIp()
local incrKey = "limit:count:" .. clientIP
local blockKey = "limit:block:" .. clientIP

local is_block = client:get(blockKey)
if tonumber(is_block) == 1 then
    ngx.exit(ngx.HTTP_FORBIDDEN)
    close_redis(client)
end

local ip_count = client:incr(incrKey)
if tonumber(ip_count) == 1 then
    client:expire(incrKey, ip_time_out)
end
if tonumber(ip_count) > ip_max_count then
    client:set(blockKey, 1)
    client:expire(blockKey, ip_block_time)
end

close_redis(client)

Summary

The Nginx + Lua + Redis solution provides a lightweight configuration with minimal performance impact, allows multiple servers to share a common blacklist via Redis, and supports dynamic updates and timed expirations.