Implementing a Dynamic IP Blacklist with Nginx, Lua, and Redis

Requirement: To block certain crawlers or malicious users, a dynamic IP blacklist with expiration is needed.

Environment: Linux (CentOS7/Ubuntu), Redis 5.0.5, Nginx OpenResty.

Design options:

OS‑level iptables – simple but requires manual updates.

Nginx deny or Lua plugin – dynamic, supports expiration, but needs Lua knowledge.

Application‑level check – easy to code, may affect performance under high concurrency.

The chosen solution combines Nginx, Lua, and Redis to achieve a lightweight, shareable blacklist.

Configure nginx.conf by adding an access_by_lua_file directive in the relevant server location block:

location / {
    # If static resources exist, you could add a conditional check here
    # if ($request_uri ~ .*\.(html|htm|jpg|js|css)) {
    #     access_by_lua_file /usr/local/lua/access_limit.lua;
    # }
    access_by_lua_file /usr/local/lua/access_limit.lua;  # Executes rate‑limit logic
    alias /usr/local/web/;
    index index.html index.htm;
}

The Lua script /usr/local/lua/access_limit.lua connects to Redis, tracks request counts per IP, blocks IPs that exceed a threshold, and sets expiration for both the count key and the block key.

-- Can automatically add IPs with high request frequency to the blacklist for a period

-- Connection pool settings
local pool_max_idle_time = 10000   -- ms
local pool_size = 100
local redis_connection_timeout = 100
local redis_host = "your redis host ip"
local redis_port = "your redis port"
local redis_auth = "your redis authpassword"
local ip_block_time = 120          -- seconds
local ip_time_out = 1              -- seconds for rate‑limit window
local ip_max_count = 3            -- max requests in the window

local function errlog(msg, ex)
    ngx.log(ngx.ERR, msg, ex)
end

local function close_redis(red)
    if not red then
        return
    end
    local ok, err = red:set_keepalive(pool_max_idle_time, pool_size)
    if not ok then
        ngx.say("redis connct err:", err)
        return red:close()
    end
end

local redis = require "resty.redis"
local client = redis:new()
local ok, err = client:connect(redis_host, redis_port)
if not ok then
    close_redis(client)
    ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
end
client:set_timeout(redis_connection_timeout)

local connCount, err = client:get_reused_times()
if 0 == connCount then
    local ok, err = client:auth(redis_auth)
    if not ok then
        errlog("failed to auth: ", err)
        return
    end
elseif err then
    errlog("failed to get reused times: ", err)
    return
end

local function getIp()
    local clientIP = ngx.req.get_headers()["X-Real-IP"]
    if clientIP == nil then
        clientIP = ngx.req.get_headers()["x_forwarded_for"]
    end
    if clientIP == nil then
        clientIP = ngx.var.remote_addr
    end
    return clientIP
end

local cliendIp = getIp()
local incrKey = "limit:count:"..cliendIp
local blockKey = "limit:block:"..cliendIp

-- Check if IP is already blocked
local is_block, err = client:get(blockKey)
if tonumber(is_block) == 1 then
    ngx.exit(ngx.HTTP_FORBIDDEN)
    close_redis(client)
end

local ip_count, err = client:incr(incrKey)
if tonumber(ip_count) == 1 then
    client:expire(incrKey, ip_time_out)
end

-- If request count exceeds limit, block the IP
if tonumber(ip_count) > tonumber(ip_max_count) then
    client:set(blockKey, 1)
    client:expire(blockKey, ip_block_time)
end

close_redis(client)

Summary: The Nginx+Lua+Redis implementation provides a simple, lightweight blacklist that can be shared across multiple servers via Redis, supports dynamic updates, and has minimal impact on server performance.

Extensions:

Common scenarios: preventing malicious access, blocking crawlers, mitigating DDoS attacks, and rate‑limiting.

Advanced features: anomaly detection with automatic blocking, white‑list support, captcha challenges for suspicious IPs, and statistical analysis of blacklist data.

By continuously improving these mechanisms, servers and applications can achieve stronger security and better user experience.