How to Build a Dynamic IP Blacklist with Nginx, Lua, and Redis

Requirement

To block certain crawlers or malicious users, we need a dynamic IP blacklist that denies service for listed IPs and allows setting an expiration time for each block.

Environment Preparation

Linux distribution: CentOS 7 / Ubuntu etc.

Redis version: 5.0.5

Nginx version: OpenResty (nginx with Lua support)

Design Options

Three possible ways to implement an IP blacklist:

OS level (iptables) – simple but requires manual updates on each server.

Nginx level (deny directive or Lua plugin) – can be dynamic, supports distributed blocking, but requires Lua knowledge.

Application level (code check) – easy to maintain but may affect performance under high concurrency.

We choose the Nginx + Lua + Redis architecture for its dynamic capabilities and ease of sharing the blacklist across multiple servers.

Configure nginx.conf

Add the following to the location / block of the server that needs protection:

location / {
    access_by_lua_file /usr/local/lua/access_limit.lua;
    alias /usr/local/web/;
    index index.html index.htm;
}

Lua Script ( access_limit.lua )

-- Path: /usr/local/lua/access_limit.lua
-- Auto‑block IPs with high request frequency
local pool_max_idle_time = 10000
local pool_size = 100
local redis_connection_timeout = 100
local redis_host = "your redis host ip"
local redis_port = "your redis port"
local redis_auth = "your redis authpassword"
local ip_block_time = 120          -- block duration (seconds)
local ip_time_out = 1              -- time window for counting (seconds)
local ip_max_count = 3             -- max requests in the window

local function errlog(msg, ex)
    ngx.log(ngx.ERR, msg, ex)
end

local function close_redis(red)
    if not red then return end
    local ok, err = red:set_keepalive(pool_max_idle_time, pool_size)
    if not ok then
        ngx.say("redis connct err:", err)
        return red:close()
    end
end

local redis = require "resty.redis"
local client = redis:new()
local ok, err = client:connect(redis_host, redis_port)
if not ok then
    close_redis(client)
    ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
end
client:set_timeout(redis_connection_timeout)

local connCount, err = client:get_reused_times()
if connCount == 0 then
    local ok, err = client:auth(redis_auth)
    if not ok then errlog("failed to auth: ", err); return end
elseif err then
    errlog("failed to get reused times: ", err); return
end

local function getIp()
    local clientIP = ngx.req.get_headers()["X-Real-IP"]
    if not clientIP then clientIP = ngx.req.get_headers()["x_forwarded_for"] end
    if not clientIP then clientIP = ngx.var.remote_addr end
    return clientIP
end

local cliendIp = getIp()
local incrKey = "limit:count:" .. cliendIp
local blockKey = "limit:block:" .. cliendIp

local is_block = client:get(blockKey)
if tonumber(is_block) == 1 then
    ngx.exit(ngx.HTTP_FORBIDDEN)
    close_redis(client)
end

local ip_count = client:incr(incrKey)
if tonumber(ip_count) == 1 then client:expire(incrKey, ip_time_out) end
if tonumber(ip_count) > tonumber(ip_max_count) then
    client:set(blockKey, 1)
    client:expire(blockKey, ip_block_time)
end

close_redis(client)

Summary

The Nginx + Lua + Redis solution provides a lightweight, easily configurable IP blacklist with minimal performance impact, supports sharing the list across multiple servers via Redis, and allows dynamic updates either manually or through automation.

Extensions

Application Scenarios

Prevent malicious access such as brute‑force attacks, SQL injection, XSS.

Block crawlers and data‑scraping bots.

Mitigate DDoS attacks.

Rate‑limit requests from a single IP.

Advanced Features and Improvements

Automatic anomaly detection and blocking based on log analysis.

Whitelist mechanism for trusted IPs.

CAPTCHA challenges for suspicious IPs.

Statistics and analysis of blocked IPs for further optimization.

Continuous refinement of the blacklist enhances server security and user experience.