How to Build a Dynamic IP Blacklist with Nginx, Lua, and Redis

To block malicious crawlers or users, a dynamic IP blacklist is required that rejects requests from listed IPs and allows setting an expiration time.

Environment Preparation

Linux (CentOS7 / Ubuntu)

Redis 5.0.5

Nginx (OpenResty)

Design Options

OS level (iptables) – simple and direct but manual and inflexible.

Nginx deny or Lua plugin – can be dynamic and support distributed bans, but requires Lua knowledge.

Application‑layer check – easy to maintain via code, yet may add overhead under high concurrency.

For a lightweight, shared, and dynamic solution we choose the Nginx + Lua + Redis architecture.

Configure nginx.conf

Add the following to the location block that needs protection:

location / {
    # access_by_lua_file loads the Lua script that implements rate limiting
    access_by_lua_file /usr/local/lua/access_limit.lua;
    alias /usr/web/;
    index index.html index.htm;
}

Lua Script ( access_limit.lua )

-- This script implements automatic blacklisting of IPs that exceed request limits
local pool_max_idle_time = 10000
local pool_size = 100
local redis_connection_timeout = 100
local redis_host = "your redis host ip"
local redis_port = "your redis port"
local redis_auth = "your redis authpassword"
local ip_block_time = 120          -- block duration (seconds)
local ip_time_out = 1               -- counting window (seconds)
local ip_max_count = 3              -- max requests in window

local function errlog(msg, ex)
    ngx.log(ngx.ERR, msg, ex)
end

local function close_redis(red)
    if not red then return end
    local ok, err = red:set_keepalive(pool_max_idle_time, pool_size)
    if not ok then
        ngx.say("redis connct err:", err)
        return red:close()
    end
end

local redis = require "resty.redis"
local client = redis:new()
local ok, err = client:connect(redis_host, redis_port)
if not ok then
    close_redis(client)
    ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
end
client:set_timeout(redis_connection_timeout)
local connCount, err = client:get_reused_times()
if connCount == 0 then
    local ok, err = client:auth(redis_auth)
    if not ok then errlog("failed to auth: ", err) return end
end

local function getIp()
    local clientIP = ngx.req.get_headers()["X-Real-IP"]
    if not clientIP then clientIP = ngx.req.get_headers()["x_forwarded_for"] end
    if not clientIP then clientIP = ngx.var.remote_addr end
    return clientIP
end

local cliendIp = getIp()
local incrKey = "limit:count:" .. cliendIp
local blockKey = "limit:block:" .. cliendIp

local is_block, err = client:get(blockKey)
if tonumber(is_block) == 1 then
    ngx.exit(ngx.HTTP_FORBIDDEN)
    close_redis(client)
end

local ip_count, err = client:incr(incrKey)
if tonumber(ip_count) == 1 then
    client:expire(incrKey, ip_time_out)
end

if tonumber(ip_count) > tonumber(ip_max_count) then
    client:set(blockKey, 1)
    client:expire(blockKey, ip_block_time)
end

close_redis(client)

Summary

Simple, lightweight configuration with minimal performance impact.

Multiple servers can share the same Redis instance to synchronize the blacklist.

Dynamic updates can be performed manually or via automation by modifying Redis entries.

Application Scenarios

Prevent malicious access such as brute‑force attacks, SQL injection, or XSS.

Block crawlers and data‑scraping bots to reduce load and protect data.

Mitigate DDoS attacks by banning offending IPs.

Rate‑limit specific IPs within a time window to stop abuse.

Advanced Features and Improvements

Automatic anomaly detection and blacklisting based on log analysis.

Whitelist mechanism to allow trusted IPs to bypass restrictions.

CAPTCHA challenges for suspicious IPs.

Statistics and analysis of blacklist data (frequency, duration, etc.) for continuous optimization.

Diagram