Master Gray Deployments: Nginx + Lua + Redis for Dynamic Canary Releases

Preface

Teach a man to fish is better than give him a fish. First learn to use, then understand the principle, then create. This article records how to use Nginx+Lua+Redis to achieve gray releases, and how the same capability can be adapted to other data sources or scripting languages.

1. Gray Release Schemes

Common implementations

Request routing: use identifiers such as user ID, device ID, or request headers to decide whether to route to a gray environment, typically via reverse proxies (Nginx, Envoy) or API gateways (Kong, Apigee).

Weight control: allocate traffic to different environments by weight, using load balancers (HAProxy, Kubernetes Ingress) or proxy servers (Nginx, Envoy).

Feature flags: embed feature switches in code and manage them via config files, databases, key‑value stores, or platforms like LaunchDarkly and Unleash.

Phased rollout: move a feature from internal testing to gray environment and finally to full release, using CI/CD tools (Jenkins, GitLab CI/CD) or cloud platforms (AWS, Azure).

A/B testing: split traffic into different versions and compare performance and feedback, using platforms such as Optimizely or Google Optimize.

Canary release: gradually shift a new version into production with a small amount of traffic, increasing it based on stability, using deployment tools, container orchestration, or cloud platforms.

Typical gray‑release methods

Based on user ID: hash or random the user ID to decide gray participation.

Based on IP address: designate a range of IPs as gray users.

Cookie/Session: set a marker in the user's cookie or session and route accordingly.

Request header: use a custom header or specific HTTP header to identify gray traffic.

Weight or percentage: randomly assign requests to environments according to configured weights.

A/B testing: compare multiple versions during the experiment and promote the best one.

2. Implementing Gray Release with Nginx+Lua+Redis

Theory

1. Install and configure Nginx and Redis, ensuring the Lua module is enabled.

2. Define gray rules in the Nginx configuration and use a Lua script to decide whether a request should be routed to the gray environment.

server {
    listen 80;
    server_name example.com;
    location / {
        access_by_lua_block {
            local redis = require "resty.redis"
            local red = redis:new()
            -- connect to Redis
            local ok, err = red:connect("redis_host", redis_port)
            if not ok then
                ngx.log(ngx.ERR, "failed to connect to Redis: ", err)
                ngx.exit(500)
            end
            -- use Redis to decide gray routing based on user ID
            local user_id = ngx.req.get_headers()["X-User-ID"]
            local is_gray = red:get("gray:" .. user_id)
            if is_gray == "1" then
                ngx.var.upstream = "gray_backend"
            end
        }
        proxy_pass http://backend;
    }
    location /gray {
        # gray environment configuration
        proxy_pass http://gray_backend;
    }
    location /admin {
        # admin backend configuration
        proxy_pass http://admin_backend;
    }
}

The script connects to Redis, reads the key gray:<user_id>, and if the value is 1 it sets the upstream variable to the gray backend.

3. Store gray users in Redis using simple SET/GET commands.

-- set user as gray user
local ok, err = red:set("gray:" .. user_id, 1)
if not ok then
    ngx.log(ngx.ERR, "failed to set gray status for user: ", err)
    ngx.exit(500)
end

-- set user as non‑gray user
local ok, err = red:set("gray:" .. user_id, 0)
if not ok then
    ngx.log(ngx.ERR, "failed to set gray status for user: ", err)
    ngx.exit(500)
end

By updating the Redis key you can dynamically control which users are routed to the gray environment.

4. Additional paths or features can be added to the Nginx configuration to support more complex gray‑release strategies.

Practice

Using OpenResty

OpenResty (ngx_openresty) is a scalable web platform based on Nginx that allows Lua scripts to call C and Lua modules, making it ideal for implementing gray releases.

OpenResty API documentation: https://www.kancloud.cn/qq13867685/openresty-api-cn/159190

1. Route by POST request URL parameters

Nginx configuration

#user  nobody;
worker_processes 1;
#error_log  logs/error.log;
#pid        logs/nginx.pid;

events {
    worker_connections 1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    log_format  logFormat '$group $time_local client:$remote_addr-$remote_port request:$request host:$http_host status:$status upstream:$upstream_status upstream_addr:$upstream_addr';
    access_log  logs/access.log  logFormat;
    sendfile        on;
    keepalive_timeout 65;
    server{
        listen       80;
        server_name  example.com;
        access_log  logs/example.access.log  logFormat;
        location / {
            default_type 'text/plain';
            content_by_lua_file 'D:/user/Downloads/openresty-1.19.9.1-win64/conf/dep.lua';
        }
    }
    upstream default {
        server 10.0.0.1:8080;
    }
    upstream new_version {
        server 10.0.0.2:8080;
    }
    upstream old_version {
        server 10.0.0.3:8080;
    }
}

Lua script

-- get request uri parameters
function SaveTableContent(file, obj)
    local szType = type(obj)
    if szType == "number" then
        file:write(obj)
    elseif szType == "string" then
        file:write(string.format("%q", obj))
    elseif szType == "table" then
        for i, v in pairs(obj) do
            SaveTableContent(file, i)
            file:write(":")
            SaveTableContent(file, v)
            file:write(",")
        end
    else
        error("can't serialize a "..szType)
    end
end

function SaveTable(obj)
    local file = io.open("D:/user/Downloads/openresty-1.19.9.1-win64/logs/params.txt", "a")
    assert(file)
    SaveTableContent(file,obj)
    file:close()
end

local request_method = ngx.var.request_method
local getargs = nil
local args = nil
local read_body = nil
local body_data = nil
local thirdPolicystatus = nil
if "GET" == request_method then
    args = ngx.req.get_uri_args()
elseif "POST" == request_method then
    getargs = ngx.req.get_uri_args()
    args   = ngx.req.get_post_args()
    read_body = ngx.req.read_body()
    body_data = ngx.req.get_body_data()
end

if getargs ~= nil then
    SaveTable(getargs)
    thirdPolicystatus = getargs["thirdPolicystatus"]
    if thirdPolicystatus ~= nil then
        SaveTable(thirdPolicystatus)
    end
end

if args ~= nil then
    SaveTable(args)
end

if read_body ~= nil then
    SaveTable(read_body)
end

if body_data ~= nil then
    SaveTable(body_data)
end

if args ~= nil then
    if type(args) == "table" then
        thirdPolicystatus = tostring(args["thirdPolicystatus"])
        if thirdPolicystatus == 1 then
            SaveTable("new_version-args-table")
            ngx.exec('@new_version')
        elseif thirdPolicystatus == 2 then
            SaveTable("old_version-args-table")
            ngx.exec('@old_version')
        else
            SaveTable("default_version-args-table")
            ngx.exec('@default_version')
        end
    elseif type(args) == "string" then
        local json = require "cjson"
        local jsonObj = json.decode(args)
        thirdPolicystatus = jsonObj['thirdPolicystatus']
        if thirdPolicystatus == 1 then
            SaveTable("new_version-args-string")
            ngx.exec('@new_version')
        elseif thirdPolicystatus == 2 then
            SaveTable("old_version-args-string")
            ngx.exec('@old_version')
        else
            SaveTable("default_version-args-string")
            ngx.exec('@default_version')
        end
    end
end

return

2. Route by request parameters or IP using Redis cache

Redis download: https://github.com/tporadowski/redis/releases

Nginx configuration

#user  nobody;
worker_processes 1;

events {
    worker_connections 1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    log_format  logFormat '$group $time_local client:$remote_addr-$remote_port request:$request host:$http_host status:$status upstream:$upstream_status upstream_addr:$upstream_addr';
    access_log  logs/access.log  logFormat;
    sendfile        on;
    keepalive_timeout 65;
    server {
        listen 80;
        server_name example.com;
        access_log logs/example.access.log logFormat;
        location /redis {
            default_type 'text/plain';
            content_by_lua 'ngx.say("hello ,lua scripts redis")';
        }
        location / {
            default_type 'text/plain';
            lua_need_request_body on;
            content_by_lua_file 'D:/user/Downloads/openresty-1.19.9.1-win64/conf/redis.lua';
        }
        location @pre-prd {
            proxy_pass http://pre-prd;
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
        location @prd {
            proxy_pass http://prd;
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
    }
    upstream prd { server 10.0.0.4:8080; }
    upstream pre-prd { server 10.0.0.5:8080; }
}

Lua script

-- get request uri parameters
function SaveTableContent(file, obj)
    local szType = type(obj)
    if szType == "number" then
        file:write(obj)
    elseif szType == "string" then
        file:write(string.format("%q", obj))
    elseif szType == "table" then
        for i, v in pairs(obj) do
            SaveTableContent(file, i)
            file:write(":")
            SaveTableContent(file, v)
            file:write(",")
        end
    else
        error("can't serialize a "..szType)
    end
end

function SaveTable(obj)
    local file = io.open("D:/user/Downloads/openresty-1.19.9.1-win64/logs/redis.txt", "a")
    assert(file)
    SaveTableContent(file,obj)
    file:close()
end

local request_method = ngx.var.request_method
local getargs = nil
local args = nil
local read_body = nil
local body_data = nil
local thirdPolicystatus = nil
if "GET" == request_method then
    args = ngx.req.get_uri_args()
elseif "POST" == request_method then
    getargs = ngx.req.get_uri_args()
    args   = ngx.req.get_post_args()
    read_body = ngx.req.read_body()
    body_data = ngx.req.get_body_data()
end

if getargs ~= nil then
    SaveTable("getargs")
    SaveTable(getargs)
    thirdPolicystatus = getargs["thirdPolicystatus"]
    if thirdPolicystatus ~= nil then
        SaveTable("thirdPolicystatus")
        SaveTable(thirdPolicystatus)
    end
end

if args ~= nil then
    SaveTable("args")
    SaveTable(args)
end

if read_body ~= nil then
    SaveTable("read_body")
    SaveTable(read_body)
end

if body_data ~= nil then
    SaveTable("body_data")
    SaveTable(body_data)
end

local redis = require "resty.redis"
local cache = redis:new()
cache:set_timeout(60000)
local ok, err = cache:connect('127.0.0.1', 6379)
if not ok then
    SaveTable("not ok")
    ngx.exec("@prd")
    return
end

local local_ip = ngx.req.get_headers()["X-Real-IP"]
if local_ip == nil then
    local_ip = ngx.req.get_headers()["x_forwarded_for"]
    SaveTable("local_ip1")
end
if local_ip == nil then
    local_ip = ngx.var.remote_addr
    SaveTable("local_ip2")
end

local res, err = cache:get(local_ip)
if res == "1" then
    SaveTable(res)
    SaveTable("pre-prd")
    ngx.exec("@pre-prd")
    return
else
    SaveTable("-------")
    SaveTable(local_ip)
    SaveTable(res)
    cache:set(local_ip, "1")
end

SaveTable("prd")
ngx.exec("@prd")

local ok, err = cache:close()
if not ok then
    ngx.say("failed to close:", err)
    return
end
return

3. Related Configuration and Syntax

1. Nginx configuration file details

Source code: https://trac.nginx.org/nginx/browser

Official site: http://www.nginx.org/

Windows installer: https://nginx.org/en/download.html

# Global directives
worker_processes 1;
error_log logs/error.log error;

events {
    worker_connections 2048;
    accept_mutex on;
    multi_accept on;
}

http {
    include mime.types;
    default_type application/octet-stream;
    log_format myFormat '$time_local client:$remote_addr-$remote_port request:$request host:$http_host status:$status upstream:$upstream_status upstream_addr:$upstream_addr referrer:$http_referer bytes:$body_bytes_sent agent:$http_user_agent uri:$request_uri';
    access_log logs/access.log myFormat;
    sendfile on;
    tcp_nopush on;
    keepalive_timeout 65;
    gzip on;
    gzip_min_length 10k;
    gzip_comp_level 6;
    gzip_buffers 16 8k;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    server {
        listen 80;
        server_name localhost;
        location / {
            root html;
            index index.html index.htm;
        }
        error_page 500 502 503 504 /50x.html;
        location = /50x.html { root html; }
    }

    server {
        listen 8081;
        server_name example.com;
        charset utf-8;
        client_max_body_size 10M;
        location / {
            proxy_pass http://stream;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_redirect off;
        }
        location ~ ^/(images|javascript|js|css|flash|media|static)/ {
            root /var/www/static_files;
            expires 10d;
        }
        error_page 500 502 503 504 /50x.html;
        location = /50x.html { root html; }
    }

    upstream insurance-pre {
        server 10.0.0.6:8080 weight=5;
        server 10.0.0.7:8080 weight=1;
        server 10.0.0.8:8080 backup;
    }
}

4. Extensibility and Alternatives

Idea 1: Build a dynamic configuration console (e.g., a Java web app) that updates Redis in real time to control gray releases.

Idea 2: Replace Redis with other data sources such as MySQL, PostgreSQL, MongoDB, HTTP APIs, or Cassandra using appropriate Lua libraries.

Idea 3: Use other scripting languages—JavaScript via ngx_http_js_module, LuaJIT for higher performance, Python via Python‑NGINX‑Module, or Java via nginx‑jvm‑clojure.

Idea 4: Swap Nginx for other reverse‑proxy or web servers like Apache HTTP Server, Microsoft IIS, Caddy, HAProxy, or Envoy.

Explore the concepts that interest you; this article does not provide exhaustive details.

Link: https://www.cnblogs.com/Jcloud/p/17910370.html

(© Original author, all rights reserved)