Implementing Gray Release with Nginx, Lua, and Memcached: Step‑by‑Step Guide

1. Gray Release Principle

According to Baidu Baike, gray release is a smooth transition method between black and white releases, where an A/B test directs a portion of users to version A and the rest to version B; if B is accepted, the rollout expands until all users run B. This approach helps maintain system stability and detect issues early.

The process for a web system involves directing some IP‑based users to a new version while others continue using the current version, as illustrated below.

Execution Flow

When a request reaches the front‑end Nginx proxy, the embedded Lua module parses the Lua script defined in the Nginx configuration.

The Lua script obtains the client IP, queries memcached for a key matching that IP, and decides whether to execute @client_test (new version) or @client (normal version). @client_test forwards the request to the server running the new code; @client forwards to the server running the stable code. The selected server returns the response.

2. Installation and Configuration Details

Step 1 – Install Nginx and dependencies

yum -y install gcc gcc-c++ autoconf libjpeg libjpeg-devel libpng libpng-devel freetype freetype-devel libxml2 libxml2-devel zlib zlib-devel glibc glibc-devel glib2 glib2-devel bzip2 bzip2-devel ncurses ncurses-devel curl curl-devel e2fsprogs e2fsprogs-devel krb5 krb5-devel libidn libidn-devel openssl openssl-devel openldap openldap-devel nss_ldap openldap-clients openldap-servers make pcre-devel

yum -y install gd gd2 gd-devel lua lua-devel

yum -y install memcached

Step 2 – Download Lua modules, memcached library, and Nginx source

wget https://github.com/simpl/ngx_devel_kit/archive/v0.2.18.tar.gz
wget https://github.com/chaoslawful/lua-nginx-module/archive/v0.8.5.tar.gz
wget https://github.com/agentzh/lua-resty-memcached/archive/v0.11.tar.gz
wget http://nginx.org/download/nginx-1.4.2.tar.gz

Extract and compile:

tar xvf nginx-1.4.2.tar.gz
cd nginx-1.4.2/
./configure \
  --prefix=/soft/nginx/ \
  --with-http_gzip_static_module \
  --add-module=/root/ngx_devel_kit-0.2.18/ \
  --add-module=/root/lua-nginx-module-0.8.5/
make && make install

Step 3 – Copy Lua memcached library

tar xvf v0.11.tar.gz
cp -r lua-resty-memcached-0.11/lib/resty/ /usr/lib64/lua/5.1/

Step 4 – Configure nginx.conf

#vim /soft/nginx/conf/nginx.conf
worker_processes  1;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;

    proxy_next_upstream  error timeout;
    proxy_redirect        off;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $http_x_forwarded_for;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    client_max_body_size 100m;
    client_body_buffer_size 256k;
    proxy_connect_timeout   180;
    proxy_send_timeout      180;
    proxy_read_timeout      180;
    proxy_buffer_size       8k;
    proxy_buffers   8 64k;
    proxy_busy_buffers_size 128k;
    proxy_temp_file_write_size 128k;

    upstream client {
        server 192.168.200.29:80;
    }
    upstream client_test {
        server 192.168.200.29:81;
    }

    server {
        listen       80;
        server_name  localhost;
        location / {
            content_by_lua '
                clientIP = ngx.req.get_headers()["X-Real-IP"]
                if clientIP == nil then
                    clientIP = ngx.req.get_headers()["x_forwarded_for"]
                end
                if clientIP == nil then
                    clientIP = ngx.var.remote_addr
                end
                local memcached = require "resty.memcached"
                local memc, err = memcached:new()
                if not memc then
                    ngx.say("failed to instantiate memc: ", err)
                    return
                end
                local ok, err = memc:connect("127.0.0.1", 11211)
                if not ok then
                    ngx.say("failed to connect: ", err)
                    return
                end
                local res, flags, err = memc:get(clientIP)
                if err then
                    ngx.say("failed to get clientIP ", err)
                    return
                end
                if res == "1" then
                    ngx.exec("@client_test")
                    return
                end
                ngx.exec("@client")
            ';
        }
        location @client { proxy_pass http://client; }
        location @client_test { proxy_pass http://client_test; }
        location /hello {
            default_type "text/plain";
            content_by_lua 'ngx.say("hello, lua")';
        }
        location = /50x.html { root html; }
    }
}

Test the configuration: /soft/nginx/sbin/nginx -t Start services:

/soft/nginx/sbin/nginx

memcached -u nobody -m 1024 -c 2048 -p 11211 -d

3. Testing and Verification

Verify the Lua module by accessing http://<em>server_ip</em>/hello; the response should be hello.

Set up two virtual hosts on the test machine (IP 192.168.200.29): port 80 runs the normal code, port 81 runs the gray‑test code.

Store the client IP as a key in memcached with value 1 (e.g., IP 192.168.68.211):

telnet localhost 11211
set 192.168.68.211 0 3600 1
1
STORED

Now a request from that IP is routed to the gray‑test server. Deleting the key from memcached returns traffic to the normal server.

Key details of the memcached set command: the first value is the key, the second is custom data, the third is expiration time (seconds), and the fourth is the byte length of the value.

The overall configuration is simple, introduces minimal overhead, and works reliably; reviewing the Lua script is recommended for production use.