How to Achieve Smooth Releases and AB Testing with Nginx: A Step‑by‑Step Guide

Smooth Release Overview

Background

The cloud‑office system originally had no graceful deployment mechanism. Updating DLLs or configuration files caused the IIS site to restart, leading to brief outages that generated complaints from over 10,000 active users.

Implementation

Define a health‑check endpoint on the Nginx reverse‑proxy.

The endpoint returns an HTTP status code that Nginx uses to decide whether to route traffic to the instance.

If the health‑check returns a 5xx status, Nginx removes the instance from the upstream pool, preventing new requests from reaching it.

Release Procedure

Open the Nginx management page (e.g., /publish/ngxconfig).

Take a specific instance (e.g., instance A) offline via the UI.

Confirm the health‑check now returns 401 (offline) instead of 200 (online).

Monitor the monitoring dashboard until the instance’s request and connection counters drop to zero.

Deploy the new version to the offline instance.

Use a tool such as Fiddler to verify that the instance serves traffic without triggering an IIS restart.

QA validates normal operation, then repeat the steps for the next instance until all instances are updated.

AB Testing Optimizations

Background

After achieving smooth releases, the team needed a way to perform gray‑scale (AB) testing for major feature rollouts, allowing a limited user group to try new functionality before a full production deployment.

1. IP‑Based Routing (Nginx Reverse Proxy)

Requests are routed based on the client’s source IP. An IP whitelist maps IPs to group A or group B. Traffic from group A is forwarded to a dedicated cluster; all other traffic goes to the default cloud‑office cluster.

server {
    listen 80;
    server_name officecloud.com;
    access_log officecloud.com/logs main;
    ip_list 192.168.254.4, 192.168.254.170;
    set $group default;
    if ($remote_addr in iplist) {
        set $group ACluster;
    }
    location / {
        proxy_pass http://$group;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        index index.html index.htm;
    }
}

Simple configuration; works well for internal static IPs.

Not suitable for external users with dynamic IPs.

2. Cookie‑Based Routing (Nginx Reverse Proxy)

The client’s HTTP cookie (or any other request key) carries a version identifier. Nginx inspects the cookie and routes traffic to the corresponding cluster (e.g., version=V1.1 → ACluster, version=V1.0 → default).

server {
    listen 80;
    server_name officecloud.com;
    access_log officecloud.com/logs main;
    ip_list 192.168.254.4, 192.168.254.170;
    set $group default;
    if ($http_cookie ~* "version=V1.0") {
        set $group default;
    }
    if ($http_cookie ~* "version=V1.1") {
        set $group ACluster;
    }
    location / {
        proxy_pass http://$group;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        index index.html index.htm;
    }
}

Simple configuration; uses the $COOKIE_version variable for routing.

Relatively stable; users can be placed in a test group by setting a specific cookie value.

First‑time visits to static pages may not have the cookie, requiring fallback handling.

3. AB Cluster + Business Proxy Method

During authentication, a routing module checks the user’s department and role against an AB list and forwards the request to the appropriate AB cluster. Each AB cluster can have a distinct domain name for easier identification.

Decoupled from the generic Nginx platform; does not rely on a shared reverse‑proxy implementation.

Requires provisioning of separate AB clusters with unique domain names.

In front‑end/back‑end separation scenarios, both static and service sites must be covered by AB clusters.

All entry points need a unified proxy layer, adding development effort.

Reference

GitHub repository: https://github.com/CNSRE/ABTestingGateway – an open‑source dynamic routing system built on Nginx and ngx‑lua, using Redis for routing policies.