Run Two Environments on One Domain with Nginx: Traffic Splitting, Auth & Cache Isolation

Why Not Just Use Office IP for Test Access

Developers and ops staff often share the same office egress IP. Being in the office does not mean the current request should hit the test environment. A clearer rule: no test marker → production; test marker present → verify IP, then verify test-entry credentials; if any check fails, reject and do not fall back to production.

Two Services, Each with Its Own Dependencies

Before touching the gateway, deploy an independent test service that uses its own database, Redis, file storage, and message queue credentials. "Independent" means test credentials cannot read/write production resources, and network rules prevent the test service from reaching production internal endpoints. Sharing a host is acceptable for cost, but it does not provide host-level isolation.

Redis DB 0 and DB 1 share the same process, memory, and admin permissions; Redis Cluster only supports DB 0. Use a separate Redis instance instead of relying on logical databases.

File storage: separate buckets with distinct credentials. A key that can access the whole bucket can still cause cross-environment writes.

Message queues: separate topics plus producer/consumer permission limits and distinct consumer groups.

External integrations (payments, SMS, email, scheduled jobs, callbacks) must be sandboxed. Test orders should trigger sandbox payments and notification allowlists; scheduled jobs default to off. Callbacks from payment providers lack browser cookies, so provide explicit environment-bound callback endpoints.

Test accounts are sufficient. If real data is needed, export required fields, anonymize, and import into the test database. Do not copy production password hashes, tokens, or customer data.

How Nginx Identifies Test Requests

Assume production listens on 127.0.0.1:8080 and test on 127.0.0.1:8081. Both serve pages, static assets, and /api/ endpoints. The following configuration lives in the http block (e.g., an included site file). It handles four concerns: IP allowlist, explicit test choice via cookie, test-entry gate, and API page-environment consistency.

Nginx's geo module handles CIDR matching; map handles string/regex mapping. Example variables:

# IP allowlist via geo
geo $qa_ip {
  default 0;
  203.0.113.5/32 1;  # replace with actual office/VPN egress
}

# Cookie only expresses intent, not entitlement
map $cookie_env $want_test {
  default 0;
  test 1;
}

# Test-entry gate, default off
map $host $test_enabled {
  default 0;
}

# Strict page-env check for API, default off (migration window)
map $host $strict_page_env {
  default 0;
}

# Combine: want_test=1 but IP not allowed, or gate closed → deny
map "$want_test:$qa_ip:$test_enabled" $test_denied {
  default 0;
  "1:0:1" 1;
  "1:0:0" 1;
  "1:1:0" 1;
}

# Lane selection
map $want_test $lane {
  default prod;
  1 test;
}

# Strip Basic auth header used for test-entry so it doesn't reach the app
map $http_authorization $business_auth {
  default $http_authorization;
  "~*^Basic\s" "";
}

# Detect API requests
map $uri $is_api {
  default 0;
  "~^/api(?:/|$)" 1;
}

# Verify API page-env matches lane
map "$is_api:$http_x_page_env:$lane:$strict_page_env" $wrong_lane {
  default 1;
  "~^0:" 0;
  "~^1:prod:prod:[01]$" 0;
  "~^1:test:test:[01]$" 0;
  "1::prod:0" 0;  # legacy production client migration window
}

If a load balancer or CDN sits in front, use the realip module to restore the true client IP, trust only known proxy ranges, and restrict origin access to those proxies. Do not blindly trust X-Forwarded-For.

Two Proxy Entrypoints

server {
  listen 443 ssl;
  server_name www.example.com;
  ssl_certificate /etc/nginx/ssl/example.com.crt;
  ssl_certificate_key /etc/nginx/ssl/example.com.key;

  # Disable caching for both environments initially
  proxy_cache off;
  proxy_hide_header Cache-Control;
  proxy_hide_header Expires;
  add_header Cache-Control "private, no-store" always;

  # Named location for internal jump, preserves method and body
  error_page 418 = @test;

  location / {
    if ($test_denied) { return 403; }
    if ($wrong_lane) { return 409; }
    if ($want_test) { return 418; }

    proxy_pass http://127.0.0.1:8080;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header If-None-Match "";
    proxy_set_header If-Modified-Since "";
    proxy_set_header X-Env prod;
    proxy_set_header Authorization $business_auth;
  }

  location @test {
    auth_basic "QA environment";
    auth_basic_user_file /etc/nginx/qa.htpasswd;

    add_header Cache-Control "private, no-store" always;
    add_header X-Robots-Tag "noindex, nofollow" always;

    proxy_pass http://127.0.0.1:8081;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header If-None-Match "";
    proxy_set_header If-Modified-Since "";
    proxy_set_header X-Env test;
    # Do not forward the Basic auth credentials to the test app
    proxy_set_header Authorization "";
  }
}

The if statements only execute return; proxy config lives in each location. 418 is an internal signal to jump to @test via named location — no browser redirect, URL unchanged. Both backends receive X-Env set by the gateway; client-sent X-Env: test is ignored. Log $lane, status, and upstream; exclude cookies, auth, and passwords. Backend ports are bound to localhost only to prevent bypass.

Basic Auth Caveats

Basic Auth consumes the Authorization header. If the application uses Bearer tokens, this example cannot be used as-is. Switch to auth_request backed by the team's existing auth service, using a separate entry cookie for test entitlement while preserving the business Authorization header. The auth subrequest must return 2xx to allow, 401/403 to deny; other codes are errors. Ensure the module is compiled in. If the app itself uses Basic Auth, also move to a separate entry auth to avoid credential collision.

Cookie Setup and Frontend Changes

Testers create a dedicated Chrome profile (not just a new tab) to avoid cookie sharing. After opening the HTTPS site, add a cookie env=test with Path /, Secure, SameSite=Lax, no Domain attribute. Refresh, enter the test-entry Basic Auth credentials, then log in with a test business account.

This manual cookie is just a selector; IP and entry auth still enforce access. No automatic expiry: admins remove the user from qa.htpasswd or revoke IP permission when testing ends. Clearing the cookie does not revoke the entry password.

For time-limited invites, session expiry, and centralized revocation, integrate auth_request later. A protected switch page can set the selector cookie server-side.

Critical issue: if the test page is open and the cookie disappears, the gateway would route new requests to production. Therefore, each frontend build embeds a fixed PAGE_ENV value ( prod or test) and all API calls (reads and writes) send it via the X-Page-Env header:

// Injected at build time: 'prod' for production bundle, 'test' for test bundle
const PAGE_ENV = 'test';

fetch('/api/orders', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'X-Page-Env': PAGE_ENV
  },
  body: JSON.stringify({ productId: 'demo' })
});

The page environment comes from the loaded build, not from the env cookie at request time. Otherwise, a cookie change would make an old test page claim to be production, breaking the check.

The wrong_lane map validates the API's X-Page-Env. A test page that loses its cookie will send X-Page-Env: test but be routed to production (lane=prod) → 409. A production page that mistakenly carries the test cookie will be routed to test (lane=test) but send X-Page-Env: prod → 409. Navigation and static assets are not guarded by this header.

During migration, legacy production requests without X-Page-Env are allowed only when strict_page_env=0. The test entry always requires X-Page-Env: test; only new test builds with the header can use it.

Page-environment headers prevent accidental cross-environment calls but do not replace business authentication, authorization, or CSRF protection. Backends still validate their own sessions, permissions, and tokens. GET requests must not perform writes. Other API prefixes, upload paths, or plain form submissions must be brought under the same protection; forms cannot add headers, so either change submission method or add environment-binding validation.

The gateway decides routing; the application only adds the page-environment header, per-environment session config, and error messaging — a more practical change than insisting the app stays completely unaware.

Same Domain, Same Origin in the Browser

Same protocol, host, and port means same origin per MDN. The browser will not automatically give the test environment a separate LocalStorage just because Nginx proxies to a different backend.

Production keeps its existing session cookie name and secret; test uses a distinct name (e.g., test_session), its own session store, and its own signing key. Both set Secure, HttpOnly, and appropriate SameSite. Do not alter production session config just to add test.

If using tokens, signing keys and audience claims must also be environment-specific.

LocalStorage, IndexedDB, and Cache Storage keys must be namespaced per environment; the test frontend must not read production storage keys.

A dedicated browser profile separates cookies, caches, and Service Workers — a prerequisite for this approach.

These are not security boundaries: test code in the same profile can still access same-origin storage. Test builds and their scripts must be trusted. For higher isolation, use a separate test domain instead of forcing same-domain.

Test pages visibly label "Test Environment"; production pages need no badge. Regular visitors notice nothing; testers know where they are.

Cache Control and CDN Awareness

If a CDN serves a cached production homepage, the request never reaches Nginx, rendering downstream splitting useless. The config above disables caching on both sides initially to verify actual routing. This increases origin load; assess capacity before enabling on a live site. Production static assets can be cached later with content-hash URLs or separate build directories, preserving old version references. HTML and dynamic API caching strategies are handled separately. Content hashes differentiate files but cannot prevent whole-page HTML, login responses, or Service Worker cross-use.

Existing browser caches do not vanish just because no-store is added (per MDN). On first test enablement, the QA profile starts clean; if it previously visited the site, clear that profile's cache for the host. Daily production profiles are not cleared.

Both upstreams may receive stale validation headers from the other environment. The example strips If-None-Match and If-Modified-Since on both sides so upstreams send full responses, avoiding incorrect 304s that reuse old bodies. If-Match for concurrency control is preserved. When re-enabling conditional requests later, ensure ETag/Last-Modified accurately distinguish content.

If existing config has dedicated .js / .css locations or static file serving, integrate them into the environment split to avoid a test page loading production scripts.

Service Workers can respond before the request reaches the gateway. If the project uses them, clear old registrations and caches in the QA profile, and disable registration in the test build. A hard refresh does not guarantee offline cache is cleared.

For search engines, add X-Robots-Tag: noindex, nofollow only to test responses. Production responses get no such header, and the shared robots.txt does not block the whole site. Google's documentation notes crawlers must fetch the response to see noindex; it is an indexing directive, not a substitute for entry authentication.

Phased Rollout with Two Switches

Production frontend updates won't magically add new headers to already-open old pages. Rejecting all unmarked requests would break in-flight user submissions. Two switches enable gradual adoption:

Preparation: test_enabled=0 (test entry closed), strict_page_env=0 (legacy production requests work). Deploy dependency isolation, cache policies, and the new production frontend with page-environment headers. Inventory other API callers.

Limited Opening: After cache, auth, and new test build validation, set test_enabled=1. Migration exception remains only for unmarked production requests; explicit cross-environment API calls are still rejected.

Strict Enforcement: Once old pages are updated, clients adapted, and legacy support window closed, set strict_page_env=1. Both production and test APIs now require correct X-Page-Env. Observing logs for a while without legacy requests is insufficient; a dormant page could reopen later.

Public APIs, platform callbacks, and existing mobile clients may not use the page header. They get their own explicit entrypoints, auth, and environment rules — adapt first, then tighten. Both switches require config test and reload after change.

Shutting Down Test Cleanly

If cross-contamination appears, set test_enabled=0, test config, reload. Requests with the test marker get 403; regular visitors still hit production. If the test backend goes down, test requests return a gateway error — they are not forwarded to production.

When the cookie disappears, the old test page still sends X-Page-Env: test; both reads and writes are rejected with 409. To end a test cycle, close the test page in the QA profile; testers return to the daily profile for production. The QA profile remains dedicated to testing.

Page navigation and existing long-lived connections (WebSocket) are not covered by the API header guard. Equating "close test" with "everyone auto-falls back to production" could catch unfinished test operations.

Reloading config does not retract operations already in the backend. WebSocket connections may persist. To stop test execution immediately, halt writes and close connections on the test service side, and drain queued test jobs.

Acceptance Checklist

No test marker → production from both office and external networks.

Allowlisted IP with marker but no entry auth → cannot enter test.

Non-allowlisted IP even with marker and credentials → cannot enter test.

Authorized access → pages, scripts, APIs, uploads all hit test service.

Delete cookie → old test page reads/writes return 409, no production DB writes.

Migration: legacy production requests work; strict phase rejects unmarked requests.

Browser cache and Service Worker handled; test responses not mixed into production cache.

Close entry or stop test service → test requests fail, regular visitors unaffected.

Acceptance also includes inspecting actual cached responses and verifying from the test service (using test credentials) that production dependencies are rejected by permissions and network rules. Absence of test business writes in production logs is a stronger proof than an unchanged address bar.

For small-to-medium teams, the rollout order is: isolate dependencies and permissions, separate sessions and page-environment headers, handle browser and CDN caching, then open the entry to a small group. Enable strict checks only after production clients have migrated.

When service count grows, consider propagating the gateway-confirmed environment tag through service calls and message passing — the "swim lane" pattern. It must cover synchronous requests, async tasks, and background jobs; a single missed tag can cross lanes, and a service mesh alone does not automate this.

One address can serve two environments. The real effort for small teams is ensuring every API read and write has a deterministic destination. When the test entry is denied or the test service fails, the request stops there — failed test operations are never handed to production.