Beyond Reverse Proxy: Production‑Ready Static/Dynamic Separation and Multi‑Level Caching with Nginx

0. Applicability and Goals

Suitable for front‑back separation / BFF scenarios, static assets and APIs can share the same domain or be cross‑domain.

Goal: reduce P99 latency by 5‑10×, cut back‑origin traffic by >80%, halve the number of backend instances.

Constraints: a single machine must handle 50‑100k concurrent long‑lived connections, cache size <50 GB, gray‑release/rollback controllable.

1. Quick Principles – Why Nginx Fits

Event‑driven + non‑blocking I/O : uses epoll and independent worker_processes without locks.

Zero‑copy with sendfile/tcp_nopush : static files are served directly from page cache, minimizing user‑kernel switches.

Built‑in multi‑level cache pipeline : client cache (Cache‑Control/ETag) → proxy cache ( proxy_cache) → upstream cache (Redis/BFF).

Cache coordination mechanisms : proxy_cache_lock merges concurrent requests, use_stale provides fallback, background_update refreshes in the background to avoid cache‑stampede or avalanche.

Layered routing : location matching based on path/type/host, static locations are matched first to keep dynamic paths from polluting static cache.

2. Target Architecture (Horizontal Scaling + Gray Release)

┌──────────────┐
   CDN    │  Edge/POP    │  Optional: image/WebP/video distribution
          └──────┬───────┘
                 │ Origin/Pre‑warm
          ┌─────▼─────┐
          │ Nginx Edge│  Static/Dynamic separation + multi‑level cache + rate‑limit
          └───┬─────┬─┘
              │     │
   Static files (SSD/OSS)   API upstream cluster (SpringBoot/Golang/BFF)
              │     │
          ┌───▼─┐ ┌─▼─────────┐
          │metrics│ │tracing   │ Monitoring / Alerting / Tracing
          └───────┘ └───────────┘

Horizontal scaling : Nginx is stateless; use LVS/SLB or keepalived VRRP for layer‑4 load balancing.

Gray/Blue‑Green : control gray traffic with map $cookie_gray, upstream weight, or split_clients.

Storage strategy : static assets on local SSD or read‑only OSS; cache metadata stored in shared memory via keys_zone.

3. Production‑Grade Configuration (Template)

Key points: modular, layered include , built‑in observability. Split the snippets into conf.d/ files and lint/reload in CI.

3.1 Global Basics (Performance + Security)

user  nginx;
worker_processes  auto;   # auto‑detect CPU cores
worker_cpu_affinity  auto; # bind workers to CPUs
worker_priority  -10;

events {
    worker_connections  65535;
    multi_accept        on;
    use                 epoll;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    server_tokens off;               # hide version

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for" '
                    'cache:$upstream_cache_status rt:$request_time uct:$upstream_connect_time';
    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    tcp_nopush      on;
    tcp_nodelay     on;
    keepalive_timeout 65;
    keepalive_requests 2000;

    gzip            on;
    gzip_comp_level 5;
    gzip_min_length 1k;
    gzip_types      text/plain text/css application/javascript application/json image/svg+xml;

    proxy_cache_path /var/cache/nginx/api levels=1:2 keys_zone=api_cache:300m \
                     max_size=30g inactive=90m use_temp_path=off;

    include conf.d/*.conf;
}

3.2 Static/Dynamic Separation (Path + Type Double Guard)

server {
    listen 80;
    server_name www.example.com;
    root /data/www/dist;      # front‑end build output
    index index.html;

    # Front‑end router: History mode fallback
    location / {
        try_files $uri $uri/ /index.html;
    }

    # Static assets: long cache + immutable
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff2?)$ {
        expires 365d;
        add_header Cache-Control "public, immutable";
        access_log off;
    }

    # Separate static directory (e.g., uploads) via alias
    location /static/ {
        alias /data/www/static/;
        expires 30d;
        add_header Cache-Control "public";
    }
}

3.3 API Proxy + Multi‑Level Cache + Cache‑Penetration Protection

upstream backend {
    server 10.0.0.10:8080 weight=3 max_fails=3 fail_timeout=30s;
    server 10.0.0.11:8080 weight=2 max_fails=3 fail_timeout=30s;
    keepalive 64;   # reuse connections to upstream
}

# Gray traffic example: 10% to gray pool
map $cookie_gray $is_gray {
    default 0;
    gray    1;
}
upstream backend_gray { server 10.0.0.12:8080; }

server {
    listen 80;
    server_name www.example.com;
    root /data/www/dist;
    index index.html;

    location /api/ {
        # Choose gray or main pool
        if ($is_gray) { proxy_pass http://backend_gray; }
        proxy_pass http://backend;

        # Cache key: method + host + URI
        proxy_cache api_cache;
        proxy_cache_key "$request_method:$host$request_uri";

        # Cache validity per status code
        proxy_cache_valid 200 301 302 10m;
        proxy_cache_valid 404 1m;
        proxy_cache_valid any 30s;

        # Background refresh + fallback
        proxy_cache_background_update on;
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;

        # Merge concurrent requests to avoid stampede
        proxy_cache_lock on;
        proxy_cache_lock_age 5s;
        proxy_cache_min_uses 3;

        # Bypass cache for authenticated requests
        proxy_cache_bypass $http_authorization $cookie_session;
        proxy_no_cache $http_authorization $cookie_session;

        # Timeouts and buffering
        proxy_connect_timeout 5s;
        proxy_send_timeout 30s;
        proxy_read_timeout 60s;
        proxy_buffering on;
        proxy_buffer_size 8k;
        proxy_buffers 16 32k;

        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Connection "";

        # Upstream failover and retry
        proxy_next_upstream error timeout http_500 http_502 http_503 http_504;
        proxy_next_upstream_tries 2;

        add_header X-Cache-Status $upstream_cache_status;
    }
}

3.4 Access Control, Rate‑Limiting & Circuit‑Breaker

http {
    # QPS‑based rate limit
    limit_req_zone $binary_remote_addr zone=req_limit:50m rate=20r/s;
    # Concurrent connection limit
    limit_conn_zone $binary_remote_addr zone=conn_limit:50m;
}

server {
    # Global burst protection
    limit_req zone=req_limit burst=60 nodelay;
    limit_conn conn_limit 200;

    # Circuit‑breaker / retry for upstream errors
    location /api/ {
        proxy_next_upstream error timeout http_500 http_502 http_503 http_504;
        proxy_next_upstream_tries 2;
        # other config same as above
    }
}

3.5 Health Check, Monitoring & Debugging

# Lightweight health check
location /healthz {
    access_log off;
    return 200 "ok
";
    add_header Content-Type text/plain;
}

# Observability: cache hit rate, slow requests, 5xx
# Example commands (awk on access log)
# awk -F'cache:' '{print $2}' /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c
# awk '$NF > 1 {print}' /var/log/nginx/access.log | head -20   # rt>1s
# awk '$9 ~ /^5/ {print}' /var/log/nginx/access.log | tail -20   # 5xx

4. Engineering Upgrade (High Concurrency + Scalability)

Configuration layering : global settings in nginx.conf, per‑site configs in conf.d/, shared snippets (gzip, security headers, cache) in snippets/.

Environment separation : inject upstream addresses, weights, rate‑limit thresholds via envsubst or Helm/Ansible templates; manage multiple environments in the same repo.

CI/CD :

Lint: nginx -t or docker run --rm -v $PWD:/etc/nginx:ro nginx:stable nginx -t.

Warm‑up: run cache-warmup.sh (parallel wrk / hey against hot endpoints).

Gray release: use cookie/weight or split_clients, observe P95/P99 and 5xx.

Rollback: keep previous config snapshot, nginx -s reload to revert quickly.

Observability : expose stub_status or status for connection/request metrics; ship access logs to Loki/Elasticsearch and build dashboards for cache hit rate and latency.

Elasticity & Scaling :

Front‑edge CDN handles ~70% of static traffic.

Horizontal Nginx scaling: stateless, share local SSD cache directory or use replicated caches (slightly lower hit rate is acceptable).

High‑concurrency protection :

Cache stampede: proxy_cache_lock + min_uses.

Cache penetration: short cache for 4xx/empty results, whitelist/validate parameters.

Cache avalanche: stagger expiration (e.g., proxy_cache_valid 200 600s) with background refresh.

Slow‑query protection: proxy_read_timeout + limit_req + upstream circuit‑breaker.

5. Production Checklist (Pre‑deployment Verification)

CPU/Connections: worker_processes auto, worker_connections ≥ 65535, keepalive_requests ≥ 1000.

Static/Dynamic Separation: static location matched first; expires + immutable for static, try_files for front‑end routing.

Cache: configure proxy_cache_path with keys_zone/max_size/inactive/use_temp_path=off, tiered status‑code validity, enable background_update, use_stale, cache_lock.

Security: server_tokens off, enforce HTTPS, image hotlink protection, method whitelist, hide backend headers.

Protection: limit_req/limit_conn, proxy_next_upstream, health check and circuit‑breaker.

Observability: log includes cache_status/rt, expose /healthz, integrate metrics.

Gray release: use map or split_clients, clear rollback path.

6. Load‑Test Recommendations and Expected Gains

Tools : wrk -t12 -c400 -d60s http://nginx/api/foo for API, siege/hey for static.

Metrics to watch : RPS, P99 latency, cache HIT/MISS, backend QPS, CPU IOWait.

Typical benefits (based on experience) :

Static/Dynamic separation + browser cache reduces back‑origin static traffic by 80‑95% and cuts front‑end TTFB by 30‑60%.

Proxy cache + background refresh lowers hot‑endpoint P99 from ~800 ms to 80‑150 ms.

Keep‑alive + upstream connection pool reduces backend CPU by 20‑40% and dramatically lowers GC/thread count.

7. Full Example (Ready to Deploy)

Split the following files into three parts: nginx.conf , conf.d/site.conf , snippets/cache.conf . They can also be merged into a single file. After adjusting upstream IPs and domains, run nginx -t && nginx -s reload .

# nginx.conf
user  nginx;
worker_processes  auto;
worker_cpu_affinity  auto;
worker_priority  -10;

events {
    worker_connections 65535;
    multi_accept on;
    use epoll;
}

http {
    include mime.types;
    default_type application/octet-stream;
    server_tokens off;
    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for" '
                    'cache:$upstream_cache_status rt:$request_time uct:$upstream_connect_time';
    access_log /var/log/nginx/access.log main;
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    keepalive_requests 2000;
    gzip on;
    gzip_comp_level 5;
    gzip_min_length 1k;
    gzip_types text/plain text/css application/javascript application/json image/svg+xml;
    proxy_cache_path /var/cache/nginx/api levels=1:2 keys_zone=api_cache:300m \
                     max_size=30g inactive=90m use_temp_path=off;
    include conf.d/*.conf;
}

# conf.d/site.conf
map $cookie_gray $is_gray {
    default 0;
    gray    1;
}

upstream backend {
    server 10.0.0.10:8080 weight=3 max_fails=3 fail_timeout=30s;
    server 10.0.0.11:8080 weight=2 max_fails=3 fail_timeout=30s;
    keepalive 64;
}
upstream backend_gray { server 10.0.0.12:8080; }

server {
    listen 80;
    server_name www.example.com;
    root /data/www/dist;
    index index.html;

    # Health check
    location /healthz { access_log off; return 200 "ok
"; add_header Content-Type text/plain; }

    # Static/Dynamic separation
    location / {
        try_files $uri $uri/ /index.html;
    }
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff2?)$ {
        expires 365d;
        add_header Cache-Control "public, immutable";
        access_log off;
    }
    location /static/ {
        alias /data/www/static/;
        expires 30d;
        add_header Cache-Control "public";
    }

    # API proxy + cache
    location /api/ {
        if ($is_gray) { proxy_pass http://backend_gray; }
        proxy_pass http://backend;
        proxy_cache api_cache;
        proxy_cache_key "$request_method:$host$request_uri";
        proxy_cache_valid 200 301 302 10m;
        proxy_cache_valid 404 1m;
        proxy_cache_valid any 30s;
        proxy_cache_background_update on;
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
        proxy_cache_lock on;
        proxy_cache_lock_age 5s;
        proxy_cache_min_uses 3;
        proxy_cache_bypass $http_authorization $cookie_session;
        proxy_no_cache $http_authorization $cookie_session;
        proxy_connect_timeout 5s;
        proxy_send_timeout 30s;
        proxy_read_timeout 60s;
        proxy_buffering on;
        proxy_buffer_size 8k;
        proxy_buffers 16 32k;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Connection "";
        proxy_next_upstream error timeout http_500 http_502 http_503 http_504;
        proxy_next_upstream_tries 2;
        add_header X-Cache-Status $upstream_cache_status;
    }

    # Rate limiting / protection
    limit_req zone=req_limit burst=60 nodelay;
    limit_conn conn_limit 200;
    error_page 500 502 503 504 /50x.html;
    location = /50x.html { root /usr/share/nginx/html; }
}

# snippets/limits.conf (optional include)
limit_req_zone $binary_remote_addr zone=req_limit:50m rate=20r/s;
limit_conn_zone $binary_remote_addr zone=conn_limit:50m;

8. Conclusion – Three Rules to Turn Nginx into a Traffic Accelerator

Divide and Conquer : match static paths/types first, route APIs separately with dedicated cache keys.

Cache First : cache everything possible, use lock, background refresh and fallback strategies to survive high concurrency.

Engineering Safeguards : gray releases for rollout, circuit‑breakers for failures, observability for data, CI validation for configuration.

Deploy this template, fine‑tune parameters to your traffic volume, and your Nginx instance will reliably handle peaks, reduce backend load, roll back quickly, and save substantial compute costs.