Ultimate Nginx Config: Load Balancing, Rate Limiting, Reverse Proxy, SSL & Cloud‑Native

Basic Configuration

Global Settings

user  nginx nginx;  # run as nginx user
worker_processes  auto;  # match CPU cores
worker_cpu_affinity  auto;  # bind workers to CPUs
worker_priority -5;  # raise process priority (requires root)
worker_rlimit_nofile 65535;  # increase file descriptor limit
pid  /var/run/nginx.pid;  # pid file path

Key notes worker_cpu_affinity on a 4‑core server:

worker_cpu_affinity 0001 0010 0100 1000;

worker_priority

range is -20 (highest) to 19 (lowest); adjust with caution.

Events Module Tuning

events {
    worker_connections  16384;  # max connections per worker
    use epoll;               # efficient Linux event model
    multi_accept on;         # accept all new connections at once
    accept_mutex on;         # prevent thundering herd (default on)
    accept_mutex_delay 50ms;# wait time when lock acquisition fails
}

Performance impact multi_accept on can boost QPS by 10%‑15% in high‑concurrency scenarios. accept_mutex_delay should be tuned per load (e.g., 0ms for low load).

Core Features: Reverse Proxy & Load Balancing

Single‑Node Reverse Proxy

server {
    listen 80;
    server_name api.example.com;

    location /testapi/ {
        proxy_pass http://178.168.1.10:9120/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        client_max_body_size 8M;
        proxy_connect_timeout 10s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
    }
}

Production optimization

Add health‑check endpoint:

location /healthz {
    access_log off;
    return 200 "OK";
}

Load Balancing with Active Health Checks

upstream openserver-api {
    zone backend 64k;               # shared memory for state sync
    least_conn;                     # least‑connection strategy
    server 127.0.0.1:8900 weight=100 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:8901 weight=100 max_fails=3 fail_timeout=30s;

    # active health check (requires nginx_upstream_check_module)
    check interval=3000 rise=2 fall=5 timeout=1000 type=http;
    check_http_send "HEAD /healthz HTTP/1.0

";
    check_http_expect_alive http_2xx http_3xx;
}

server {
    location /test-api/ {
        proxy_pass http://openserver-api;
        proxy_next_upstream error timeout http_500 http_502 http_503 http_504;
    }
}

Strategy comparison

Round Robin – applicable to stateless services; drawback: ignores server load.

Least Connections – suitable for long‑lived connections (e.g., WebSocket); drawback: requires state synchronization.

IP Hash – used for session‑sticky scenarios; drawback: potential imbalance.

Security Hardening (Zero‑Trust Practices)

Transport‑Layer Security

server {
    listen 443 ssl http2;
    server_name secure.example.com;

    ssl_certificate /etc/ssl/certs/fullchain.pem;
    ssl_certificate_key /etc/ssl/private/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers 'TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256';
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 1d;
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_compress off;  # disable SSL compression (CRIME mitigation)
}

Certificate management

Use Let’s Encrypt with Certbot for free auto‑renewal.

Verify certificate dates periodically:

echo | openssl s_client -servername secure.example.com -connect secure.example.com:443 2>/dev/null | openssl x509 -noout -dates

Protocol‑Level Protections

server {
    add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always;
    add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline' https://trusted.cdn.com";
    add_header X-Frame-Options "DENY";
    add_header X-Content-Type-Options "nosniff";
    add_header X-XSS-Protection "1; mode=block";
}

CSP can be generated with Mozilla Observatory.

Access Control

# IP whitelist (CIDR support)
geo $allowed_ip {
    default deny;
    192.168.1.0/24 allow;
    203.0.113.0/24 allow;
}

server {
    location /admin/ {
        if ($allowed_ip = deny) { return 403; }
        auth_basic "Admin Area";
        auth_basic_user_file /etc/nginx/.htpasswd;
    }
}

Dynamic IP blocking with fail2ban

# /etc/fail2ban/jail.d/nginx.conf
[nginx-http-auth]
enabled = true
port = http,https
filter = nginx-http-auth
action = iptables-multiport[name=nginx-http-auth, port="http,https", protocol=tcp]
logpath = /var/log/nginx/error.log
maxretry = 3

Performance Optimization: From Milliseconds to Microseconds

Static Resource Acceleration

server {
    location /static/ {
        alias /var/www/static/;
        expires 1y;                     # long‑term cache
        add_header Cache-Control "public, no-transform";
        gzip_static on;                 # serve pre‑compressed files
        etag off;                       # disable ETag (choose one with Last‑Modified)
    }
}

Browser cache policies

HTML – no-cache, must-revalidate CSS/JS – public, max-age=31536000 Fonts – public, max-age=31536000 API responses –

private, max-age=0

Dynamic Content Caching (OpenResty)

# Multi‑level cache with Lua
location /api/data {
    set $cache_key "$host$request_uri$cookie_user_id";
    proxy_cache_key $cache_key;
    proxy_cache my_cache;
    proxy_cache_valid 200 302 10m;
    proxy_cache_use_stale error timeout updating http_500;

    access_by_lua_block {
        local cache_key = ngx.var.cache_key
        local res = ngx.location.capture("/cache_check", { args = { key = cache_key } })
        if res.status == 200 then
            ngx.exit(ngx.HTTP_OK)
        end
    }
}

Cache design

Hot data – short TTL (1‑5 min), frequent updates.

Cold data – long TTL (≥24 h).

Sensitive data – no cache or signed verification.

Connection‑Pool Tuning

upstream backend {
    server 127.0.0.1:8080;
    keepalive 32;  # number of persistent connections
}

server {
    location / {
        proxy_http_version 1.1;
        proxy_set_header Connection "";  # disable short‑lived connections
    }
}

TCP kernel parameters ( /etc/sysctl.conf )

net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_keepalive_intvl = 15
net.core.somaxconn = 4096
net.ipv4.tcp_max_syn_backlog = 4096

High‑Availability Architecture

Master‑Slave Configuration Sync

# Master configuration
http {
    upstream backend {
        server master.example.com:8080;
        server slave.example.com:8080 backup;
    }
}

# Slave sync via rsync every 5 minutes
*/5 * * * * /usr/bin/rsync -avz /etc/nginx/ [email protected]:/etc/nginx/

Automation options: ansible or puppet for config management; Git post‑receive hook for zero‑downtime deployment:

# .git/hooks/post-receive
#!/bin/bash
TARGET="/etc/nginx"
GIT_DIR="/var/repo/nginx.git"
BRANCH="master"
while read oldrev newrev ref; do
  if [[ $ref = refs/heads/$BRANCH ]]; then
    echo "Ref $ref received. Deploying $BRANCH branch to production..."
    git --work-tree=$TARGET --git-dir=$GIT_DIR checkout -f $BRANCH
    nginx -t && systemctl reload nginx
  else
    echo "Ref $ref received. Doing nothing: only $BRANCH may be deployed."
  fi
done

Layer‑4 Load Balancing (TCP/UDP)

stream {
    upstream db_backend {
        server 192.168.1.10:3306;
        server 192.168.1.11:3306;
    }

    server {
        listen 3306;
        proxy_pass db_backend;
        proxy_connect_timeout 1s;
        proxy_timeout 3s;
    }
}

Applicable to MySQL/Redis clusters, game server load balancing, custom VPN access.

Dynamic DNS Resolution

resolver 8.8.8.8 114.114.114.114 valid=30s;

upstream dynamic_backend {
    server backend.example.com:8080 resolve;
}

server {
    location / { proxy_pass http://dynamic_backend; }
}

Monitor DNS changes:

watch -n 1 "dig +short backend.example.com"

Monitoring & Fault Diagnosis

Real‑Time Metrics Exposure

# Using nginx-module-vts
http {
    vhost_traffic_status_zone;
    server {
        listen 8080;
        location /status {
            vhost_traffic_status_display;
            vhost_traffic_status_display_format prometheus;
        }
    }
}

Grafana dashboards import metrics nginx_http_requests_total, nginx_connections_active, nginx_upstream_responses_total.

Dynamic Tracing (OpenTelemetry)

load_module modules/ngx_http_opentelemetry_module.so;

http {
    opentelemetry_config /etc/nginx/otel.conf;
    opentelemetry_tracer "nginx-exporter";
    server {
        location / {
            opentelemetry_propagate_context on;
            opentelemetry_attribute "http.method=$request_method";
        }
    }
}

Query traces in Jaeger:

curl -s "http://jaeger:16686/api/traces?service=nginx" | jq .

Diagnostic Toolkit

nginx -T

– test configuration and dump full config<br>

nginx -T -c /etc/nginx/nginx.conf

strace

– trace system calls<br>

strace -p $(cat /var/run/nginx.pid)

tcpdump

– packet capture<br>

tcpdump -i eth0 port 80 -w nginx.pcap

slowlog

– record slow requests (requires compiled module)<br>

slowlog_file /var/log/nginx/slow.log; slowlog_threshold 5s;

Cloud‑Native Adaptation

Nginx Ingress Controller (Kubernetes)

# ingress-nginx annotation example
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: "/"
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "20"
    nginx.ingress.kubernetes.io/configuration-snippet: |
      add_header X-Robots-Tag "noindex";
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: "/"
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

Canary traffic is controlled via canary-weight and can be auto‑adjusted based on Prometheus error‑rate metrics.

Service Mesh (Istio) Integration

# Nginx behind Istio sidecar
server {
    location / {
        proxy_pass http://127.0.0.1:15001;  # Envoy port
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Traffic mirroring example (VirtualService):

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: mirror-example
spec:
  hosts:
  - web-service
  http:
  - route:
    - destination:
        host: web-service
        subset: v1
      weight: 90
    mirror:
      host: web-service
      subset: v2
    mirror_percentage:
      value: 10.0

Best Practices

Configuration Management

Version control with Git.

CI/CD pipelines for automated testing and deployment.

Critical configuration changes require dual‑person review.

Performance Benchmarking

Use wrk or locust for load testing.

Target P99 latency <200 ms and error rate <0.1 %.

Security Auditing

Monthly check for SSL module: nginx -V 2>&1 | grep -o with-http_ssl_module.

Monitor Nginx security advisories.

Disaster Recovery

Deploy across multiple availability zones.

Automated rollback mechanisms.

Regular failover drills.

Commonly Used Modules

nginx-module-vts

– real‑time traffic monitoring; install with ./configure --add-module. nginx-opentelemetry – distributed tracing; compile and install. nginx-wasm-module – WebAssembly runtime; experimental module. nginx-upstream-check – active health checks; third‑party module. nginx-lua-module – dynamic scripting (OpenResty core); compile and install.