Practical Guide to Optimizing Nginx for High Concurrency and Troubleshooting

Introduction: Why Many Teams "Use Nginx" but Can't Handle High Concurrency

Nginx is the default entry point for most internet systems—reverse proxy, load balancing, static file serving, TLS termination, rate limiting, gray releases, and API‑gateway capabilities—all built on top of it. The problem is often not a lack of configuration knowledge, but a failure to understand Nginx as part of the whole system.

In high‑concurrency scenarios, Nginx faces a chain of issues:

When traffic spikes, is the bottleneck the NIC, kernel backlog, Nginx worker, upstream connection pool, or the backend thread pool?

502, 504, 499, connection saturation, TIME_WAIT explosion—who fails first?

Why does increasing worker_connections sometimes not improve throughput and even make the system less stable?

In container and Kubernetes environments, why does a seemingly correct configuration still cause rolling‑release jitter, probe kills, or connection‑reuse failures?

This article does not merely show "how to set parameters"; it provides a production‑grade Nginx high‑concurrency optimization method from the perspective of architects and frontline operators:

Explain the principles and identify which layer each parameter affects.

Offer a scalable engineering architecture.

Provide production‑ready configuration templates, not just snippets.

Establish a reusable troubleshooting method that can locate 80% of online issues quickly.

1. Establish the Correct Mindset: Nginx Optimization Is Not Just "Tweak a Few Parameters"

Many teams' first reaction when optimizing Nginx is to set:

worker_processes auto;

worker_connections 65535;

keepalive_timeout 120;

gzip on;

These may be correct actions, but they are not the starting point for optimization.

Nginx sits at the intersection of four layers:

Network layer: TCP three‑way handshake, SYN backlog, accept queue, TIME_WAIT, retransmission.

System layer: file descriptors, page cache, CPU scheduling, NIC interrupts, epoll.

Proxy layer: connection reuse, buffers, timeouts, retries, upstream scheduling.

Application layer: backend thread pools, slow DB queries, cache stampede, idempotent APIs.

The effective optimization order should be:

Define traffic model → Estimate capacity → Configure Nginx → Tune kernel parameters → Validate with load test → Set up monitoring → Conduct failure drills

Without a capacity model and a validation loop, any "optimization" is just guesswork.

2. Understand the Essence of Nginx High Concurrency

2.1 Master‑Worker Multi‑Process Model

Nginx uses a classic master‑worker multi‑process architecture:

+----------------------+
|   Master Process     |
| Reads config / manages signals |
+----------+-----------+
           |
+---------------------+---------------------+
|                     |                     |
| +------+-------+   +------+-------+   +------+-------+
| | Worker 1    |   | Worker 2    |   | Worker N    |
| | epoll loop |   | epoll loop |   | epoll loop |
| +--------------+   +--------------+   +--------------+

Key advantages:

Master only manages, does not handle traffic.

Workers are isolated, avoiding massive thread‑lock contention.

Each worker runs an event loop using epoll to handle many concurrent connections.

Graceful reload achieves near‑zero interruption.

This is the biggest difference from the traditional "one thread per connection" model: Nginx excels at handling a large number of short‑lived, I/O‑intensive requests.

2.2 Why epoll Supports High Concurrency

On Linux, Nginx typically uses epoll. The traditional blocking model requires a thread per connection, wasting CPU on context switches. epoll only monitors connections that are ready, allowing a single worker to process readable/writable events without allocating a dedicated thread per connection.

In other words, Nginx does not process requests faster; it manages more connections with lower scheduling cost.

2.3 What Happens to a Request Inside Nginx

A typical HTTPS reverse‑proxy request follows this path:

Client
 → TCP handshake
 → TLS handshake
 → Nginx reads request headers/body
 → rewrite / access / content / filter phases
 → selects upstream
 → establishes or reuses connection to upstream
 → reads upstream response
 → writes back to client
 → keep‑alive or close connection

Any failure in these stages can manifest as "slow API" or "502/504" errors.

TCP accept queue full → connection delay or failure.

Heavy TLS handshake → high CPU usage.

Insufficient upstream connection pool → frequent new connections.

Slow backend response → Nginx accumulates Writing/Waiting connections.

Client disconnects early → many 499 responses.

Thus, Nginx tuning must match capacity at each layer, not just increase a single parameter.

3. Capacity Design First: Calculations Required for High Concurrency

3.1 Maximum Connections Are Not Equal to Maximum Throughput

Many articles state:

max_connections = worker_processes × worker_connections

This is a theoretical upper bound, not a production‑usable value.

In a reverse‑proxy scenario, a request usually occupies two connections:

Client → Nginx

Nginx → upstream

A more realistic estimate is:

Concurrent connections ≈ worker_processes × worker_connections / 2

If long‑lived connections (keep‑alive, WebSocket, HTTP/2 multiplexing, large file transfers) are enabled, connection residence time increases and actual throughput is further limited.

3.2 Capacity Planning Should Look at Three Metrics

To judge whether Nginx can still handle load, you must monitor not only QPS but also:

Connections: Active / Reading / Writing / Waiting Latency:

request_time / upstream_response_time / upstream_connect_time

Resources: CPU, memory, file descriptors, NIC, SYN backlog, TIME_WAIT

An empirical formula:

Peak concurrent connections = Peak QPS × Average request residence time

Example: Peak QPS = 20,000, average request time = 120 ms → concurrent connections ≈ 2,400. For long‑polling, uploads, or occasional spikes, reserve a safety margin of 3‑5×.

3.3 Production‑Level Capacity Budget Example

Assume a 16‑core, 32 GB Nginx gateway machine with the following targets:

Peak 30,000 QPS

P99 latency < 80 ms

Reverse proxy to 6 Java service instances

30% TLS traffic

First‑round budget: worker_processes: 16 worker_connections: 16384

Theoretical connections: 262,144

Conservative proxy half: 131,072

Further checks are required:

Is ulimit -n sufficient?

Is fs.file-max enough?

Can the kernel listen/backlog queue survive the traffic burst?

Does TLS handshake become the real CPU bottleneck?

Can the backend connection pool keep up with Nginx reuse?

These checks illustrate why high‑concurrency optimization must involve both kernel and application layers.

4. Production‑Ready Base Configuration: From "Can Run" to "Can Run Stably"

The following is a production‑grade nginx.conf template. The focus is on understanding the trade‑offs behind each parameter, not merely copying.

4.1 Production Template

user  nginx;
worker_processes  auto;
worker_rlimit_nofile 200000;

error_log  /var/log/nginx/error.log  warn;
pid        /var/run/nginx.pid;

events {
    use  epoll;
    worker_connections 16384;
    multi_accept on;
    accept_mutex off;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    server_tokens off;
    charset       utf-8;
    sendfile      on;
    tcp_nopush    on;
    tcp_nodelay   on;
    aio           threads;
    directio      8m;

    keepalive_timeout 65s;
    keepalive_requests 10000;
    reset_timedout_connection on;
    client_header_timeout 10s;
    client_body_timeout   15s;
    send_timeout          15s;

    client_max_body_size 20m;
    client_body_buffer_size 256k;
    large_client_header_buffers 4 16k;

    types_hash_max_size 4096;
    server_names_hash_bucket_size 128;

    log_format main escape=json '{'
        "time":"$time_iso8601",
        "remote_addr":"$remote_addr",
        "request_id":"$request_id",
        "host":"$host",
        "method":"$request_method",
        "uri":"$uri",
        "status":$status,
        "body_bytes_sent":$body_bytes_sent,
        "request_time":$request_time,
        "upstream_addr":"$upstream_addr",
        "upstream_status":"$upstream_status",
        "upstream_connect_time":"$upstream_connect_time",
        "upstream_header_time":"$upstream_header_time",
        "upstream_response_time":"$upstream_response_time",
        "http_referer":"$http_referer",
        "http_user_agent":"$http_user_agent",
        "xff":"$http_x_forwarded_for"
    }';

    access_log  /var/log/nginx/access.log main buffer=256k flush=1s;

    open_file_cache max=200000 inactive=30s;
    open_file_cache_valid 60s;
    open_file_cache_min_uses 2;
    open_file_cache_errors on;

    gzip  on;
    gzip_comp_level 5;
    gzip_min_length 1k;
    gzip_vary on;
    gzip_proxied any;
    gzip_types text/plain text/css application/json application/javascript application/xml text/xml image/svg+xml;

    upstream user_service {
        least_conn;
        server 10.0.1.11:8080 max_fails=3 fail_timeout=10s max_conns=2000;
        server 10.0.1.12:8080 max_fails=3 fail_timeout=10s max_conns=2000;
        server 10.0.1.13:8080 max_fails=3 fail_timeout=10s max_conns=2000;
        keepalive 256;
    }

    server {
        listen 80 reuseport;
        server_name api.example.com;

        location /healthz {
            access_log off;
            return 200 'ok';
            add_header Content-Type text/plain;
        }

        location /api/ {
            proxy_pass http://user_service;
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            proxy_set_header Host $host;
            proxy_set_header X-Request-Id $request_id;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;

            proxy_connect_timeout 1s;
            proxy_send_timeout    10s;
            proxy_read_timeout    10s;

            proxy_buffering on;
            proxy_buffer_size 8k;
            proxy_buffers 16 16k;
            proxy_busy_buffers_size 64k;
            proxy_temp_file_write_size 128k;

            proxy_next_upstream error timeout http_502 http_503 http_504;
            proxy_next_upstream_tries 2;
            proxy_next_upstream_timeout 3s;
        }

        location /static/ {
            root /data/www;
            expires 30d;
            add_header Cache-Control "public, immutable";
        }
    }
}

4.2 Why This Configuration Is More Suitable for Production

worker_rlimit_nofile

must be coordinated with the system ulimit -n; otherwise increasing worker_connections is useless. accept_mutex off + listen reuseport is usually better for multi‑core Linux, reducing single‑point accept contention. keepalive_requests 10000 significantly reduces the cost of repeatedly establishing connections. proxy_set_header Connection "" is key for upstream keep‑alive to work.

JSON log format facilitates unified analysis in ELK, Loki, ClickHouse, etc.

4.3 Parameters Frequently Misused

worker_connections

It limits the maximum connections per worker, not the overall throughput. Blindly increasing it may cause:

File descriptor exhaustion.

Higher memory consumption.

Upstream pressure increase.

keepalive_timeout

Longer is not always better. Excessive timeout holds idle connections; too short loses the benefit of connection reuse. Public and internal services should have different policies.

proxy_read_timeout

Many teams set 30 s, 60 s, or even 300 s as a "safety value", which makes slow backend requests occupy the gateway for too long and amplify snowball effects. A more reasonable approach is to configure per‑API SLA‑based timeouts.

5. Nginx Cannot Work Alone: High‑Concurrency Requires Kernel Tuning

5.1 Linux Kernel Tuning Recommendations

# /etc/sysctl.d/99-nginx.conf
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 65535
net.ipv4.tcp_max_syn_backlog = 65535

net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 5

net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_syncookies = 1

fs.file-max = 1000000
fs.nr_open = 1000000
vm.max_map_count = 262144

Apply with:

sysctl --system

5.2 Why These Parameters Matter

somaxconn

sets the upper limit of the listen socket's accept queue. If listen backlog in Nginx does not match the kernel value, connections may be dropped during spikes. tcp_max_syn_backlog controls the half‑open (SYN) queue, directly affecting connection‑establishment capacity during flash‑sales or seckill events. ip_local_port_range limits the number of local ports. If Nginx initiates many outbound connections (e.g., many upstreams) and the range is too small, local ports can be exhausted. tcp_tw_reuse helps short‑lived connections but is not a universal fix for TIME_WAIT problems; the real issue often lies in upstream connection reuse.

5.3 Additional System Items to Watch

ulimit -n

systemd service LimitNOFILE NIC RSS/RPS configuration

Container runtime fd limits

Cloud provider SLB/NLB connection and bandwidth limits

If any of these do not match, Nginx configuration improvements are only partial.

6. Core Reverse‑Proxy Optimizations: Not Just "Can Forward", but "Can Reuse Connections Stably"

6.1 Upstream Connection Pools Are Critical

In high‑concurrency scenarios, most performance loss occurs not in business logic but in:

Frequent TCP connection creation.

Repeated TLS handshakes.

Backend accept‑queue jitter.

Thread‑pool spikes caused by sudden bursts.

Without upstream keepalive, each request must create a new connection, dragging down both Nginx and the backend.

Production recommendation:

upstream order_service {
    least_conn;
    server 10.0.2.21:8080 max_conns=1000;
    server 10.0.2.22:8080 max_conns=1000;
    keepalive 512;
}

location /orders/ {
    proxy_pass http://order_service;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
}

Key points: keepalive is the number of idle connections cached per worker, not the total cluster capacity.

Backend services must also support keep‑alive; otherwise Nginx cannot reuse connections.

6.2 Upstream Scheduling Strategy

round_robin

: default, simple and stable. least_conn: prefers the server with the fewest active connections; suitable when request latency varies widely. ip_hash: provides session stickiness; higher scaling cost.

Consistent hashing: good for cache‑hit or state‑binding scenarios.

Architectural advice:

Stateless services → least_conn or default round‑robin.

Services with local cache hits → consider consistent hashing.

Session stickiness should be moved to Redis/Token rather than relying on ip_hash long‑term.

6.3 Retry Must Be Cautious

Many incidents are not caused by "no retry" but by overly aggressive retries.

Example of a problematic retry chain:

Backend instance slows down.

Nginx times out and retries.

Original request is still pending; the retry hits another instance.

Backend load spikes, leading to a full‑site snowball.

Correct practice:

Enable retry only for clearly idempotent APIs.

Control tries and total retry time.

Set timeouts to fail fast rather than occupy connections for a long time.

Example:

location /query/ {
    proxy_pass http://query_service;
    proxy_next_upstream error timeout http_502 http_503 http_504;
    proxy_next_upstream_tries 2;
    proxy_next_upstream_timeout 2s;
}

7. Static Resources, Caching, and Compression: Low‑Cost Wins for High Throughput

7.1 Static Resource Caching

location ~* \.(js|css|png|jpe?g|gif|svg|woff2?)$ {
    root /data/www;
    access_log off;
    expires 30d;
    add_header Cache-Control "public, immutable";
}

Suitable for versioned assets, e.g., app.3f1c4d.js or vendor.a81bb2.css. If filenames lack a hash, avoid the immutable directive.

7.2 gzip Benefits and Limits

Compression can dramatically reduce bandwidth but consumes CPU. It is worthwhile for text responses, but offers little benefit for already compressed binaries, images, or videos.

Enable gzip for text APIs.

Be cautious with large file downloads.

If CPU is saturated, prefer pre‑compressed static files or offload to CDN.

7.3 When Proxy Cache Is Appropriate

proxy_cache

is effective for:

Hot product detail pages.

Public configuration queries.

News article details.

Read‑heavy, write‑light aggregation APIs.

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=api_cache:256m max_size=20g inactive=10m use_temp_path=off;

map $request_method $cache_bypass {
    default 1;
    GET 0;
    HEAD 0;
}

server {
    location /api/catalog/ {
        proxy_cache api_cache;
        proxy_cache_key "$scheme$proxy_host$request_uri";
        proxy_cache_valid 200 1m;
        proxy_cache_valid 404 10s;
        proxy_no_cache $cache_bypass;
        proxy_cache_bypass $cache_bypass;
        add_header X-Cache-Status $upstream_cache_status always;
        proxy_pass http://catalog_service;
    }
}

Before deployment, clarify:

How long data consistency can tolerate delay.

Whether cache invalidation will cause a sudden origin‑pull surge.

8. Containerization and Kubernetes: Real Challenges for Nginx in Cloud‑Native Environments

8.1 Why Containerized Nginx Appears "Normal" but Actually Jitters

CPU limits cause throttling.

Memory limits trigger OOM‑kill of workers.

Rolling updates cause uneven connection migration.

Improper probe configuration leads to premature pod removal or accidental kills.

ConfigMap updates cause configuration drift that is hard to detect.

8.2 Production‑Grade Dockerfile

FROM nginx:1.27-alpine
RUN apk add --no-cache curl tzdata
COPY nginx.conf /etc/nginx/nginx.conf
COPY conf.d/ /etc/nginx/conf.d/
RUN mkdir -p /var/cache/nginx /var/log/nginx /var/run \
    && chown -R nginx:nginx /var/cache/nginx /var/log/nginx /var/run
USER nginx
EXPOSE 8080
HEALTHCHECK --interval=10s --timeout=3s --start-period=10s --retries=3 \
    CMD curl -sf http://127.0.0.1:8080/healthz || exit 1
CMD ["nginx", "-g", "daemon off;"]

Production points:

Run as a non‑root user.

Health check must verify real readiness, not just process existence.

Prefer stdout/stderr logging or centralized collection; avoid unlimited file writes inside the container.

8.3 Kubernetes Deployment Recommendations

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-gateway
  namespace: prod
spec:
  replicas: 4
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app: nginx-gateway
  template:
    metadata:
      labels:
        app: nginx-gateway
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: nginx
        image: registry.example.com/nginx-gateway:1.0.0
        ports:
        - containerPort: 8080
        resources:
          requests:
            cpu: "500m"
            memory: "512Mi"
          limits:
            cpu: "2"
            memory: "2Gi"
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sh", "-c", "nginx -s quit; sleep 15"]
        readinessProbe:
          httpGet:
            path: /healthz
            port: 8080
          periodSeconds: 5
          timeoutSeconds: 2
          failureThreshold: 3
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
          periodSeconds: 10
          timeoutSeconds: 2
          failureThreshold: 3

Key points: maxUnavailable: 0 reduces impact of rolling upgrades on entry capacity. preStop + nginx -s quit enables graceful connection draining. terminationGracePeriodSeconds must be sufficient; otherwise pods are killed before connections finish.

8.4 Ingress Controllers Are Not a Universal Gateway

Many teams shove all traffic governance into Ingress annotations, but for complex scenarios you should consider a dedicated gateway layer:

Complex routing orchestration.

Dynamic service discovery and Lua/plugin extensions.

Fine‑grained gray releases and canary deployments.

Security authentication, traffic audit, WAF capabilities.

Multi‑cluster, multi‑region unified entry.

Recommended pattern:

SLB / NLB → Independent Nginx / APISIX / Envoy gateway → Kubernetes Service

9. Scalable Architecture Design: From Single‑Node Nginx to Enterprise‑Grade Entry

9.1 Evolution Roadmap

Stage 1: Single‑node Nginx
Stage 2: Dual‑node active‑passive + Keepalived
Stage 3: L4 LB + Nginx cluster
Stage 4: Containerized Ingress / Gateway
Stage 5: Multi‑region traffic scheduling + gray release + full‑link observability

9.2 Recommended Enterprise Entry Architecture

+----------------------+
                |   DNS / GSLB / CDN   |
                +----------+-----------+
                           |
                +----------v-----------+
                |   L4 Load Balancer   |
                |   SLB / LVS / NLB   |
                +----------+-----------+
                           |
          +----------------+----------------+
          |                                 |
+---------v---------+               +---------v---------+
| Nginx Gateway A   |               | Nginx Gateway B   |
| Rate‑limit / Gray / TLS termination |
+---------+---------+               +---------+---------+
          |                                 |
          +----------------+----------------+
                           |
                +----------v-----------+
                | Service Mesh / K8s or |
                | Service Registry      |
                +----------+-----------+
                           |
          +----------------+----------------+
          |                                 |
+---------v---------+               +---------v---------+
| Application Pool |               | Application Pool |
+-------------------+               +-------------------+

Benefits:

L4 layer solves high‑availability and traffic splitting.

Nginx layer focuses on seven‑layer traffic governance.

Backend services remain stateless, easing scaling.

Clear responsibilities simplify troubleshooting.

9.3 Proper Way to Do Dynamic Service Discovery

In micro‑service environments, manually maintaining upstreams is hard. Three common approaches:

Config generation + hot reload.

OpenResty/Lua dynamic discovery.

Delegate to Ingress/Gateway controller.

If you already have Nacos/Consul/Eureka, consider:

Service list maintained by the registry.

Config center or sidecar generates upstream config.

After change, run nginx -t && nginx -s reload.

This method is more stable than querying the registry on every request.

10. Real Production Case: How a Mixed 502/504 Failure Was Diagnosed

10.1 Business Background

2 × 16C 32G Nginx gateways.

8 Java service instances.

MySQL + Redis backend.

Normal traffic 8,000 QPS; expected peak 25,000 QPS during a promotion.

10.2 First Impression: It Looked Like Nginx Was the Problem

Insufficient worker_connections?

Nginx timeout too short?

Not enough gateway instances?

Metrics showed: stub_status reported high Waiting and growing Writing. upstream_connect_time normal. upstream_response_time skyrocketed.

Gateway CPU only at 45% while connections surged.

Conclusion: Nginx could establish connections; the bottleneck was upstream response latency.

10.3 Deeper Diagnosis

Application‑side monitoring revealed:

Tomcat thread pool near its limit.

Significant increase in DB slow queries.

Redis hit rate dropped; hotspot keys expired and fell back to DB.

Root cause chain:

Hot cache keys expired simultaneously.

Massive requests hit the database.

Application thread pool blocked.

Nginx waited a long time for upstream response.

Some requests timed out → 504.

Retry on idempotent reads amplified backend pressure.

Overloaded instances began returning 502.

10.4 Final Mitigation Actions

Gateway layer:

Shorten total retry time for idempotent read APIs.

Add rate‑limit and degradation for non‑core APIs.

Introduce proxy_cache for hotspot interfaces.

Application layer:

Pre‑warm hotspot caches.

Apply logical expiration + single‑flight protection for hotspot keys.

Adjust thread‑pool isolation settings.

Database layer:

Add missing indexes.

Remove the slowest aggregation SQL during the promotion.

10.5 Post‑mortem Conclusion

Nginx is often not the problem creator but the problem amplifier and alarm system. It surfaces the anomaly first, but the root cause usually lies downstream.

Therefore, high‑concurrency troubleshooting must follow the call chain beyond Nginx.

11. Systematic Troubleshooting: How to Investigate 502, 504, 499, and Connection Surges Under High Load

11.1 Establish the Troubleshooting Order

Client → L4 Load Balancer → Nginx → Upstream Network → Application Service → Cache/Database

Do not change configuration immediately; first identify which category the problem belongs to:

Connection establishment failure.

Connection established but upstream timeout.

Client aborts early.

Gateway resource exhaustion.

Backend cascade jitter.

11.2 Common Causes of 502 Bad Gateway

Typical log lines:

connect() failed (111: Connection refused) while connecting to upstream
no live upstreams while connecting to upstream
upstream prematurely closed connection while reading response header from upstream

Key checks:

Is the upstream instance alive?

Is the port listening?

Does the backend actively reset the connection?

Has Nginx marked all instances as failed?

Has the backend reached its connection limit?

Diagnostic commands:

curl -I http://10.0.1.11:8080/health
ss -tlnp | grep 8080
ss -s
tail -f /var/log/nginx/error.log

11.3 Common Causes of 504 Gateway Timeout

Typical log line:

upstream timed out (110: Connection timed out) while reading response header from upstream

Focus on three time metrics:

$upstream_connect_time

$upstream_header_time

$upstream_response_time

Interpretation:

High connect_time → network or connection‑establishment issue.

High header_time → slow application processing.

High response_time → large objects, slow SQL, or slow streaming.

11.4 499 Client Closed Request Is Not Simply a "Client Problem"

499 indicates the client closed the connection early, but common root causes include:

Page waits too long, user aborts.

App or browser timeout.

CDN/L4 timeout earlier than Nginx.

Over‑aggressive front‑end retry mechanisms.

If 499 rises together with high latency, the backend is likely already slow.

11.5 File Descriptor Exhaustion

Typical error: too many open files Investigation steps:

ulimit -n
cat /proc/$(pgrep -o nginx)/limits | grep "open files"
lsof -p $(pgrep -f "nginx: worker" | head -n 1) | wc -l
ss -tan | awk 'NR>1 {print $1}' | sort | uniq -c

If TIME_WAIT, ESTABLISHED, or CLOSE_WAIT are abnormally high, merely increasing fd limits is insufficient; you must revisit connection reuse and backend close policies.

12. Monitoring and Observability: No Metrics, No High‑Concurrency Governance

12.1 Essential Nginx Metrics

Active connections.

Reading / Writing / Waiting.

QPS.

2xx / 4xx / 5xx ratios.

P50 / P95 / P99 latency.

Upstream connect / header / response times.

Per‑upstream node error rate and latency.

12.2 stub_status Configuration

server {
    listen 127.0.0.1:18080;
    location /nginx_status {
        stub_status;
        access_log off;
        allow 127.0.0.1;
        deny all;
    }
}

12.3 Prometheus Exporter

docker run -d --name nginx-exporter -p 9113:9113 \
  nginx/nginx-prometheus-exporter:1.4.1 \
  --nginx.scrape-uri=http://127.0.0.1:18080/nginx_status

12.4 Logging Must Support Per‑Request Tracing

Unified request_id.

Log includes upstream address, status code, and timings.

When integrating with tracing systems, propagate the trace id to upstreams.

Without these fields, it is hard to answer which upstream is slow, whether the slowness is isolated, or whether the error originated at the entry or the backend.

12.5 Load Testing Is a Pre‑Release Condition, Not a After‑The‑Fact Fix

Recommended load‑test categories:

Baseline test: single‑API peak capability.

Mixed test: read/write mix, cache hit/miss mix.

Failure injection test: downstream slowdown, node failure, network jitter, hot‑reload.

wrk -t12 -c2000 -d60s --latency http://api.example.com/api/v1/users

POST script example:

wrk.method = "POST"
wrk.body   = '{"skuId":1001,"count":1}'
wrk.headers["Content-Type"] = "application/json"

Load‑test conclusions should examine not only QPS but also:

P99 latency degradation.

Error rate increase.

Upstream connection‑pool contention.

Gateway CPU or fd bottleneck.

13. Security and Traffic Governance: Production Capabilities Required at the Entry Layer

13.1 Rate‑Limiting Example

limit_conn_zone   $binary_remote_addr zone=per_ip_conn:20m;
limit_req_zone    $binary_remote_addr zone=per_ip_rate:20m rate=20r/s;

server {
    location /api/ {
        limit_conn per_ip_conn 50;
        limit_req zone=per_ip_rate burst=40 nodelay;
        limit_conn_status 429;
        limit_req_status 429;
        proxy_pass http://user_service;
    }
}

Rate limiting protects against attacks, abusive crawlers, and snowball effects during spikes.

13.2 Security Headers and TLS

server {
    listen 443 ssl http2;
    server_name api.example.com;

    ssl_certificate     /etc/nginx/tls/fullchain.pem;
    ssl_certificate_key /etc/nginx/tls/privkey.pem;
    ssl_session_cache  shared:SSL:50m;
    ssl_session_timeout 1d;
    ssl_session_tickets off;
    ssl_protocols       TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers off;

    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header Referrer-Policy "strict-origin-when-cross-origin" always;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
}

If TLS traffic is a large proportion, also evaluate CPU cost of handshakes, certificate chain length, session reuse, and whether TLS termination should be offloaded to CDN or L4.

14. Production Checklist

1. worker_processes matches CPU cores.
2. worker_connections, ulimit, fs.file-max are coordinated.
3. upstream keepalive enabled and backend truly supports reuse.
4. Logs contain request_id, upstream timings, upstream status.
5. Critical APIs have SLA‑based timeout tiers.
6. Retries are enabled only for idempotent requests.
7. Rate‑limit, circuit‑breaker, degradation strategies are in place.
8. Hotspot APIs have cache or back‑origin protection.
9. Kubernetes rollout has graceful shutdown.
10. Fault‑injection load tests have been performed.
11. 502/504/499 alerts and thresholds are defined.
12. Capacity boundaries for entry, application, and database layers are documented.

15. Conclusion: True Nginx High‑Concurrency Capability Comes from Systematic Design

Nginx’s strength is not a handful of "magic parameters"; it lies in being the system entry point that ties together the Linux network stack, connection reuse, traffic governance, and application architecture.

Key takeaways:

Capacity design precedes parameter tuning.

Most bottlenecks are not in Nginx itself but in downstream services and system layers.

Upstream keepalive, tiered timeouts, rate limiting, and logging provide the highest cost‑performance gains.

Containerization and Kubernetes amplify connection‑management and release‑stability requirements.

Without load testing, monitoring, and failure drills, a high‑concurrency architecture cannot be reliable.

When you treat Nginx as a traffic‑dispatch hub rather than a simple reverse proxy, it evolves from a component that "can run" into a scalable, observable, and governable production entry.