Designing Nginx for Million‑Scale WebSocket Connections: Architecture, Configuration, and Pitfalls

1. Core Conclusion

Running WebSocket in production requires more than adding two headers; it must satisfy five conditions: successful handshake, long‑lived connections, scalable routing, smooth deployment, and full observability.

2. Typical Business Scenarios

Live‑stream chat (massive connections, dense broadcast)

Instant‑messaging or enterprise notifications (stable online users, complex subscription)

Collaborative editing (strict ordering and low latency)

Real‑time market data (high push frequency, complex subscription graph)

IoT device channels (long idle periods, frequent reconnections)

All share the same resource pressure:

connection count + heartbeat frequency + broadcast fan‑out + reconnection storms

.

3. WebSocket Handshake Mechanics

Client sends HTTP request with Upgrade: websocket, Connection: Upgrade, Sec-WebSocket-Key, Sec-WebSocket-Version.

Nginx forwards the request to the upstream.

Upstream replies 101 Switching Protocols.

The connection switches to a full‑duplex TCP tunnel.

After the handshake Nginx stops treating the traffic as HTTP body and forwards raw bytes.

4. Why Default Nginx Settings Fail

Missing proxy_http_version 1.1 Headers Upgrade and Connection not proxied proxy_read_timeout too small

Buffering left enabled

Upstream must truly support Upgrade

Typical symptoms: browser error "WebSocket connection failed", Nginx 400/426/502, handshake succeeds then disconnects after 60 s, or constant reconnection spikes.

5. Nginx’s Real Role

TLS termination

Reverse proxy & routing

Load balancing

Basic access control & rate limiting

Access logging & exposing a few metrics

It should NOT store session state, manage complex subscriptions, act as a cross‑node broadcast bus, or perform business‑level ACK/retry.

6. Production‑Level Architecture Diagram

Client / App / Browser
    |
    v
+---------------------------+
|  SLB / CLB / Ingress VIP  |
+---------------------------+
    |
    v
+---------------------------+
|      Nginx Cluster       |
| TLS / LB / Rate Limit    |
| Upgrade / Access Log     |
+---------------------------+
    |
    v
+---------------------------+
|  WebSocket Gateway Pods  |
| Conn Mgmt / Auth / Route |
+---------------------------+
    |
    +----+----+
    |         |
    v         v
Redis      Kafka / NATS
Presence   Broadcast / Event Bus
    |
    v
Business Services / User Service / Doc Service / IM Service

Key principles: Nginx only handles ingress & forwarding; the gateway owns connection lifecycle, authentication, heart‑beats, and message entry control; broadcast is off‑loaded to Redis, Kafka or NATS.

7. Minimal Working Configuration

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

upstream websocket_backend {
    least_conn;
    server 10.0.0.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:8080 max_fails=3 fail_timeout=30s;
}

server {
    listen 443 ssl;
    server_name ws.example.com;
    ssl_certificate /etc/nginx/certs/fullchain.pem;
    ssl_certificate_key /etc/nginx/certs/privkey.pem;

    location /ws/ {
        proxy_pass http://websocket_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 5s;
        proxy_read_timeout 75s;
        proxy_send_timeout 75s;
        proxy_buffering off;
        proxy_cache off;
    }
}

This config guarantees a stable handshake but is far from production‑ready.

8. Critical Nginx Parameters

8.1 worker_processes & worker_connections

worker_processes auto;

events {
    use epoll;
    worker_connections 65535;
    multi_accept on;
}

Each WebSocket consumes two file descriptors (client‑to‑Nginx and Nginx‑to‑upstream). Therefore the theoretical max is not simply worker_processes * worker_connections; you must also account for logs, listening sockets, and keep‑alive connections.

8.2 worker_rlimit_nofile & system limits

worker_rlimit_nofile 200000;

Ensure LimitNOFILE in the systemd unit, ulimit -n, and fs.file-max are all raised; otherwise you will see “too many open files”.

8.3 proxy_read_timeout

Setting it to a huge value only hides the real problem. The proper solution is a server‑side Ping every 25 s, a client‑side Pong, and a timeout slightly larger than the heartbeat interval.

8.4 proxy_buffering

WebSocket is a real‑time stream; buffering must be turned off (also proxy_request_buffering off) to avoid latency, small‑packet slowdown, and false “backend slowdown” signals.

8.5 Logging for Observability

log_format ws_access '\$remote_addr - \$remote_user [\$time_local] "\$request" \$status \$body_bytes_sent '\
    'rt=\$request_time urt=\$upstream_response_time '\
    'ua="\$upstream_addr" ustatus="\$upstream_status" '\
    'upgrade="\$http_upgrade" conn="\$connection"';

access_log /var/log/nginx/ws-access.log ws_access;
error_log  /var/log/nginx/ws-error.log warn;

The log can answer three key questions: handshake success, which upstream handled the request, and where the failure occurred.

9. Backend Gateway Design (Go Example)

The gateway must handle authentication, connection registration, heart‑beat, message routing, graceful shutdown, and back‑pressure control. A trimmed version of the production code is shown below.

package main

import (
    "context"
    "encoding/json"
    "errors"
    "log"
    "net/http"
    "os"
    "os/signal"
    "sync"
    "sync/atomic"
    "syscall"
    "time"

    "github.com/gorilla/websocket"
)

type Envelope struct {
    Type      string          `json:"type"`
    RequestID string          `json:"requestId,omitempty"`
    Body      json.RawMessage `json:"body,omitempty"`
}

type Client struct {
    UserID  string
    ConnID  string
    Conn    *websocket.Conn
    SendCh  chan []byte
    LastSeen atomic.Int64
}

type Hub struct {
    mu          sync.RWMutex
    clients      map[string]*Client
    closing      atomic.Bool
    activeConn   atomic.Int64
}

func NewHub() *Hub { return &Hub{clients: make(map[string]*Client)} }

func (h *Hub) Register(c *Client) error {
    if h.closing.Load() { return errors.New("gateway is shutting down") }
    h.mu.Lock(); defer h.mu.Unlock()
    h.clients[c.ConnID] = c
    h.activeConn.Add(1)
    return nil
}

func (h *Hub) Unregister(connID string) {
    h.mu.Lock(); defer h.mu.Unlock()
    if c, ok := h.clients[connID]; ok {
        delete(h.clients, connID)
        close(c.SendCh)
        h.activeConn.Add(-1)
    }
}

func (h *Hub) Shutdown() { h.closing.Store(true) }

func validateToken(r *http.Request) (string, error) {
    token := r.URL.Query().Get("token")
    if token == "" { return "", errors.New("missing token") }
    return "user-" + token, nil // placeholder for real JWT check
}

var upgrader = websocket.Upgrader{CheckOrigin: func(r *http.Request) bool { return r.Header.Get("Origin") == "https://app.example.com" }, ReadBufferSize: 4096, WriteBufferSize: 4096}

func serveWS(hub *Hub, w http.ResponseWriter, r *http.Request) {
    if hub.closing.Load() { http.Error(w, "server draining", http.StatusServiceUnavailable); return }
    userID, err := validateToken(r)
    if err != nil { http.Error(w, "unauthorized", http.StatusUnauthorized); return }
    conn, err := upgrader.Upgrade(w, r, nil)
    if err != nil { log.Printf("upgrade failed: %v", err); return }
    client := &Client{UserID: userID, ConnID: userID + "-" + time.Now().Format("20060102150405.000000000"), Conn: conn, SendCh: make(chan []byte, 256)}
    client.LastSeen.Store(time.Now().Unix())
    if err := hub.Register(client); err != nil { conn.WriteControl(websocket.CloseMessage, websocket.FormatCloseMessage(websocket.CloseGoingAway, "server draining"), time.Now().Add(3*time.Second)); conn.Close(); return }
    defer func(){ conn.Close(); hub.Unregister(client.ConnID) }()
    conn.SetReadLimit(64 * 1024)
    conn.SetReadDeadline(time.Now().Add(75 * time.Second))
    conn.SetPongHandler(func(appData string) error { client.LastSeen.Store(time.Now().Unix()); return conn.SetReadDeadline(time.Now().Add(75 * time.Second)) })
    go writePump(client)
    readPump(client)
}

func readPump(c *Client) {
    for {
        _, payload, err := c.Conn.ReadMessage()
        if err != nil { return }
        c.LastSeen.Store(time.Now().Unix())
        var env Envelope
        if err := json.Unmarshal(payload, &env); err != nil { continue }
        if env.Type == "ping" {
            resp, _ := json.Marshal(map[string]any{"type": "pong", "ts": time.Now().UnixMilli()})
            select { case c.SendCh <- resp: default: return }
        }
    }
}

func writePump(c *Client) {
    ticker := time.NewTicker(25 * time.Second)
    defer ticker.Stop()
    for {
        select {
        case msg, ok := <-c.SendCh:
            if !ok { c.Conn.WriteControl(websocket.CloseMessage, websocket.FormatCloseMessage(websocket.CloseNormalClosure, "closed"), time.Now().Add(3*time.Second)); return }
            c.Conn.SetWriteDeadline(time.Now().Add(10 * time.Second))
            if err := c.Conn.WriteMessage(websocket.TextMessage, msg); err != nil { return }
        case <-ticker.C:
            c.Conn.SetWriteDeadline(time.Now().Add(10 * time.Second))
            if err := c.Conn.WriteMessage(websocket.PingMessage, nil); err != nil { return }
        }
    }
}

func main() {
    hub := NewHub()
    mux := http.NewServeMux()
    mux.HandleFunc("/ws", func(w http.ResponseWriter, r *http.Request) { serveWS(hub, w, r) })
    mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusOK); w.Write([]byte("ok")) })
    mux.HandleFunc("/readyz", func(w http.ResponseWriter, r *http.Request) { if hub.closing.Load() { w.WriteHeader(http.StatusServiceUnavailable); w.Write([]byte("draining")); return } w.WriteHeader(http.StatusOK); w.Write([]byte("ready")) })
    srv := &http.Server{Addr: ":8080", Handler: mux, ReadHeaderTimeout: 5 * time.Second}
    go func(){ log.Printf("ws gateway started on :8080"); if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) { log.Fatalf("listen failed: %v", err) } }()
    stopCh := make(chan os.Signal, 1)
    signal.Notify(stopCh, syscall.SIGTERM, syscall.SIGINT)
    <-stopCh
    hub.Shutdown()
    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()
    if err := srv.Shutdown(ctx); err != nil { log.Printf("shutdown failed: %v", err) }
}

The code demonstrates read/write separation, bounded send channel, read limit, heartbeat handling, graceful draining, and health/readiness probes.

10. Capacity Modeling for a Million Connections

Assume each connection consumes 8‑20 KB; 1 M connections need 8‑20 GB RAM. With a 25 s heartbeat the system sees ~40 k heart‑beats per second. You must also budget memory for per‑connection buffers, Redis/Kafka bandwidth, and upstream network capacity.

11. Load‑Balancing Strategies

least_conn

– best for stateless gateways. ip_hash – only when you deliberately need client‑IP stickiness (rare).

Consistent hash on a business key (e.g., $arg_userId) – useful when session state is not fully externalized, but it re‑introduces hot‑spot risk.

12. Graceful Rolling Updates

Workflow: set pod to draining (readiness probe fails), Nginx stops sending new connections, existing connections stay alive for a buffer window, optionally push a “will reconnect soon” message, then close after timeout.

13. Kernel & System Tuning

fs.file-max = 1000000
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535

These raise the file‑descriptor ceiling and the listen‑queue size, preventing SYN backlog overflow and connection‑setup stalls.

14. Observability Essentials

Key Nginx metrics: active connections, handshake QPS, 101 success rate, 4xx/5xx ratios, upstream failures, rate‑limit hits. Gateway metrics: online connections per node, connect/disconnect rates, ping RTT, message QPS, broadcast latency, per‑connection queue depth, auth failures, forced disconnects.

15. Top 10 Real‑World Pitfalls

426 Upgrade Required – missing proxy_http_version 1.1 or header forwarding.

Stable 60 s disconnect – default proxy_read_timeout too low or missing heart‑beats.

Clients reconnect endlessly – killing pods without graceful drain.

Low CPU but high latency – slow‑connection backlog, broadcast blocking, or event‑loop blockage.

Uneven connection distribution – using ip_hash or NAT hotspots.

Benchmarks pass but production fails – tests ignore long‑idle connections, heartbeat overlap, or release storms.

Mobile apps disconnect often – network hand‑offs, aggressive NAT idle reclaim, weak Wi‑Fi.

TLS adds noticeable latency – missing session reuse, large cert chain, CPU‑bound crypto.

Redis Pub/Sub stalls at scale – limited fan‑out buffering; switch to Kafka/NATS when needed.

“Million connections” becomes “million idle sockets” – need connection garbage collection and state externalization.

16. When to Upgrade from Redis Pub/Sub to Kafka

Redis works for small‑to‑mid scale, simple rooms, no persistence. Kafka (or NATS) is chosen when you need high‑throughput broadcast, durable storage, multiple consumer groups, replay, or decoupled event pipelines.

17. Real‑World Case: Collaborative Document Editing

Client authenticates via JWT to wss://doc.example.com/ws.

Nginx terminates TLS and forwards to gateway.

Gateway writes userId → gatewayNodeId and docId → local subscriber set into Redis.

Document service emits edit events to Kafka; gateways consume the topic and fan‑out to local connections.

Gateway handles only real‑time delivery; document service guarantees ordering via OT/CRDT.

18. Production‑Grade Test Matrix

Handshake stress test – peak connection rate, TLS cost, rate‑limit enforcement.

Long‑run stability – hours of idle connections, memory curve, zombie cleanup.

Broadcast load – hotspot room fan‑out, per‑node queue depth, Redis/Kafka throughput.

Failure injection – node restart, network jitter, Redis outage, rolling deployment with drain.

The goal is not just a QPS number but confidence that the system remains controllable under anomalies.

19. Checklist Before Going Live

Validate handshake configuration (Upgrade, Connection, proxy_http_version).

Define and tune server‑side heart‑beat, align Nginx timeouts.

Raise OS file‑descriptor and listen‑queue limits.

Externalize session state; avoid session‑sticky load‑balancing.

Implement graceful draining for deployments.

Instrument connection, handshake, disconnect, broadcast latency metrics.

Run realistic lifecycle load tests, not just connection spikes.

When broadcast scales, evaluate migration to Kafka or NATS.

20. Conclusion

Nginx excels at the ingress layer of a WebSocket system but cannot replace a full‑featured real‑time communication platform. A production‑grade solution combines correct Nginx settings, a robust Go gateway, externalized state, proper load‑balancing, thorough observability, and systematic capacity planning.