Why HTTP Feels Slow: TCP Bottlenecks, Delayed ACK, Nagle & Keep‑Alive Tuning

Background

HTTP performance ultimately depends on the underlying TCP layer; HTTPS adds TLS/SSL overhead, making it slower. The analysis assumes a single‑server model, excludes client/server overload, different I/O models, DNS resolution, and application‑level bugs.

Key TCP Factors Impacting HTTP

TCP connection establishment : Each short‑lived HTTP request triggers a three‑way handshake, which becomes costly when many resources are fetched.

TCP slow start : After a new connection is created, TCP limits the initial transmission rate to avoid congestion, delaying data transfer for short connections.

TCP delayed ACK : Receivers wait 100‑200 ms to batch acknowledgments, reducing small packet overhead but increasing latency for many tiny HTTP requests.

Nagle algorithm : Small packets are held until previous data is ACKed, further slowing transmission of numerous small HTTP fragments.

Consequences of Short‑Lived Connections

Because HTTP/1.1 defaults to persistent connections, enabling keep‑alive reduces the number of handshakes and mitigates slow‑start penalties. Without keep‑alive, each request suffers from the above TCP inefficiencies.

TIME_WAIT and Port Exhaustion

Clients that actively close connections enter the TIME_WAIT state for 2 MSL (typically a few seconds). Rapid, repeated connections can exhaust the pool of available ephemeral ports, causing connection failures.

Server‑Side Port Exhaustion

In proxy mode (e.g., Nginx forwarding to a backend), Nginx acts as a client to the backend, consuming random ports. If the rate of outbound connections exceeds port release speed, the server may run out of ports.

File‑Descriptor Limits

Each TCP socket consumes a file descriptor. On Unix‑like systems, the maximum open files per process (checked with ulimit -f) can become a bottleneck when handling many concurrent connections.

Persistent Connections (HTTP Keep‑Alive)

HTTP keep‑alive reuses a single TCP connection for multiple requests, avoiding repeated handshakes and slow‑start. In HTTP/1.0 the client must send Connection: Keep-alive; in HTTP/1.1 it is the default unless Connection: close is specified.

Key Nginx directives: keepalive_timeout 65s – idle timeout for persistent connections. keepalive_requests 100 – maximum requests per connection (default 100).

TCP Keep‑Alive vs HTTP Keep‑Alive

TCP keep‑alive is a low‑level heartbeat mechanism that keeps the socket open; it is unrelated to HTTP keep‑alive, which is a protocol feature for connection reuse.

Linux TCP Keep‑Alive Settings

Inspect current values with: sysctl -a | grep tcp_keepalive Typical defaults on CentOS:

net.ipv4.tcp_keepalive_intvl = 75   # interval between probes

net.ipv4.tcp_keepalive_probes = 9   # number of probes before giving up

net.ipv4.tcp_keepalive_time = 7200  # time before first probe (2 hours)

Enabling TCP Keep‑Alive in Nginx

Use the listen directive with so_keepalive parameter:

# Enable with system defaults

listen 80 default_server so_keepalive=on;

# Explicitly disable

listen 80 default_server so_keepalive=off;

# Custom interval (30 minutes) and probe count (10)

listen 80 default_server so_keepalive=30m::10;

Whether to enable this depends on the deployment scenario; it is unrelated to HTTP keep‑alive.

Practical Testing with Nginx

Setting keepalive_timeout 65s in Nginx and accessing the default page with a browser shows a Keep‑Alive marker in the packet capture, followed by a server‑initiated close after the timeout, leaving the connection in TIME_WAIT.

In summary, optimizing HTTP performance requires understanding and tuning TCP connection establishment, slow start, delayed ACK, Nagle algorithm, and keep‑alive settings both at the OS and web server level.