Why Your HTTP Calls Take 40 ms: Uncovering TCP_NODELAY, Nagle and Delayed ACK

1. Background

In a test environment an HTTP call using Apache HttpClient to a simple service that only converts a 100‑character string to uppercase took an average of 39.2 ms, far higher than the expected 2‑3 ms given a ping of 1.9 ms.

2. External Metrics

2.1 System Metrics

CPU and load were idle; top showed no pressure.

2.2 Process Metrics

GC pauses were under 10 ms and thread pools were not busy, so the delay was not caused by the application code.

3. Local Reproduction

A simple test program on a Mac using Apache HttpClient measured about 55 ms; the backend server was in a different region with a ping of ~26 ms, so the expected time was ~26 ms, not 55 ms.

4. Diagnosis

4.1定位

Since system and process metrics were normal, the problem was suspected to be at the TCP layer.

4.2 Verification

Adding the JVM option -Dsun.net.httpserver.nodelay=true to the backend HttpServer reduced the average latency from 39.2 ms to 2.8 ms.

5. Explanation

5.1 What is TCP_NODELAY?

TCP_NODELAY disables Nagle’s algorithm; when true, small packets are sent immediately instead of being coalesced.

5.2 What is Nagle’s algorithm?

Nagle reduces the number of packets by buffering small writes until an ACK is received or a full MSS is available.

5.3 What is Delayed ACK?

Delayed ACK waits up to ~40 ms (Linux default) before sending an ACK, allowing it to be combined with outgoing data.

5.4 Interaction of Nagle and Delayed ACK

If the sender uses Nagle and the receiver uses Delayed ACK, the sender may wait for an ACK that is delayed, adding ~40 ms to each request.

5.5 Packet Capture

Running

sudo tcpdump -i eth0 tcp and host 10.48.159.165 -s 0 -w traffic.pcap

and analysing with Wireshark shows a 40 ms gap between the request and the ACK, confirming the Nagle‑Delayed ACK effect.

5.6 Why TCP_NODELAY fixes the issue

Disabling Nagle forces the client to send packets without waiting for an ACK, eliminating the 40 ms delay caused by Delayed ACK.

6. Summary

The case demonstrates a typical performance problem where TCP‑level settings (Nagle and Delayed ACK) inflate HTTP latency, and shows how enabling TCP_NODELAY or adjusting ACK timers resolves it.