Why a Single TLS Handshake Can Saturate a 100Mbps Uplink and How to Reduce It by 70%

Problem description

A high‑concurrency data‑collection service was deployed on a dedicated 100 Mbps uplink. The uplink became saturated because each GET request, although small in payload, transmitted 1.68 KB upstream.

Network capture showed that the TLS handshake alone accounted for 1.27 KB of that size.

1.68 KB × 20 000 requests ÷ 1024 × 8 ≈ 262.5 Mbps

This calculation explains why a 100 Mbps link was easily overwhelmed.

TLS handshake overhead

HTTPS is HTTP over TLS. For every new TCP connection a full TLS handshake is performed, during which the client and server exchange:

Client and server random numbers

Supported cipher suites and TLS version

Server digital certificate (public key)

Pre‑Master Secret used to derive symmetric keys

The handshake consumes both bandwidth and CPU cycles, adding latency to each request.

Reducing upstream bandwidth

Option 1 – Use plain HTTP

Switching the request protocol from HTTPS to HTTP eliminates the TLS handshake. The request size drops from 1.68 KB to roughly 0.4 KB, a reduction of about 70 %.

Packet capture after the change confirms that the TLS handshake is no longer present.

Option 2 – Enable HTTP Keep‑Alive

If HTTPS is required, enable Keep‑Alive so that multiple HTTP requests can reuse the same TCP connection. Only the first request incurs the full TLS handshake; subsequent requests send only the HTTP payload.

To enable it, add the following header to the request: Connection: keep-alive Server‑side keep‑alive timeout defaults:

Nginx: 75 seconds

Apache HTTP Server: 5 seconds

When the timeout expires, the connection is closed and a new TLS handshake is required for the next request.

If the crawler heavily rotates proxy IPs, keep‑alive may have limited effect; using plain HTTP can still provide the greatest bandwidth savings.

Technical illustration

Below is a sample capture of a single HTTPS request showing the TLS handshake size.