How HTTP Pipelining Supercharges Request Throughput: Real‑World Performance Tests

High performance here means that the network card can send requests as fast as it receives them, and even a single client can cause noticeable latency on a typical server.

Practice

The article first shows a comparison test where a single client sends 10,000 requests as quickly as possible and receives the responses.

Four common client approaches are described:

Single‑process or thread polling (low performance).

Multiple threads prepared with data waiting for a signal (high client‑side demand).

A group of threads polling simultaneously.

Using the system/platform’s asynchronous sending mechanism (essentially a thread‑pool where sending and receiving use different threads).

Only the last two methods are evaluated because the first two perform poorly.

The test scenarios include:

A pipelined (pipe) method using 100 pipelines, each sending 100 requests (most servers limit a pipeline to about 100 concurrent requests).

A thread‑group method with 100 threads, each sending 100 requests.

An asynchronous method where all 10,000 requests are submitted to a thread pool that also handles receiving.

Test environment: a typical home PC (i5 quad‑core, 12 GB RAM, 100 Mb telecom bandwidth).

Test request example:

GET http://www.baidu.com HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: www.baidu.com
Connection: Keep-Alive

All pipe tests use the PipeHttpRuner tool (download link provided in the original article).

Results

Images (shown below) illustrate CPU, network send/receive rates, and task‑manager metrics. The pipelined method completes the whole test in about 5 seconds, with peak send and receive rates reached within 0.5 seconds.

Thread‑group testing takes roughly 25 seconds, while the asynchronous method exceeds one minute because the default thread pool cannot create enough threads quickly enough, leading to heavy context switching.

Further tests on JD, Taobao, Youku, and internal servers show similar ten‑fold performance differences when the server can keep up.

A second test against NetEase e‑commerce API (10,000 requests, ~326 MB response data) finishes in under 30 seconds, limited mainly by network bandwidth.

Principle

Typical HTTP flow: after TCP handshake, a request is sent, the client waits for the response, then the connection may be kept alive for the next request. With HTTP/1.1 keep‑alive, the same connection can be reused, but each request still waits for its response.

HTTP pipelining removes this wait: multiple requests are sent back‑to‑back without waiting for responses, allowing sending and receiving to proceed in parallel. In practice, several requests may be packed into a single TCP segment, and the server may start replying only after many requests have been received.

Because of the rapid send rate, issues such as duplicate ACKs, out‑of‑order packets, and retransmissions appear, but TCP’s recovery mechanisms handle them without degrading overall pipelining performance.

High‑speed transmission can also cause TCP window exhaustion (ZeroWindow/Window‑full), so both client and server must read from the TCP buffer promptly.

Drawbacks

While pipelining solves head‑of‑line blocking, it breaks the one‑request‑one‑response correspondence, making it harder to match responses to requests for operations like POST. Adding unique identifiers to each request/response pair or using HTTP/2 (which tags each frame with a stream ID) resolves this.

Implementation

A simple .NET library ( MyPipeHttpHelper) and a demo tool ( PipeHttpRuner) implement the pipelined client. The source code is available on GitHub:

https://github.com/lulianqi/PipeHttp/

https://github.com/lulianqi/PipeHttp/tree/master/MyPipeHttpHelper

https://github.com/lulianqi/PipeHttp/tree/master/PipeHttpRuner

Images illustrating the test results and the pipeline mechanism are included below.