Boost Your Mac Load Test: Tuning sysctl and HttpClient for 20k+ Connections

While preparing load‑testing scripts for a server‑side performance test, the author encountered a large number of sockets stuck in the TIME_WAIT state on macOS. To resolve this, several macOS kernel parameters were adjusted.

System‑wide limits:

sudo sysctl -w kern.maxfiles=1048600    # maximum file descriptors
sudo sysctl -w kern.maxfilesperproc=1048576  # per‑process file descriptor limit

Port range configuration:

net.inet.ip.portrange.first = 10240   # start of dynamic port range
net.inet.ip.portrange.last = 65535    # end of dynamic port range

Temporary port settings (allowing up to 49 151 ports):

sysctl -w net.inet.ip.portrange.hifirst = 16384
sysctl -w net.inet.ip.portrange.first = 16384

After the macOS tweaks, the focus shifted to Apache HttpClient configuration. Several approaches were tried:

Setting the request header Connection: close:

Attempting to enable keep‑alive via the deprecated SimpleHttpConnectionManager – abandoned because the newer PoolingHttpClientConnectionManager is recommended.

Calling method.releaseConnection() – caused occasional SocketException (timeouts, closed sockets) and offered no clear benefit.

Implementing a custom connection‑recycling thread based on the official HttpClient example:

The final HttpClient settings were split into two categories:

Connection pool configuration

The distinction between MAX_TOTAL_CONNECTION (overall pool size) and MAX_PER_ROUTE_CONNECTION (per‑host limit) is explained, e.g., MAX_TOTAL=400 and MAX_PER_ROUTE=200 means a single host can use at most 200 concurrent connections, while the pool can hold up to 400 total.

connManager.setMaxTotal(MAX_TOTAL_CONNECTION);
connManager.setDefaultMaxPerRoute(MAX_PER_ROUTE_CONNECTION);

Timeout configuration

RequestConfig.custom()
    .setConnectionRequestTimeout(CONNECT_REQUEST_TIMEOUT)
    .setConnectTimeout(CONNECT_TIMEOUT)
    .setSocketTimeout(SOCKET_TIMEOUT)
    .setCookieSpec(CookieSpecs.STANDARD)
    .build();

Using HttpClient 4.5.5 on an iMac, the author achieved more than 20 000 concurrent requests without observing a local bottleneck, demonstrating that the combined macOS kernel tweaks and HttpClient tuning effectively support high‑concurrency load testing.