Optimizing Apache HttpClient for High-Concurrency Scenarios

1. Background

Our service calls an HTTP‑based API provided by another department, with daily request volume reaching tens of millions. Using Apache HttpClient, the original average execution time was 250 ms, causing frequent thread‑exhaustion alerts.

After optimization, the average time dropped to 80 ms, a two‑third reduction, and the container became stable.

2. Analysis

The original implementation created a new HttpClient and HttpPost for each request, extracted the entity to a string, and explicitly closed the response and client each time.

2.1 Repeated HttpClient creation

HttpClient is thread‑safe; creating a new instance per request incurs unnecessary overhead. A singleton client should be retained.

2.2 Repeated TCP connection creation

Each request performed a full TCP three‑way handshake and four‑way teardown, consuming several milliseconds. Switching to keep‑alive reuses connections and eliminates this cost.

2.3 Redundant entity buffering

The code copied the response content into a string while the original HttpResponse still held the entity, leading to double memory usage and the need for manual stream closure.

HttpEntity entity = httpResponse.getEntity();
String response = EntityUtils.toString(entity);

3. Implementation

We address three main tasks: a singleton client, connection pooling with keep‑alive, and better response handling.

3.1 Define a keep‑alive strategy

Customize the keep‑alive duration based on the server’s "timeout" header, defaulting to 60 seconds.

ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {
    @Override
    public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
        HeaderElementIterator it = new BasicHeaderElementIterator(
            response.headerIterator(HTTP.CONN_KEEP_ALIVE));
        while (it.hasNext()) {
            HeaderElement he = it.nextElement();
            String param = he.getName();
            String value = he.getValue();
            if (value != null && param.equalsIgnoreCase("timeout")) {
                return Long.parseLong(value) * 1000;
            }
        }
        return 60 * 1000; // default 60s
    }
};

3.2 Configure a pooling connection manager

PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(500);
connectionManager.setDefaultMaxPerRoute(50); // adjust per business needs

3.3 Build the HttpClient

httpClient = HttpClients.custom()
    .setConnectionManager(connectionManager)
    .setKeepAliveStrategy(myStrategy)
    .setDefaultRequestConfig(RequestConfig.custom()
        .setStaleConnectionCheckEnabled(true)
        .build())
    .build();

Note: Using setStaleConnectionCheckEnabled is discouraged; a dedicated thread should periodically invoke closeExpiredConnections and closeIdleConnections .

public class IdleConnectionMonitorThread extends Thread {
    private final HttpClientConnectionManager connMgr;
    private volatile boolean shutdown;
    public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
        super();
        this.connMgr = connMgr;
    }
    @Override
    public void run() {
        try {
            while (!shutdown) {
                synchronized (this) {
                    wait(5000);
                    connMgr.closeExpiredConnections();
                    connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
                }
            }
        } catch (InterruptedException ex) {
            // terminate
        }
    }
    public void shutdown() {
        shutdown = true;
        synchronized (this) { notifyAll(); }
    }
}

3.4 Reduce overhead when executing methods

Prefer a ResponseHandler so that the client automatically consumes the entity and closes streams.

public static String getResponse(String url) throws IOException {
    HttpGet get = new HttpGet(url);
    return httpclient.execute(get, responseHandler);
}

4. Additional Configurations

4.1 Timeout settings

HttpParams params = new BasicHttpParams();
int CONNECTION_TIMEOUT = 2 * 1000; // 2 seconds
int SO_TIMEOUT = 2 * 1000; // 2 seconds
long CONN_MANAGER_TIMEOUT = 500L;
params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, CONNECTION_TIMEOUT);
params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, SO_TIMEOUT);
params.setLongParameter(ClientPNames.CONN_MANAGER_TIMEOUT, CONN_MANAGER_TIMEOUT);
params.setBooleanParameter(CoreConnectionPNames.STALE_CONNECTION_CHECK, true);
httpClient.setHttpRequestRetryHandler(new DefaultHttpRequestRetryHandler(0, false));

4.2 Nginx keep‑alive settings

If Nginx is in front of the service, configure keepalive_timeout , keepalive_requests on the client side and keepalive on the upstream side.

By applying these changes, the request latency dropped from 250 ms to roughly 80 ms, eliminating thread‑exhaustion warnings and achieving stable high‑throughput performance.