Optimizing Apache HttpClient for High‑Concurrency: Connection Pooling, Keep‑Alive, and Configuration

Background : The business calls an HTTP service millions of times per day. The original implementation created a new HttpClient for each request, resulting in an average execution time of 250 ms and frequent thread‑exhaustion alerts.

Analysis : The main performance bottlenecks were (1) repeated creation of HttpClient instances, (2) repeated TCP three‑way handshakes, (3) unnecessary duplication of the response entity, and (4) lack of connection reuse and proper configuration.

Implementation :

1. Define a keep‑alive strategy

ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {
    @Override
    public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
        HeaderElementIterator it = new BasicHeaderElementIterator(
            response.headerIterator(HTTP.CONN_KEEP_ALIVE));
        while (it.hasNext()) {
            HeaderElement he = it.nextElement();
            String param = he.getName();
            String value = he.getValue();
            if (value != null && param.equalsIgnoreCase("timeout")) {
                return Long.parseLong(value) * 1000;
            }
        }
        return 60 * 1000; // default 60 seconds
    }
};

2. Configure a PoolingHttpClientConnectionManager

PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(500);
connectionManager.setDefaultMaxPerRoute(50); // adjust per business needs

3. Build a singleton HttpClient

httpClient = HttpClients.custom()
    .setConnectionManager(connectionManager)
    .setKeepAliveStrategy(myStrategy)
    .setDefaultRequestConfig(RequestConfig.custom()
        .setStaleConnectionCheckEnabled(true)
        .build())
    .build();

Note: Using setStaleConnectionCheckEnabled is discouraged; a dedicated thread to close expired/idle connections is preferred.

4. Idle connection monitor thread

public static class IdleConnectionMonitorThread extends Thread {
    private final HttpClientConnectionManager connMgr;
    private volatile boolean shutdown;
    public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
        super();
        this.connMgr = connMgr;
    }
    @Override
    public void run() {
        try {
            while (!shutdown) {
                synchronized (this) {
                    wait(5000);
                    connMgr.closeExpiredConnections();
                    connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
                }
            }
        } catch (InterruptedException ex) {
            // terminate
        }
    }
    public void shutdown() {
        shutdown = true;
        synchronized (this) { notifyAll(); }
    }
}

5. Execute requests without manually closing connections

Using a ResponseHandler lets HttpClient automatically consume the entity:

public
T execute(final HttpHost target, final HttpRequest request,
        final ResponseHandler
responseHandler, final HttpContext context)
        throws IOException, ClientProtocolException {
    Args.notNull(responseHandler, "Response handler");
    final HttpResponse response = execute(target, request, context);
    final T result;
    try {
        result = responseHandler.handleResponse(response);
    } catch (final Exception t) {
        final HttpEntity entity = response.getEntity();
        try { EntityUtils.consume(entity); } catch (final Exception t2) { log.warn("Error consuming content after an exception.", t2); }
        if (t instanceof RuntimeException) throw (RuntimeException) t;
        if (t instanceof IOException) throw (IOException) t;
        throw new UndeclaredThrowableException(t);
    }
    final HttpEntity entity = response.getEntity();
    EntityUtils.consume(entity);
    return result;
}

6. Timeout and retry configuration

HttpParams params = new BasicHttpParams();
int CONNECTION_TIMEOUT = 2 * 1000; // 2 seconds
int SO_TIMEOUT = 2 * 1000; // 2 seconds
long CONN_MANAGER_TIMEOUT = 500L;
params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, CONNECTION_TIMEOUT);
params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, SO_TIMEOUT);
params.setLongParameter(ClientPNames.CONN_MANAGER_TIMEOUT, CONN_MANAGER_TIMEOUT);
params.setBooleanParameter(CoreConnectionPNames.STALE_CONNECTION_CHECK, true);
httpClient.setHttpRequestRetryHandler(new DefaultHttpRequestRetryHandler(0, false));

7. Nginx keep‑alive settings (if applicable)

Adjust keepalive_timeout , keepalive_requests on the client side and upstream keepalive on the server side to ensure both ends reuse connections.

Result : After applying the above changes, the average request latency dropped from 250 ms to roughly 80 ms, and the container no longer reported thread‑exhaustion warnings.