Optimizing Apache HttpClient for High Concurrency: Pooling, Keep‑Alive, and Configuration

HttpClient optimization ideas:

Pooling

Long connections (keep‑alive)

Reuse HttpClient and HttpGet

Reasonable configuration parameters (max concurrent requests, timeouts, retry count)

Asynchronous execution

Read source code thoroughly

1. Background

Our business calls an HTTP service provided by another department with a daily call volume of tens of millions. Using HttpClient, the average execution time before optimization was 250 ms; after optimization it dropped to 80 ms, eliminating thread‑exhaustion alarms.

2. Analysis

The original implementation created a new HttpClient and HttpPost for each request, extracted the entity to a string, and explicitly closed the response and client.

2.1 Repeated creation of HttpClient

HttpClient is thread‑safe and should be kept as a global singleton rather than instantiated per request.

2.2 Repeated TCP connection creation

Each handshake adds several milliseconds; using keep‑alive allows connection reuse and removes this overhead.

2.3 Redundant entity copying

Original code copied the response content into a string while the HttpResponse still held the entity, causing extra memory consumption.

HttpEntity entity = httpResponse.getEntity();

String response = EntityUtils.toString(entity);

3. Implementation

Based on the analysis, we performed three main actions: a singleton client, a pooled keep‑alive connection manager, and better response handling.

3.1 Define a keep‑alive strategy

Keep‑alive usage depends on the business scenario. The following strategy returns the timeout value from the response header or defaults to 60 seconds.

ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {
    @Override
    public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
        HeaderElementIterator it = new BasicHeaderElementIterator(response.headerIterator(HTTP.CONN_KEEP_ALIVE));
        while (it.hasNext()) {
            HeaderElement he = it.nextElement();
            String param = he.getName();
            String value = he.getValue();
            if (value != null && param.equalsIgnoreCase("timeout")) {
                return Long.parseLong(value) * 1000;
            }
        }
        return 60 * 1000; // default 60s
    }
};

3.2 Configure a PoolingHttpClientConnectionManager

PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(500);
connectionManager.setDefaultMaxPerRoute(50); // adjust per business needs

3.3 Build the HttpClient

httpClient = HttpClients.custom()
    .setConnectionManager(connectionManager)
    .setKeepAliveStrategy(myStrategy)
    .setDefaultRequestConfig(RequestConfig.custom().setStaleConnectionCheckEnabled(true).build())
    .build();

Note: Using setStaleConnectionCheckEnabled is not recommended; a better approach is to run a background thread that periodically calls closeExpiredConnections and closeIdleConnections .

public static class IdleConnectionMonitorThread extends Thread {
    private final HttpClientConnectionManager connMgr;
    private volatile boolean shutdown;
    public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
        super();
        this.connMgr = connMgr;
    }
    @Override
    public void run() {
        try {
            while (!shutdown) {
                synchronized (this) {
                    wait(5000);
                    connMgr.closeExpiredConnections();
                    connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
                }
            }
        } catch (InterruptedException ex) {
            // terminate
        }
    }
    public void shutdown() {
        shutdown = true;
        synchronized (this) {
            notifyAll();
        }
    }
}

3.4 Reduce overhead when executing methods

Do not close the connection manually.

res = EntityUtils.toString(response.getEntity(), "UTF-8");
EntityUtils.consume(response1.getEntity());

Using a ResponseHandler simplifies exception handling and ensures the entity is consumed automatically.

public
T execute(final HttpHost target, final HttpRequest request,
        final ResponseHandler
responseHandler, final HttpContext context)
        throws IOException, ClientProtocolException {
    Args.notNull(responseHandler, "Response handler");
    final HttpResponse response = execute(target, request, context);
    final T result;
    try {
        result = responseHandler.handleResponse(response);
    } catch (final Exception t) {
        final HttpEntity entity = response.getEntity();
        try {
            EntityUtils.consume(entity);
        } catch (final Exception t2) {
            this.log.warn("Error consuming content after an exception.", t2);
        }
        if (t instanceof RuntimeException) {
            throw (RuntimeException) t;
        }
        if (t instanceof IOException) {
            throw (IOException) t;
        }
        throw new UndeclaredThrowableException(t);
    }
    final HttpEntity entity = response.getEntity();
    EntityUtils.consume(entity);
    return result;
}

4. Other Settings

4.1 Timeout configuration

Connection timeout and socket timeout are distinct; both should be tuned according to business needs.

HttpParams params = new BasicHttpParams();
// connection timeout (ms)
Integer CONNECTION_TIMEOUT = 2 * 1000; // 2 seconds
Integer SO_TIMEOUT = 2 * 1000; // 2 seconds
Long CONN_MANAGER_TIMEOUT = 500L; // ms
params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, CONNECTION_TIMEOUT);
params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, SO_TIMEOUT);
params.setLongParameter(ClientPNames.CONN_MANAGER_TIMEOUT, CONN_MANAGER_TIMEOUT);
params.setBooleanParameter(CoreConnectionPNames.STALE_CONNECTION_CHECK, true);
httpClient.setHttpRequestRetryHandler(new DefaultHttpRequestRetryHandler(0, false));

4.2 Nginx keep‑alive settings

If Nginx is used, configure keepalive_timeout , keepalive_requests , and upstream keepalive accordingly.

After applying all these configurations, the average request latency dropped from 250 ms to about 80 ms, and the system became stable under high QPS.

Dependency:

org.apache.httpcomponents
httpclient
4.5.6

Key code snippets (singleton client, request execution, URL helpers) are included above.