Optimizing HttpClient for High‑Concurrency Scenarios: Pooling, Keep‑Alive, and Configuration

Hello, I am a senior architect sharing practical HttpClient optimization techniques.

Optimization ideas for HttpClient:

Pooling

Keep‑alive connections

Reuse HttpClient and HttpGet instances

Configure parameters wisely (max concurrent requests, timeouts, retry count)

Asynchronous execution

Read the source code thoroughly

1. Background

Our business calls an HTTP service provided by another department with a daily request volume of tens of millions. Using HttpClient, the average response time was 250 ms, which caused thread exhaustion warnings.

After optimization, the average time dropped to 80 ms, a two‑third reduction, and the container became stable.

2. Analysis

The original implementation created a new HttpClient and HttpPost for every request, read the entity into a string, and explicitly closed the response and client each time.

Key problems identified:

2.1 Repeated creation of HttpClient

HttpClient is thread‑safe; creating it per request adds unnecessary overhead. A single shared instance is sufficient.

2.2 Repeated TCP connection establishment

Each request performed a full TCP three‑way handshake and four‑way teardown, consuming several milliseconds. Switching to keep‑alive reuses connections and eliminates this cost.

2.3 Duplicate entity buffering

Original code:

HttpEntity entity = httpResponse.getEntity();

String response = EntityUtils.toString(entity);

This copies the response content into a string while the original HttpResponse still holds the stream, leading to high memory usage and the need for explicit connection closure.

3. Implementation

We performed three main actions: a singleton client, a pooled keep‑alive connection manager, and better result handling.

3.1 Define a keep‑alive strategy

Keep‑alive usage depends on the business; it is not a universal cure. Example strategy:

ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {
    @Override
    public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
        HeaderElementIterator it = new BasicHeaderElementIterator(
            response.headerIterator(HTTP.CONN_KEEP_ALIVE));
        while (it.hasNext()) {
            HeaderElement he = it.nextElement();
            String param = he.getName();
            String value = he.getValue();
            if (value != null && param.equalsIgnoreCase("timeout")) {
                return Long.parseLong(value) * 1000;
            }
        }
        return 60 * 1000; // default 60 s
    }
};

3.2 Configure a PoolingHttpClientConnectionManager

PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(500);
connectionManager.setDefaultMaxPerRoute(50); // adjust per business needs

You can also set per‑route limits if required.

3.3 Build the HttpClient

httpClient = HttpClients.custom()
    .setConnectionManager(connectionManager)
    .setKeepAliveStrategy(myStrategy)
    .setDefaultRequestConfig(RequestConfig.custom()
        .setStaleConnectionCheckEnabled(true)
        .build())
    .build();

Note: Using setStaleConnectionCheckEnabled is not recommended; instead, run a background thread to periodically call closeExpiredConnections() and closeIdleConnections() .

public static class IdleConnectionMonitorThread extends Thread {
    private final HttpClientConnectionManager connMgr;
    private volatile boolean shutdown;

    public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
        super();
        this.connMgr = connMgr;
    }

    @Override
    public void run() {
        try {
            while (!shutdown) {
                synchronized (this) {
                    wait(5000);
                    connMgr.closeExpiredConnections();
                    connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
                }
            }
        } catch (InterruptedException ex) {
            // terminate
        }
    }

    public void shutdown() {
        shutdown = true;
        synchronized (this) {
            notifyAll();
        }
    }
}

3.4 Reduce overhead when executing methods

Do not close the connection manually; let the client manage it.

One way to obtain the response content:

String res = EntityUtils.toString(response.getEntity(), "UTF-8");
EntityUtils.consume(response1.getEntity());

A better approach is to use a ResponseHandler so that the client automatically consumes the entity:

public
T execute(final HttpHost target, final HttpRequest request,
        final ResponseHandler
responseHandler, final HttpContext context)
        throws IOException, ClientProtocolException {
    Args.notNull(responseHandler, "Response handler");
    final HttpResponse response = execute(target, request, context);
    final T result;
    try {
        result = responseHandler.handleResponse(response);
    } catch (final Exception t) {
        final HttpEntity entity = response.getEntity();
        try { EntityUtils.consume(entity); } catch (final Exception t2) {
            this.log.warn("Error consuming content after an exception.", t2);
        }
        if (t instanceof RuntimeException) throw (RuntimeException) t;
        if (t instanceof IOException) throw (IOException) t;
        throw new UndeclaredThrowableException(t);
    }
    final HttpEntity entity = response.getEntity();
    EntityUtils.consume(entity);
    return result;
}

The consume method simply closes the input stream if the entity is streaming:

public static void consume(final HttpEntity entity) throws IOException {
    if (entity == null) return;
    if (entity.isStreaming()) {
        InputStream instream = entity.getContent();
        if (instream != null) {
            instream.close();
        }
    }
}

4. Other considerations

4.1 HttpClient timeout settings

Configure connection timeout and socket timeout separately:

HttpParams params = new BasicHttpParams();
int CONNECTION_TIMEOUT = 2 * 1000; // 2 s
int SO_TIMEOUT = 2 * 1000; // 2 s
long CONN_MANAGER_TIMEOUT = 500L;
params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, CONNECTION_TIMEOUT);
params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, SO_TIMEOUT);
params.setLongParameter(ClientPNames.CONN_MANAGER_TIMEOUT, CONN_MANAGER_TIMEOUT);
params.setBooleanParameter(CoreConnectionPNames.STALE_CONNECTION_CHECK, true);
httpClient.setHttpRequestRetryHandler(new DefaultHttpRequestRetryHandler(0, false));

4.2 Nginx keep‑alive settings

If Nginx is in front of the service, configure keepalive_timeout , keepalive_requests , and the upstream keepalive parameters accordingly.

By applying the above steps, the HttpClient implementation can handle high concurrency efficiently, reducing average latency from 250 ms to roughly 80 ms.