How to Cut HttpClient Latency by 70% with Connection Pooling and Keep-Alive

HttpClient optimization ideas: pooling, keep-alive, reuse HttpClient and HttpGet, proper configuration parameters (max concurrent requests, timeouts, retry count), asynchronous execution, and reading source code.

1. Background

Our service calls an HTTP API millions of times per day. Using HttpClient, the average request time was 250 ms. After optimization it dropped to about 80 ms, reducing two‑thirds of the latency and eliminating thread‑exhaustion alerts.

2. Analysis

The original implementation created a new HttpClient and HttpPost for each request, copied the response entity into a string, and manually closed the response and client, causing high overhead.

2.1 Repeated HttpClient creation

HttpClient is thread‑safe; a single shared instance should be used.

2.2 Repeated TCP connection creation

Each request performed a full TCP handshake and teardown, adding several milliseconds. Using keep‑alive allows connection reuse.

2.3 Duplicate entity handling

Original code:

HttpEntity entity = httpResponse.getEntity();

String response = EntityUtils.toString(entity);

This duplicated the content into a string while the original HttpResponse still held the entity, leading to extra memory consumption and the need to manually close the connection.

3. Implementation

Three main steps: singleton client, connection‑pooling with keep‑alive, and better response handling.

3.1 Keep‑alive strategy

ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {
    @Override
    public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
        HeaderElementIterator it = new BasicHeaderElementIterator(response.headerIterator(HTTP.CONN_KEEP_ALIVE));
        while (it.hasNext()) {
            HeaderElement he = it.nextElement();
            String param = he.getName();
            String value = he.getValue();
            if (value != null && param.equalsIgnoreCase("timeout")) {
                return Long.parseLong(value) * 1000;
            }
        }
        return 60 * 1000; // default 60 seconds
    }
};

3.2 PoolingHttpClientConnectionManager

PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(500);
connectionManager.setDefaultMaxPerRoute(50); // adjust per business needs

3.3 Build HttpClient

httpClient = HttpClients.custom()
    .setConnectionManager(connectionManager)
    .setKeepAliveStrategy(myStrategy)
    .setDefaultRequestConfig(RequestConfig.custom().setStaleConnectionCheckEnabled(true).build())
    .build();

Note: Using setStaleConnectionCheckEnabled is not recommended. A better approach is to run a background thread that periodically calls closeExpiredConnections and closeIdleConnections .

public static class IdleConnectionMonitorThread extends Thread {
    private final HttpClientConnectionManager connMgr;
    private volatile boolean shutdown;
    public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
        super();
        this.connMgr = connMgr;
    }
    @Override
    public void run() {
        try {
            while (!shutdown) {
                synchronized (this) {
                    wait(5000);
                    connMgr.closeExpiredConnections();
                    connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
                }
            }
        } catch (InterruptedException ex) {
            // terminate
        }
    }
    public void shutdown() {
        shutdown = true;
        synchronized (this) {
            notifyAll();
        }
    }
}

3.4 Reduce overhead when executing methods

Do not close the connection manually.

A convenient way is to use a ResponseHandler which automatically consumes the entity:

public <T> T execute(final HttpHost target, final HttpRequest request,
        final ResponseHandler<? extends T> responseHandler, final HttpContext context)
        throws IOException, ClientProtocolException {
    Args.notNull(responseHandler, "Response handler");
    final HttpResponse response = execute(target, request, context);
    final T result;
    try {
        result = responseHandler.handleResponse(response);
    } catch (final Exception t) {
        final HttpEntity entity = response.getEntity();
        try {
            EntityUtils.consume(entity);
        } catch (final Exception t2) {
            log.warn("Error consuming content after an exception.", t2);
        }
        if (t instanceof RuntimeException) {
            throw (RuntimeException) t;
        }
        if (t instanceof IOException) {
            throw (IOException) t;
        }
        throw new UndeclaredThrowableException(t);
    }
    final HttpEntity entity = response.getEntity();
    EntityUtils.consume(entity);
    return result;
}

4. Other Settings

4.1 Timeout configuration

HttpParams params = new BasicHttpParams();
// connection timeout (ms)
Integer CONNECTION_TIMEOUT = 2 * 1000;
// socket timeout (ms)
Integer SO_TIMEOUT = 2 * 1000;
Long CONN_MANAGER_TIMEOUT = 500L;
params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, CONNECTION_TIMEOUT);
params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, SO_TIMEOUT);
params.setLongParameter(ClientPNames.CONN_MANAGER_TIMEOUT, CONN_MANAGER_TIMEOUT);
params.setBooleanParameter(CoreConnectionPNames.STALE_CONNECTION_CHECK, true);
httpClient.setHttpRequestRetryHandler(new DefaultHttpRequestRetryHandler(0, false));

4.2 Nginx keep‑alive

Configure keepalive_timeout and keepalive_requests on the client side, and keepalive on the upstream side.

With these settings the request time fell from 250 ms to around 80 ms.

Maven dependency:

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.6</version>
</dependency>

Key code snippets (static fields, initialization block, and utility methods):

private static final CredentialsProvider credsProvider = new BasicCredentialsProvider();
private static final CloseableHttpClient httpclient;
private static final HttpGet httpget;
private static final RequestConfig requestConfig;
private static final ResponseHandler<String> responseHandler;
private static final ObjectMapper mapper = new ObjectMapper();
static {
    System.setProperty("http.maxConnections", "50");
    System.setProperty("http.keepAlive", "true");
    credsProvider.setCredentials(new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, AuthScope.ANY_REALM),
            new UsernamePasswordCredentials("", ""));
    httpclient = HttpClients.custom()
        .useSystemProperties()
        .setRetryHandler(new DefaultHttpRequestRetryHandler(3, true))
        .setDefaultCredentialsProvider(credsProvider)
        .build();
    httpget = new HttpGet();
    requestConfig = RequestConfig.custom()
        .setContentCompressionEnabled(true)
        .setSocketTimeout(100)
        .setAuthenticationEnabled(true)
        .setConnectionRequestTimeout(100)
        .setConnectTimeout(100)
        .build();
    httpget.setConfig(requestConfig);
    responseHandler = new BasicResponseHandler();
}
public static String getResponse(String url) throws IOException {
    HttpGet get = new HttpGet(url);
    return httpclient.execute(get, responseHandler);
}
public static JSONObject getUrl(String url) throws Exception {
    try {
        httpget.setURI(URI.create(url));
        String response = httpclient.execute(httpget, responseHandler);
        return JSONObject.fromObject(response);
    } catch (IOException e) {
        e.printStackTrace();
    }
    return null;
}
public static JsonNode getUrl2(String url) {
    try {
        httpget.setURI(URI.create(url));
        String response = httpclient.execute(httpget, responseHandler);
        return mapper.readTree(response);
    } catch (IOException e) {
        e.printStackTrace();
    }
    return null;
}
public static com.alibaba.fastjson.JSONObject getUrl3(String url) {
    try {
        httpget.setURI(URI.create(url));
        String response = httpclient.execute(httpget, responseHandler);
        return com.alibaba.fastjson.JSONObject.parseObject(response);
    } catch (IOException e) {
        e.printStackTrace();
    }
    return null;
}