HTTP Retry Strategies for Offline Store Systems: Simple Loop, Apache HttpClient, and Asynchronous MQ-Based Retries

In offline store system development, many components need to communicate with third‑party services via HTTP, such as synchronizing product data to electronic price tags or sending staff punch‑in records to an external EHR system. Network instability often leads to timeouts, making a robust HTTP retry strategy essential.

Simple retry can be implemented by wrapping the request in a loop that repeats until success or a maximum number of attempts is reached. Example code:

int retryTimes = 3;
for (int i = 0; i < retryTimes; i++) {
    try {
        // request the interface
        break;
    } catch (Exception e) {
        // handle exception
    }
}

While straightforward, this approach retries on any exception without distinguishing between recoverable and non‑recoverable errors.

Apache HttpClient retry mechanism is examined next. The typical client creation steps are:

CloseableHttpClient httpClient = HttpClientBuilder.create().build();
HttpGet httpGet = new HttpGet("url");
CloseableHttpResponse response = httpClient.execute(httpGet);
HttpEntity entity = response.getEntity();

During client construction, HttpClientBuilder.build() checks the automaticRetriesDisabled flag and, if not disabled, wraps the execution chain with RetryExec using a HttpRequestRetryHandler . If no handler is supplied, DefaultHttpRequestRetryHandler.INSTANCE is used.

The core RetryExec logic retries the request when an IOException occurs and the configured RetryHandler decides to retry. HttpClient distinguishes between IOException (considered recoverable) and ClientProtocolException (non‑recoverable).

DefaultHttpRequestRetryHandler defines three fields:

retryCount : maximum retry attempts (default 3).

requestSentRetryEnabled : whether to retry after the request has been sent (default false).

nonRetriableClasses : a set of exception classes that should never be retried, including InterruptedIOException , UnknownHostException , ConnectException , and SSLException .

The retryRequest method first checks the execution count, then whether the exception belongs to nonRetriableClasses , then whether the request is idempotent, and finally whether the request has already been sent and requestSentRetryEnabled is true.

Consequently, the default handler will not retry SocketTimeoutException (a subclass of InterruptedIOException) and will not retry non‑idempotent POST requests.

Asynchronous retry via message queue leverages the fact that many store workflows already emit MQ messages. By introducing a consumer that performs the HTTP call, failures can be handled by the MQ's own retry mechanism (e.g., RocketMQ's back‑off retries), isolating the business logic from transient third‑party outages.

Final hybrid solution combines the asynchronous MQ retry with a customized HttpClient RetryHandler to address the shortcomings of the default strategy. The custom handler code is:

public boolean retryRequest(IOException exception, int executionCount, HttpContext context) {
    if (executionCount > this.retryCount) {
        RequestLine requestLine = null;
        if (context instanceof HttpClientContext) {
            requestLine = ((HttpClientContext)context).getRequest().getRequestLine();
        }
        return false;
    } else if (exception instanceof NoHttpResponseException) {
        return true;
    } else if (exception instanceof SSLHandshakeException) {
        return false;
    } else if (exception instanceof InterruptedIOException) {
        return true;
    } else if (exception instanceof UnknownHostException) {
        return false;
    } else if (exception instanceof ConnectTimeoutException) {
        return false;
    } else if (exception instanceof SSLException) {
        return false;
    } else {
        HttpClientContext clientContext = HttpClientContext.adapt(context);
        HttpRequest request = clientContext.getRequest();
        return !(request instanceof HttpEntityEnclosingRequest);
    }
}

When building the client, the custom handler is injected:

CloseableHttpClient httpClient = HttpClientBuilder.create()
    .setRetryHandler(StoreRequestRetryHandler.INSTANCE)
    .build();

This configuration solves two problems: it enables retries for POST requests when appropriate and allows retrying SocketTimeoutException . The overall flow is:

The MQ consumer uses HttpClient with a 3‑retry loop.

If all three attempts fail, the message consumption fails and RocketMQ retries the message (default 16 times).

The combined maximum retry count reaches 51 attempts, providing both synchronous real‑time retries and asynchronous back‑off retries.

After adopting this hybrid approach, the team observed no further complaints about delayed price‑tag updates or missing punch‑in records.

Author: Hou Wanxing, Backend Development Engineer at ZhaiZhai Store Business.