Spring Boot Retry Pitfall: Timeout ≠ Failure — Idempotency & Concurrency Fixes

Problem: Retry After Timeout Creates Duplicate Orders

The author describes a real order-synchronization issue: after creating an order locally, the system pushes it to a downstream service. Normally the call returns in tens of milliseconds, but occasional network jitter, connection timeout, or HTTP 503 occurs. The initial code simply logged the failure. Later a retry loop was added (three attempts with no delay), which seemed more reliable but soon produced duplicate orders in production.

The root cause: the first request actually succeeded on the third-party side (INSERT + COMMIT), but the response was lost on the network. The client saw a ReadTimeoutException and immediately retried, causing a second order creation.

First request:  create order A
Second request: create order B

Why Naive Retry Fails

The core difficulty is not retry mechanics but deciding which exceptions are retryable, which operations must not be retried, and how to guarantee business logic executes exactly once after a retry . The article shows typical copy-pasted retry blocks scattered across services (order sync, refund, logistics, inventory, SMS, coupon) with inconsistent retry counts, sleep intervals, and overly broad catch (Exception e) that retries even on programming errors.

Spring Framework 7 Resilience Annotations

Spring Framework 7 introduces @Retryable and @ConcurrencyLimit in the core framework. Enable them with:

@Configuration
@EnableResilientMethods
public class ResilienceConfiguration {}

A minimal usage:

@Service
public class RemoteOrderService {
    @Retryable
    public void pushOrder(OrderPushRequest request) {
        remoteOrderClient.createOrder(request);
    }
}

Defaults: max 3 retries (4 total executions), 1-second fixed interval. However, the author stresses that the first question is not "how many retries" but "which exceptions deserve a retry" .

Classify Retryable Exceptions

Third-party calls may throw:

ConnectTimeoutException
SocketTimeoutException
HTTP 429
HTTP 503

HTTP 400
HTTP 401
HTTP 403

The first group are transient; the second are permanent (bad request, auth failure). Retrying a 400 Bad Request 100 times still fails and wastes downstream capacity. The solution: wrap transient failures into a custom exception and retry only that:

public class RemoteTemporaryException extends RuntimeException { ... }

@Component
@RequiredArgsConstructor
public class RemoteOrderClient {
    private final RestClient restClient;
    public RemoteOrderResponse createOrder(OrderPushRequest request) {
        try {
            return restClient.post()
                .uri("/api/orders")
                .body(request)
                .retrieve()
                .body(RemoteOrderResponse.class);
        } catch (ResourceAccessException e) {
            throw new RemoteTemporaryException("第三方订单接口网络异常", e);
        }
    }
}

@Retryable(includes = RemoteTemporaryException.class, maxRetries = 2)
public void pushOrder(OrderPushRequest request) {
    remoteOrderClient.createOrder(request);
}

Now the contract is explicit: only temporary network errors trigger retry; business errors, validation errors, and auth failures do not.

Idempotency: The Missing Piece

Even with correct exception classification, duplicate orders persist if the operation is not idempotent. For side-effecting calls (create order, charge, refund, issue coupon, create payment, deduct inventory) retry alone is unsafe . The author's fix: generate a stable requestId per business operation (not per attempt) and send it as an idempotency key.

public record OrderPushRequest(
    String requestId,
    Long orderId,
    BigDecimal amount
) {}

Creation: String requestId = "ORDER_PUSH_" + order.getId(); — not a new UUID per retry. If each retry generated a new UUID, the downstream would see three distinct requests. With a stable key, the downstream can create a unique index:

CREATE UNIQUE INDEX uk_request_id ON remote_order(request_id);

Now duplicate HTTP POSTs result in a single business execution.

When the Third Party Lacks Idempotency Support

Many legacy ERPs have no idempotency key. The pattern changes to query-before-retry :

public void pushOrder(Order order) {
    try {
        remoteOrderClient.createOrder(order);
    } catch (RemoteTimeoutException e) {
        boolean exists = remoteOrderClient.existsByBizOrderNo(order.getOrderNo());
        if (exists) return;
        throw new RemoteTemporaryException("远程订单不存在，可以重新尝试");
    }
}

@Retryable(includes = RemoteTemporaryException.class, maxRetries = 2)
public void execute(Order order) {
    pushOrder(order);
}

The author's rule of thumb:

GET query interfaces          → usually safe to retry
POST create interfaces        → confirm idempotency first
Payment, refund, inventory    → never retry blindly without idempotency

Concurrency Limit to Prevent Cascading Failure

When a downstream slows from 50 ms to several seconds, threads pile up. Combined with retries, a modest incoming load (1000 requests) becomes 3000 concurrent calls, amplifying the outage. Spring Framework 7's @ConcurrencyLimit caps simultaneous executions:

@ConcurrencyLimit(20)
public RemoteOrderResponse callRemote(OrderPushRequest request) {
    return remoteOrderClient.createOrder(request);
}

This replaces manual Semaphore boilerplate. The author sets the limit based on known downstream capacity (e.g., ERP degrades above 50 concurrent calls).

Transaction and Retry Interaction

Spring's proxy chain applies retry outside the transaction:

Retry
  ↓
Transaction
  ↓
Business method

Each retry starts a new transaction. This is ideal for transient database errors (deadlock, momentary unavailability): the first transaction rolls back, the retry begins a fresh transaction and commits. Manual for loops inside @Transactional methods often leave the transaction in an ambiguous state.

Final Layered Architecture

The production code separates three concerns:

OrderPushService   → business logic (load order, build request)
OrderGateway       → remote-call strategy (@Retryable, @ConcurrencyLimit)
RemoteOrderClient  → HTTP communication (RestClient, exception translation)

Changes to retry count or concurrency limit only touch OrderGateway; HTTP endpoint changes only touch RemoteOrderClient; business changes only touch OrderPushService. No more hunting for Thread.sleep(1000) across dozens of services.

Key Questions Before Adding Retry

The author now asks five questions before applying @Retryable:

Did the first failure actually mean the operation didn't execute?

Is the operation idempotent?

Which exceptions are truly transient?

Will repeated retries overload the downstream?

What is the fallback when all retries exhaust?

Answering these makes the retry code itself trivial. Spring Framework 7 solves "how to retry"; developers must still decide "whether to retry" — especially in order, payment, refund, and inventory systems where a timeout often means success, not failure.