How to Guarantee Single Order Processing When Payment Callbacks Fire 10 Times

Problem: Duplicate Payment Callbacks Are Normal

Payment platforms often resend the same success notification multiple times due to slow responses, network glitches, or at-least-once delivery guarantees. The author observed a real case where a single order received six callbacks within 50 seconds. The core requirement: no matter how many times the callback arrives, the business effect (order status change, inventory deduction, coupon issuance, etc.) must happen exactly once.

Naive Approach and Its Concurrency Flaw

A typical first implementation updates the order status and then calls downstream services without any idempotency guard:

@PostMapping("/payment/callback")
public String callback(@RequestBody PaymentNotify notify) {
    Order order = orderRepository.findByOrderNo(notify.orderNo());
    order.setStatus("PAID");
    orderRepository.save(order);
    inventoryService.reduce(order.getSkuId(), order.getQuantity());
    couponService.issue(order.getUserId());
    messageService.sendPaymentSuccess(order.getUserId());
    return "SUCCESS";
}

This works for a single callback but fails when duplicates arrive. A common quick fix is to check the current status before processing:

if ("PAID".equals(order.getStatus())) {
    return "SUCCESS";
}
order.setStatus("PAID");
orderRepository.save(order);

However, under high concurrency two threads can both read UNPAID and then both proceed to execute the business logic, causing double deductions or double coupon issuance. This is a classic check-then-act race condition.

Solution 1: State Machine Idempotency with Conditional Update

Instead of reading then writing, let the database decide which thread wins by using a conditional UPDATE that only succeeds when the order is still in the expected state.

Order table schema (simplified):

CREATE TABLE orders (
    id BIGINT PRIMARY KEY AUTO_INCREMENT,
    order_no VARCHAR(64) NOT NULL,
    user_id BIGINT NOT NULL,
    amount DECIMAL(18,2) NOT NULL,
    status VARCHAR(32) NOT NULL,
    paid_at DATETIME,
    transaction_id VARCHAR(128),
    created_at DATETIME NOT NULL,
    updated_at DATETIME NOT NULL,
    UNIQUE KEY uk_order_no(order_no)
);

Conditional update SQL:

UPDATE orders
SET status = 'PAID',
    paid_at = ?,
    transaction_id = ?,
    updated_at = NOW()
WHERE order_no = ?
  AND status = 'WAITING_PAYMENT';

Java repository using JdbcTemplate:

@Repository
public class OrderPaymentRepository {
    private final JdbcTemplate jdbcTemplate;
    public OrderPaymentRepository(JdbcTemplate jdbcTemplate) {
        this.jdbcTemplate = jdbcTemplate;
    }
    public int markPaid(String orderNo, String transactionId, LocalDateTime paidAt) {
        return jdbcUpdate.update("""
            UPDATE orders
            SET status = 'PAID',
                paid_at = ?,
                transaction_id = ?,
                updated_at = NOW()
            WHERE order_no = ?
              AND status = 'WAITING_PAYMENT'
            """,
            paidAt, transactionId, orderNo);
    }
}

If the update returns 1, this thread performed the state transition; if 0, the order was already paid, closed, missing, or in an invalid state. This eliminates the race window because the database enforces atomicity.

Solution 2: Payment Event Deduplication with Unique Key

Payment platforms provide a unique event identifier (e.g., transactionId, eventId, notifyId). Store each incoming event in a dedicated table with a unique constraint on (provider, event_id).

CREATE TABLE payment_event (
    id BIGINT PRIMARY KEY AUTO_INCREMENT,
    provider VARCHAR(32) NOT NULL,
    event_id VARCHAR(128) NOT NULL,
    order_no VARCHAR(64) NOT NULL,
    transaction_id VARCHAR(128),
    event_type VARCHAR(64) NOT NULL,
    raw_payload TEXT,
    status VARCHAR(32) NOT NULL,
    created_at DATETIME NOT NULL,
    processed_at DATETIME,
    UNIQUE KEY uk_provider_event(provider, event_id)
);

On each callback, attempt to insert the event. A DuplicateKeyException means this exact event has already been processed, so return immediately. This layer answers "Have I seen this platform event before?" while the conditional order update answers "Can this business action still be executed?" Both layers are kept because they guard different failure modes.

Solution 3: Outbox Pattern for Reliable Event Publishing

Downstream actions (coupon issuance, messaging, ERP notification) often call external HTTP services or publish to message queues. A local @Transactional boundary cannot guarantee atomicity across these external calls. If the database commits but the MQ send fails, the event is lost; if the external call succeeds but the database rolls back, the downstream effect is duplicated on retry.

The Outbox pattern solves this by writing the outgoing event into an outbox_event table within the same database transaction that updates the order status.

CREATE TABLE outbox_event (
    id VARCHAR(64) PRIMARY KEY,
    event_type VARCHAR(64) NOT NULL,
    aggregate_id VARCHAR(64) NOT NULL,
    payload TEXT NOT NULL,
    status VARCHAR(32) NOT NULL,
    retry_count INT NOT NULL DEFAULT 0,
    next_retry_at DATETIME,
    created_at DATETIME NOT NULL,
    sent_at DATETIME,
    INDEX idx_outbox_status(status, next_retry_at)
);

Transactional handler:

@Transactional
public void handle(PaymentCallback callback) {
    boolean inserted = paymentEventRepository.tryInsert(callback);
    if (!inserted) return;
    int updated = orderPaymentRepository.markPaid(
        callback.orderNo(), callback.transactionId(), callback.paidAt());
    if (updated == 0) return;
    PaymentSucceededEvent event = new PaymentSucceededEvent(
        callback.orderNo(), callback.transactionId());
    outboxRepository.insert(
        UUID.randomUUID().toString(),
        "PAYMENT_SUCCEEDED",
        callback.orderNo(),
        json.writeValueAsString(event));
}

Because the order update and outbox insert share the same transaction, only two outcomes are possible: both succeed, or both are rolled back. A separate scheduled task polls the outbox table for PENDING events, publishes them to the message broker, and marks them SENT (with retry logic, FOR UPDATE SKIP LOCKED for multi-instance safety, exponential backoff, dead-letter handling, etc.).

Downstream Consumers Must Also Be Idempotent

Even if the producer guarantees exactly-once publishing, the message broker may redeliver. Each consumer must deduplicate using a business key. Example: a membership service that grants 30-day VIP on payment success.

CREATE TABLE order_benefit (
    id BIGINT PRIMARY KEY AUTO_INCREMENT,
    order_no VARCHAR(64) NOT NULL,
    benefit_type VARCHAR(64) NOT NULL,
    user_id BIGINT NOT NULL,
    created_at DATETIME NOT NULL,
    UNIQUE KEY uk_order_benefit(order_no, benefit_type)
);

@Transactional
public void consume(PaymentSucceededEvent event) {
    try {
        benefitRepository.insert(event.orderNo(), "VIP_30_DAYS", event.userId());
    } catch (DuplicateKeyException e) {
        return; // already processed
    }
    membershipRepository.extendVip(event.userId(), 30);
}

Whether the MQ delivers the event once or ten times, the VIP extension runs exactly once.

Why Redis Locks Are Not a Silver Bullet

A distributed lock (e.g., Redisson) only prevents concurrent threads from entering the critical section simultaneously. It does not protect against a callback that arrives minutes later after the lock has been released. The lock reduces contention but the ultimate guarantee must come from database unique constraints and state machine checks.

Additional Validations Before Processing

Verify signature, merchant ID, event source, order existence, and payment status from the platform.

Check that the callback amount and currency match the order (e.g., order expects 99.00 but callback says 9.90).

private void verify(Order order, PaymentCallback callback) {
    if (!order.getAmount().equals(callback.amount())) {
        throw new PaymentVerifyException("payment amount mismatch");
    }
    if (!Objects.equals(order.getCurrency(), callback.currency())) {
        throw new PaymentVerifyException("currency mismatch");
    }
}

Return SUCCESS Quickly

Do not perform slow synchronous operations (SMS, email, PDF generation, ERP/CRM calls, logistics, recommendation systems) inside the callback request. Long processing times increase the chance the payment platform times out and retries. The callback should only do verification, deduplication, state update, outbox write, and then immediately return SUCCESS. All side effects are handled asynchronously via the outbox publisher.

Complete Callback Flow Summary

Payment Platform
    ↓
POST /payment/callback
    ↓
Verify Signature
    ↓
Payment Event Unique Key Deduplication
    ↓
UPDATE orders WHERE status = WAITING_PAYMENT
    ↓
Write outbox_event
    ↓
Commit Transaction
    ↓
Return SUCCESS

Outbox Publisher
    ↓
Message Queue
    ↓
┌─────────────────┐
│ Grant Membership │
│ Add Points       │
│ Send Notification│
│ Notify ERP       │
│ Analytics        │
└─────────────────┘
Each consumer enforces its own business idempotency

Conclusion

Idempotency is not a single annotation or a Redis lock. It is a layered defense built from:

Unique event identifiers + database unique constraints

Explicit business state machine

Conditional state updates (compare-and-set)

Transactional outbox for reliable event emission

Downstream consumer-side deduplication

These are ordinary database features, but they are what ultimately protect real-money systems when the payment platform sends the same success notification ten times — or a hundred times.