Skip to content
API Idempotency Deep Dive: 5 Patterns, Code & Interview Guide

API Idempotency Deep Dive: 5 Patterns, Code & Interview Guide

This comprehensive guide covers five proven idempotency patterns for distributed systems—database unique indexes, token mechanisms, distributed locks, state machines, and deduplication tables—with code implementations, Lua-script atomicity, Spring AOP aspects, optimistic locking, and a simulated interview covering race conditions, clock drift, and hotspot mitigation.

Rain's Path to Java Mastery
Rain's Path to Java Mastery
Rain's Path to Java Mastery

Core Concepts

API Idempotency means calling an endpoint once or multiple times produces identical effects—no data anomalies or business errors from duplicate calls.

Why Idempotency Is Mandatory

Network jitter causing duplicate requests

Frontend button double-clicks

Message queue duplicate consumption

Service retry mechanisms (Feign, Dubbo retries)

Data consistency in distributed systems

Five Main Solutions (Ordered by Complexity)

1. Database Unique Index (Simplest, Most Common)

Principle: Use DB unique constraint to prevent duplicate inserts.

Implementation: Add request_id unique index on business table.

Applicable: Insert operations (order creation, payment records).

// Check if already processed
Order order = orderMapper.selectByRequestId(requestId);
if (order != null) {
    return Result.success("Order already processed");
}
// Execute business logic
try {
    orderMapper.insert(order);
} catch (DuplicateKeyException e) {
    // Unique index conflict → already processed
    return Result.success("Order already processed");
}

2. Token Mechanism (Frontend-Backend Cooperation)

Key: Use Redis DEL command atomic check-and-delete.

Pros: Intercepts before business logic.

Cons: Requires frontend cooperation, extra round-trip.

3. Distributed Lock (Redis/ZooKeeper)

Principle: Only one request acquires lock at a time.

Implementation: Redis SETNX+EXPIRE (recommend Redisson).

RLock lock = redissonClient.getLock("order:" + requestId);
try {
    if (lock.tryLock(0, 10, TimeUnit.SECONDS)) {
        // Execute business logic
    } else {
        return Result.error("Request processing, retry later");
    }
} finally {
    if (lock.isHeldByCurrentThread()) {
        lock.unlock();
    }
}

4. State Machine Idempotency (Business-Layer Control)

Principle: Leverage unidirectional business state transitions.

Example: Order status flow: UNPAID → PAID → SHIPPED → COMPLETED.

Implementation: Add state condition on update.

UPDATE `order` SET status = 2 WHERE id = #{orderId} AND status = 1;

5. Global Unique ID + Deduplication Table

Principle: Separate deduplication table stores processed request IDs.

Applicable: Cross-service, cross-database complex workflows.

Pros: No intrusion on business table schema.

Solution Comparison

Unique Index: Low complexity, high performance, insert-only, cannot prevent updates.

Token: Medium complexity, medium performance, frontend submissions, early interception but needs frontend sync.

Distributed Lock: Medium complexity, medium performance, most business scenarios, universal but deadlock risk and performance overhead.

State Machine: Low complexity, high performance, stateful workflows, best performance but only where state is explicit.

Deduplication Table: High complexity, lower performance, cross-service complex workflows, non-intrusive but adds DB pressure.

Best Practices by Scenario

Simple inserts → Unique index.

Frontend form submissions → Token + button disable.

MQ consumption → Unique index + consumer idempotency.

Update operations → State machine + optimistic locking.

Complex distributed workflows → Global unique ID + distributed lock.

Core Principles

Idempotency is a distributed system baseline—design for it upfront.

No silver bullet; choose/combine per scenario.

Prefer database-level guarantees for highest reliability.

Avoid over-engineering; simple cases don't need complex solutions.

All external calls (third-party APIs, MQ) must be idempotent.

Technical Highlights & Production-Grade Code

1. Redis Token with Lua Script (Atomicity)

❌ Wrong: Check-then-delete has race condition.

// Wrong: two requests both see token, both proceed
if (redisTemplate.hasKey(token)) {
    redisTemplate.delete(token);
    // execute business
}

✅ Correct: Lua script guarantees atomic check-and-delete.

public boolean checkAndDeleteToken(String token) {
    String luaScript = """
        if redis.call('EXISTS', KEYS[1]) == 1 then
            return redis.call('DEL', KEYS[1])
        else
            return 0
        end
        """;
    Long result = redisTemplate.execute(
        new DefaultRedisScript<>(luaScript, Long.class),
        Collections.singletonList(token)
    );
    return result != null && result == 1;
}

Highlight: Redis single-threaded Lua execution eliminates race conditions; 10x faster than transactions.

2. Spring AOP Global Idempotent Aspect (Zero Intrusion)

Annotation-driven, zero business code pollution.

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface Idempotent {
    int expireTime() default 300;
    String prefix() default "idempotent:";
}

@Aspect
@Component
public class IdempotentAspect {
    @Autowired
    private RedissonClient redissonClient;

    @Around("@annotation(idempotent)")
    public Object around(ProceedingJoinPoint point, Idempotent idempotent) throws Throwable {
        String requestId = ((ServletRequestAttributes) RequestContextHolder.getRequestAttributes())
            .getRequest().getHeader("X-Request-Id");
        String lockKey = idempotent.prefix() + requestId;
        RLock lock = redissonClient.getLock(lockKey);
        try {
            if (!lock.tryLock(0, idempotent.expireTime(), TimeUnit.SECONDS)) {
                return Result.error("Duplicate submission");
            }
            return point.proceed();
        } finally {
            if (lock.isHeldByCurrentThread()) lock.unlock();
        }
    }
}

// Usage
@PostMapping("/createOrder")
@Idempotent(expireTime = 600)
public Result createOrder(@RequestBody OrderDTO orderDTO) {
    orderService.create(orderDTO);
    return Result.success();
}

Highlights: Zero business intrusion; Redisson watchdog auto-renewal solves lock expiry; configurable TTL and key prefix.

3. State Machine + Optimistic Locking (High-Concurrency Updates)

@Transactional(rollbackFor = Exception.class)
public boolean payOrder(Long orderId, String paySerialNo) {
    Order order = orderMapper.selectById(orderId);
    if (order == null) throw new BusinessException("Order not found");
    // State machine pre-check
    if (order.getStatus() != OrderStatus.UNPAID.getCode()) {
        log.warn("Invalid status for payment: orderId={}, status={}", orderId, order.getStatus());
        return true; // idempotent success
    }
    // Optimistic lock update (version + state condition)
    int rows = orderMapper.updateStatus(
        orderId,
        OrderStatus.PAID.getCode(),
        order.getVersion(),
        OrderStatus.UNPAID.getCode()
    );
    return rows > 0;
}

// Mapper SQL
@Update("""
    UPDATE `order` 
    SET status = #{newStatus}, version = version + 1, pay_serial_no = #{paySerialNo}
    WHERE id = #{orderId} AND version = #{oldVersion} AND status = #{oldStatus}
    """)
int updateStatus(@Param("orderId") Long orderId,
                 @Param("newStatus") Integer newStatus,
                 @Param("oldVersion") Integer oldVersion,
                 @Param("oldStatus") Integer oldStatus,
                 @Param("paySerialNo") String paySerialNo);

Highlight: State machine + version dual guarantee prevents duplicate updates and solves ABA problem.

Production Challenges & Solutions

Redis Token Race Condition: High concurrency passes hasKey check → duplicate execution. Fix: Lua script atomic "exists-then-delete".

Distributed Lock Expiry Renewal: Business exceeds lock TTL → premature release. Fix: Redisson watchdog renews every 1/3 TTL until unlock.

Unique Index Hotspot: High concurrency conflicts spike DB CPU. Fix: 1) Redis pre-check cache; 2) INSERT ... ON DUPLICATE KEY UPDATE to avoid exceptions.

Inconsistent Return Values: First request succeeds but times out; retry returns "duplicate" → frontend shows failure. Fix: Cache first response in Redis keyed by requestId; repeat requests return cached result.

MQ Duplicate Consumption: Retries cause multi-consumption. Fix: Message carries global messageId; check deduplication table before consume; update table after success in same transaction.

Cross-Service Idempotency: Distributed transaction partial failure → retry inconsistency. Fix: Global transaction ID ( XID) propagated across call chain; each service idempotent on XID.

Deduplication Table Pressure: Large data volume slows queries. Fix: Time-based sharding, archive cold data, hot/cold separation; retain 30 days online.

Deep Dive: Duplicate Request Return Consistency

Scenario: Frontend submits order → backend succeeds → network timeout → user retries → backend returns "duplicate" → frontend shows failure but order exists.

Solution Code:

public Result createOrder(String requestId, OrderDTO orderDTO) {
    // 1. Check cached response
    String cacheKey = "result:" + requestId;
    String cacheResult = redisTemplate.opsForValue().get(cacheKey);
    if (cacheResult != null) {
        return JSON.parseObject(cacheResult, Result.class);
    }
    // 2. Idempotent check
    if (!idempotentService.check(requestId)) {
        return Result.error("Duplicate submission");
    }
    try {
        // 3. Execute business
        Order order = orderService.create(orderDTO);
        Result result = Result.success(order);
        // 4. Cache response (1h TTL)
        redisTemplate.opsForValue().set(cacheKey, JSON.toJSONString(result), 1, TimeUnit.HOURS);
        return result;
    } catch (Exception e) {
        // Business exception → don't cache, allow retry
        throw e;
    }
}

Simulated Interview Highlights

Concept Alignment

Idempotency = same side-effects for 1 or N calls. Critical because distributed networks are unreliable (timeouts, retries, duplicate deliveries).

HTTP methods: GET, PUT, DELETE are idempotent per spec; POST is not. Most write ops use POST → need application-level idempotency patches.

Solution Deep Dive

Unique Index: Global unique business ID + unique constraint. Catch DuplicateKeyException. Pitfall: only prevents inserts, not updates; sharding requires global unique routing.

Token: Server issues token → Redis; client submits with token; atomic Lua check-and-delete. Pitfall: frontend cooperation needed; token expiry; atomicity of delete.

State Machine: Update with WHERE status = 'UNPAID'. First update affects 1 row, subsequent 0. Pitfall: concurrent same-state updates may overwrite → add optimistic lock (version).

Distributed Lock: Key = business identifier (e.g., lock:submit_order:userId+cartId). Pitfall: must set TTL, verify ownership on unlock (Lua), reduces concurrency—only for moderate-load complex aggregates.

Combination Example (Payment Callback): 1) Unique index on out_trade_no as safety net; 2) Redis distributed lock on out_trade_no to serialize concurrent callbacks; 3) State machine constraint ensuring only UNPAID→PAID transition.

Advanced Follow-ups

Global Unique ID Generation: Snowflake algorithm. Clock drift handling: 1) throw exception wait for catch-up; 2) use last max timestamp; 3) external segment-based allocator (Meituan Leaf) decoupled from time.

Select-Then-Insert Race: Fundamentally broken. Correct pattern: "optimistic insert" — attempt insert directly; on unique conflict, query existing record and return it.

Upstream Idempotency Key Contract: Require callers to send same idempotency key (e.g., bizId) on retries; key must be globally unique and propagated through entire call chain.

Unique Index Hotspot Mitigation: 1) Shard unique key by suffix; 2) Bloom filter pre-filters 80% duplicates; 3) For non-financial, downgrade to Redis-based eventual consistency.

Live Coding: Two Classic Patterns

Token Mechanism (Redis + Lua)

-- token_check.lua
-- KEYS[1]: token key, e.g., "submit_token:abc123"
-- ARGV[1]: token value from frontend
if redis.call('GET', KEYS[1]) == ARGV[1] then
    return redis.call('DEL', KEYS[1])
else
    return 0
end
public boolean tryReleaseToken(String tokenKey, String tokenValue) {
    String lua = "if redis.call('GET', KEYS[1]) == ARGV[1] then " +
                 "return redis.call('DEL', KEYS[1]) else return 0 end";
    Long result = jedis.eval(lua, Collections.singletonList(tokenKey),
                              Collections.singletonList(tokenValue));
    return result != null && result == 1L;
}

Highlight: Single network round-trip completes compare-and-delete; zero concurrency window.

State Machine + Optimistic Lock (Anti-Duplicate Update)

<update id="updateOrderStatusPaid">
    UPDATE `order`
    SET status = 'PAID',
        version = version + 1,
        update_time = NOW()
    WHERE order_id = #{orderId}
      AND status = 'UNPAID'
      AND version = #{version} <!-- optimistic lock -->
</update>
public void payOrder(String orderId) {
    Order order = orderMapper.selectByOrderId(orderId);
    if (order == null) throw new BizException("Order not found");
    int rows = orderMapper.updateOrderStatusPaid(orderId, order.getVersion());
    if (rows == 0) {
        Order latest = orderMapper.selectByOrderId(orderId);
        if ("PAID".equals(latest.getStatus())) {
            return; // idempotent success
        }
        throw new BizException("Order state abnormal, refresh and retry");
    }
}

Highlight: WHERE clause uses status for business idempotency and version for concurrency control—single DB round-trip solves both.

Technical Difficulties Summary

Concurrency Atomicity: "Check-then-act" windows → solved by unique index optimistic insert, Lua scripts, conditional updates.

Global Unique ID Generation: Snowflake + drift tolerance or segment-based (Leaf).

Sharding Breaks Unique Constraints: Key must contain shard key; global allocator; route by shard key at gateway.

High-Concurrency Performance: Key suffix sharding, Bloom filter pre-filter, non-financial eventual consistency downgrade.

Upstream Key Propagation: Enforce idempotent-key header; gateway rejects missing keys; trace context propagation.

Complex Orchestration Idempotency: Entry-point distributed lock serializes whole block; each sub-service uses unique index/state machine; async via transactional message table.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

Distributed SystemsRedisState Machineinterview preparationSpring AOPDistributed LocksLua ScriptingOptimistic LockingAPI Idempotencydatabase unique index
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.