Understanding Idempotent Design: Concepts, Scenarios, and Eight Practical Implementation Strategies

1. What Is Idempotency?

Idempotency is a concept originating from mathematics and computer science. In mathematics it is expressed as f(x) = f(f(x)); for example, the absolute‑value function abs(x) = abs(abs(x)) is idempotent. In computing, a request is idempotent when executing it once or many times yields the same side‑effects and result.

2. Why Idempotency Is Needed

Consider a money‑transfer operation where the downstream service times out. Without idempotent control, a retry could cause a duplicate transfer. If the downstream system enforces idempotency, retries are safe: the transfer either succeeds once or is ignored on subsequent attempts. Similar problems appear in message‑queue consumption, rapid form submissions, and other duplicate‑request scenarios.

3. How to Handle Interface Timeouts

When a downstream call times out, two common solutions exist:

Query the downstream system for the transaction record; if it succeeded, proceed, otherwise treat it as a failure.

Require the downstream interface itself to be idempotent, allowing the upstream system to retry safely.

In message‑queue duplicate‑consumption cases, the second solution (idempotent downstream) is preferred.

4. Designing Idempotency

All idempotent designs rely on a globally unique identifier (UID) for each request. The UID can be enforced via unique indexes, primary keys, or explicit lock mechanisms.

4.1 Generating a Global Unique ID

Common approaches include:

Using UUID (simple but large and non‑sequential).

Using the Snowflake algorithm ( Snowflake IDs) which produces 64‑bit, time‑ordered IDs.

Using specialized generators such as Baidu's UidGenerator or Meituan's Leaf.

4.2 Basic Idempotent Flow

The process is: store the request together with its UID, check the storage before processing; if the UID already exists, return the previous result, otherwise handle the request and record the UID.

5. Eight Practical Idempotent Solutions

5.1 Select + Insert + Primary/Unique‑Key Conflict

First query the business‑flow table by bizSeq. If a record exists, treat it as a duplicate; otherwise insert the record. If the insert raises a duplicate‑key exception, catch it and return success.

/**
 * Idempotent handling
 */
Rsp idempotent(Request req) {
    Object requestRecord = selectByBizSeq(bizSeq);
    if (requestRecord != null) {
        // duplicate request
        log.info("Duplicate request, bizSeq: {}", bizSeq);
        return rsp;
    }
    try {
        insert(req);
    } catch (DuplicateKeyException e) {
        // duplicate request
        log.info("Primary key conflict, duplicate request, bizSeq: {}", bizSeq);
        return rsp;
    }
    // normal processing
    dealRequest(req);
    return rsp;
}

5.2 Direct Insert + Primary/Unique‑Key Conflict

Skip the initial select and directly insert; a duplicate‑key exception indicates a repeated request.

/**
 * Idempotent handling
 */
Rsp idempotent(Request req) {
    try {
        insert(req);
    } catch (DuplicateKeyException e) {
        log.info("Primary key conflict, duplicate request, bizSeq: {}", bizSeq);
        return rsp;
    }
    dealRequest(req);
    return rsp;
}

5.3 State‑Machine Idempotency

Use a status field (e.g., 0‑pending, 1‑processing, 2‑success, 3‑failure). An UPDATE that changes status from 1 to 2 succeeds only once; subsequent attempts affect zero rows and are ignored.

int rows = update transfr_flow set status=2 where biz_seq='666' and status=1;
if (rows == 1) {
    // process request
} else {
    // duplicate or already processed
}

5.4 Separate Anti‑Duplicate Table

Maintain a dedicated table keyed by the UID. Inserting the UID succeeds only once; a conflict means the request has already been handled.

5.5 Token (One‑Time Token) Mechanism

Clients first request a token; the server stores the token in Redis with an expiration. Subsequent requests present the token; the server deletes it atomically (e.g., redis.del(token)) and proceeds only if the delete succeeds.

5.6 Pessimistic Lock (SELECT FOR UPDATE)

Within a transaction, lock the target row (e.g., an order) using SELECT ... FOR UPDATE. If the row’s status is not “processing”, return early; otherwise update and commit. This prevents concurrent processing but may cause contention.

5.7 Optimistic Lock

Add a version column to the table. When updating, include the current version in the WHERE clause (e.g., WHERE id='666' AND version=1). If the update affects one row, increment the version and proceed; otherwise treat it as a duplicate.

5.8 Distributed Lock

Acquire a lock in Redis (or ZooKeeper) using a command such as SET key value NX PX ttl where the key is the request’s UID. If the lock is obtained, execute the business logic; otherwise skip the request.

6. HTTP Idempotency

HTTP methods differ in idempotency:

GET, HEAD, OPTIONS, DELETE, and PUT are idempotent.

POST is not idempotent because repeated submissions create distinct resources.

Reference

“弹力设计篇之‘幂等性设计’” – https://time.geekbang.org/column/article/4050