Ensuring Idempotency in Distributed Systems: Strategies and Code Samples

Concept

Idempotency means that multiple executions have the same effect as a single execution. Mathematically it can be expressed as f(x) = f(f(x)).

Problem and Scenarios

Service calls can result in three states: success, failure, and timeout. Timeout creates an unknown state that can lead to duplicate actions in the following scenarios:

Scenario 1: Creating an order – repeated clicks or page refresh may cause multiple order submissions.

Scenario 2: Redeeming a coupon – a timeout and retry may cause the coupon to be used multiple times.

Scenario 3: Deducting inventory – a timeout and retry may cause inventory to be deducted repeatedly.

Analysis and Solution

Because timeouts can cause inconsistent server state, the service interface must guarantee idempotency. Two key questions arise:

How to identify whether multiple requests are the same?

What mechanism ensures that repeated executions do not change the state?

The answer consists of two elements:

Idempotent identifier – a unique token (e.g., order number) generated by the client to recognize repeated attempts.

Uniqueness guarantee mechanism – the server checks the identifier and its associated state; if it exists, the request is considered already processed.

A generic processing flow is illustrated below:

Implementation Steps

Create a request log table that includes an idempotent identifier column.

Ensure the log data stays consistent with business processing, using transactions when necessary.

Sample pseudo‑code:

sql = "select count(*) from request_log where idempotent_id = 123;"
int count = db.select(sql);
if (count == 0) {
    // Business not executed yet
    // 1. Save request log
    insert into request_log (idempotent_id, ...) values (123, ...);
    // 2. Execute business logic
} else {
    // Business already executed, return idempotent response
}

Concurrent Scenarios

When multiple identical requests arrive simultaneously, a simple lock can prevent duplicate inserts. In distributed environments, a distributed lock is commonly used.

lock(123);
try {
    sql = "select count(*) from request_log where idempotent_id = 123;"
    int count = db.select(sql);
    if (count == 0) {
        // Save request log and execute business
        insert into request_log (idempotent_id, ...) values (123, ...);
    } else {
        // Already processed
    }
} finally {
    unlock(123);
}

Optimizing the Query

Because duplicate requests are rare, performing a query for every request can be wasteful. Instead, rely on a unique index on

idempotent_id

and handle duplicate‑key errors.

// Save request log
sql = "insert into request_log (idempotent_id, ...) values (123, ...);"
try {
    db.insert(sql);
} catch (DuplicateKeyException e) {
    // Duplicate detected, treat as idempotent response
}
// Execute business logic

MySQL’s

INSERT IGNORE

can achieve the same effect by checking the affected‑row count.

sql = "insert ignore into request_log (idempotent_id, ...) values (123, ...);"
int affectedRows = db.insert(sql);
if (affectedRows == 0) {
    // Duplicate key, idempotent response
}
// Execute business logic

Custom Solutions for Specific Scenarios

Scenario 1 – Front‑end button disabling (PRG pattern) combined with server‑side token validation.

Scenario 2 – Coupon redemption can use a state‑machine approach: check coupon status before updating; the status itself guarantees idempotency.

Example using coupon status:

sql = "select coupon_status from coupon where coupon_id = 123;"
int status = db.select(sql);
if (status == '已使用') {
    // Already redeemed, idempotent response
} else {
    // Process redemption
    update coupon set coupon_status = '已使用' where coupon_id = 123;
}

Optimized with optimistic locking:

sql = "update coupon set coupon_status = '已使用' where coupon_id = 123 and coupon_status = '未使用';"
int affectedRows = db.update(sql);
if (affectedRows == 0) {
    // Update failed, treat as idempotent response
}

Summary

To achieve idempotency, any solution must satisfy two basic elements: an idempotent identifier and a uniqueness‑guarantee mechanism. Common approaches include:

Lock + select + insert

Database unique constraint (insert + DuplicateKeyException / insert ignore + affected rows)

State‑machine based idempotency

Combined front‑end and back‑end duplicate‑submission prevention

Key points to remember:

Locks (pessimistic or optimistic) only address concurrency; the core issue remains the identifier plus uniqueness guarantee.

Ensure that the request payload matches the original when retrying.

For multi‑channel clients, combine channel information with the identifier to achieve global uniqueness.

Additional: HTTP Idempotency

GET (and HEAD) are safe and idempotent.

DELETE should be idempotent.

POST is not idempotent.

PUT is idempotent.

Refer to RFC 2616 section 9 for details.