Understanding Interface Call Idempotency and Practical Solutions for Backend Systems

What is the Interface Call Idempotency Problem?

In modern distributed or micro‑service architectures, services often call each other via RPC or RESTful APIs. If a server crashes after processing a request but before returning a response, the client may retry the request, leading to multiple executions of the same operation, which must produce a consistent result.

Idempotency means that one or multiple identical requests yield the same outcome without side effects, e.g., a payment should not be deducted twice even if the client resends the request due to a network error.

When Should Idempotency Be Prevented?

1. Users click a button multiple times.

2. Users navigate back and resubmit a form.

3. Micro‑service calls fail and trigger retry mechanisms (e.g., Feign retries).

4. Other business scenarios.

Common Idempotent and Non‑Idempotent Examples

SQL statements that are naturally idempotent: SELECT * FROM table WHERE id = ? – read‑only, no state change. UPDATE tab1 SET col1=1 WHERE col2 = 2 – setting a column to a constant yields the same state each time. DELETE FROM user WHERE userid = 1 – deleting the same row repeatedly has no further effect. INSERT INTO user(userid,name) VALUES(1,'a') – if userid is a unique primary key, duplicate inserts are rejected, making the operation idempotent. UPDATE tab1 SET col1 = col1 + 1 WHERE col2 = 2 – each execution changes the value, thus non‑idempotent. INSERT INTO user(userid,name) VALUES(1,'a') – if userid is not unique, repeated inserts create multiple rows, also non‑idempotent.

Idempotency Solutions

1. Token Mechanism (code example below)

1) The server provides an endpoint to generate a token and stores it in Redis before the business logic runs.

2) The client includes the token in the request header when calling the business API.

3) The server checks if the token exists in Redis; if it does, it treats the request as the first one, deletes the token, and proceeds.

4) If the token is absent, the request is considered a duplicate and is rejected.

Risks

1) Order of token deletion

a) Deleting the token first may cause a retry to use the same token, leading to a false duplicate rejection.

b) Deleting after business execution may leave the token in Redis if the service crashes, allowing a retry to execute the business again.

c) A recommended approach is to delete the token first, but if the business call fails, obtain a new token and retry.

2) Token get/compare/delete must be atomic

a) The sequence redis.get(token), token.equals(...), redis.del(token) must be performed atomically; otherwise, concurrent requests may all see the same token and execute the business multiple times.

b) Use a Redis Lua script for atomicity:

if redis.call('get', KEYS[1]) == ARGV[1] then
  return redis.call('del', KEYS[1])
else
  return 0
end

2. Various Lock Mechanisms

1) Database Pessimistic Lock

select * from xxxx where id = 1 for update

Pessimistic locks are usually used within a transaction; the locked row must have a primary key or unique index to avoid table‑level locking.

2) Database Optimistic Lock

Suitable for update‑heavy scenarios using a version column.

update t_goods set count = count - 1, version = version + 1 where good_id = 2 and version = 1

The first request updates the version from 1 to 2; a subsequent request still using version 1 will fail the where clause, ensuring only one successful execution.

3) Business‑Level Distributed Lock

When multiple machines may process the same data concurrently (e.g., scheduled tasks), acquire a distributed lock (e.g., via Redis) before handling the data and release it afterward.

3. Various Unique Constraints

1) Database Unique Constraint

Insert operations should rely on unique indexes (e.g., order numbers) to prevent duplicate rows. In sharding scenarios, routing must ensure the same logical request lands in the same physical table for the unique constraint to be effective.

2) Redis Set for Deduplication

Compute an MD5 of the data, store it in a Redis set, and skip processing if the hash already exists.

4. Deduplication Table

Use a dedicated table with a unique index (e.g., orderNo) to record processed requests within the same transaction as the business operation; duplicate attempts fail due to the unique constraint.

Note: The deduplication table and business table should reside in the same database so that both are rolled back together if the business transaction fails.

5. Global Request Unique ID

Generate a globally unique ID for each request and store it in a Redis set; presence indicates the request has already been processed.

proxy_set_header X-Request-Id $request_id;

Token Mechanism Example (Java + Redis)

Creating a token when the user proceeds to checkout:

// Create a anti‑duplicate token
String token = UUID.randomUUID().toString().replace("-", "");
// Key: order:token:<userId>
redisTemplate.opsForValue().set("order:token:" + userId, token, 30, TimeUnit.MINUTES);
// Return token to the client for later verification

When the user submits the order, retrieve and compare the token atomically using a Lua script:

// Retrieve user ID and token from request
String userId = xxxVo.getUserId();
String token = xxxVo.getOrderToken();
String script = "if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end";
Long result = redisTemplate.execute(new DefaultRedisScript<Long>(script, Long.class),
    Arrays.asList("order:token:" + userId), token);
if (result == 0L) {
    // Verification failed – duplicate request
} else {
    // Verification succeeded – proceed with business logic
}

By ensuring token generation, comparison, and deletion are atomic, the system prevents duplicate order processing even under high concurrency.