How to Ensure API Idempotency: 8 Proven Strategies for Backend Systems

Introduction

API idempotency means that multiple identical requests produce the same result without side effects. Duplicate data often appears in insert, update, timeout‑retry, or message‑consumer scenarios. This article presents practical solutions that have been used in real projects.

1. Insert Before Select

Before performing an INSERT, query the table by a unique field (e.g., name or code). If the record exists, execute an UPDATE; otherwise, proceed with the INSERT. This approach works well in single‑threaded contexts but needs additional safeguards under high concurrency.

2. Pessimistic Lock

In a payment scenario, use a row‑level lock to ensure only one request updates the balance at a time.

UPDATE user SET amount = amount-100 WHERE id=123;

Acquire the lock with: SELECT * FROM user WHERE id=123 FOR UPDATE; Steps:

All requests query the user row.

Check if the balance is sufficient.

If sufficient, attempt to acquire the lock with FOR UPDATE.

The first request obtains the lock; others wait.

After locking, re‑check the balance and perform the update.

If the balance is insufficient, treat the request as a duplicate and return success.

MySQL must use the InnoDB engine, and the locked column should be a primary key or unique index; otherwise the whole table may be locked.

3. Optimistic Lock

Add a version (or timestamp) column. Update statements include the current version in the WHERE clause and increment the version atomically.

SELECT id, amount, version FROM user WHERE id=123;

UPDATE user SET amount=amount+100, version=version+1 WHERE id=123 AND version=1;

If the affected rows count is 0, the request is a duplicate; otherwise, it succeeded.

4. Unique Index

Define a unique index on fields that must be unique (e.g., code). On duplicate insertion, the database throws a Duplicate entry error, which can be caught and treated as a successful idempotent response.

ALTER TABLE `order` ADD UNIQUE KEY `un_code` (`code`);

5. Anti‑Duplicate Table

Create a lightweight table that stores only the unique key (e.g., id and code). Insert the key first; if the insert succeeds, proceed with the main business operation. If a duplicate‑key error occurs, treat the request as already processed.

6. State‑Machine Check

When a table has a status field, update it only if the current status matches the expected previous state. Example for an order:

UPDATE `order` SET status=3 WHERE id=123 AND status=2;

If the affected rows count is 0, the request is a duplicate.

7. Distributed Lock

Use a distributed lock (e.g., Redis SET NX, SET with expiration, or Redisson) to ensure only one instance processes a request. The lock must have a reasonable TTL to avoid deadlocks.

Generate a unique business key (e.g., order code).

Attempt to set the key in Redis with an expiration.

If setting succeeds, treat it as the first request and execute business logic.

If setting fails, consider the request a duplicate and return success.

The lock’s expiration time should balance between preventing duplicate processing and avoiding unnecessary resource consumption.

8. Token‑Based Validation

Issue a one‑time token (stored in Redis) on the first request. The second request must present the token; if the token exists, the operation proceeds, otherwise it is rejected. Tokens expire automatically, ensuring idempotency.

Client requests a token; server generates and stores it in Redis.

Client sends the token with the actual business request.

Server verifies the token’s existence; if present, processes the request and deletes the token.

If the token is missing, treat the request as a duplicate and return success.