Ensuring API Idempotency: Tokens, Dedup Tables, Redis SETNX, and Lock Strategies

Introduction

The focus of this article is on solutions for API idempotency, especially under high‑concurrency scenarios.

Body

1 API Idempotency

Idempotency means that multiple identical requests produce the same final result.

If an API does not guarantee idempotency, duplicate operations can cause serious issues, such as a transfer being executed multiple times.

1.1 Example

Consider a transfer request where user A sends 1000 CNY to user B. Due to network glitches, the request is retried three times, and without idempotency the system processes three transfers, resulting in 3000 CNY being moved.

2 Solutions

2.1 Token Mechanism

The token approach works as follows:

Client requests a token from the server; the server generates a new token, stores it in Redis with an expiration, and returns it.

The client includes the token in subsequent business requests.

The server checks Redis: if the token exists, it deletes the token and proceeds; if not, the request is considered already processed and is ignored.

Diagram:

The token method adds an extra request to obtain the token, which may affect latency‑critical services.

2.2 Deduplication Table

This method leverages MySQL's unique index feature:

Client sends a request; the server inserts a record representing the request into a deduplication table with a unique constraint on a specific field.

If the insert succeeds, the business logic continues; if it fails, the request has already been processed.

Diagram:

Issues include MySQL reliability and the overhead of disk‑based unique checks under high load.

2.3 Redis SETNX Key

The process using Redis SETNX is:

The server attempts to store a unique request identifier in Redis with the SETNX command and sets an expiration.

If SETNX returns 1 (success), the request proceeds; if it returns 0, the request is considered duplicate.

SETNX sets a key only if it does not already exist; it returns 1 on success and 0 on failure.

Diagram:

2.4 State‑Machine Idempotency

When business processes have distinct states (e.g., pending payment, locked, paid), the current state can be used to decide whether further actions should be executed.

2.5 Optimistic Lock (Update)

An optimistic lock adds a version column to the table; the version is checked before updating.

SELECT version FROM table WHERE ...; UPDATE table SET ... , version = version+1 WHERE ... AND version = :version;

This approach avoids locking but may require retry logic on version conflicts.

2.6 Pessimistic Lock (Update)

Pessimistic locking acquires a row lock via SELECT … FOR UPDATE, ensuring exclusive access.

START TRANSACTION; SELECT * FROM TABLE WHERE ... FOR UPDATE; UPDATE TABLE SET ... WHERE ...; COMMIT;

Conclusion

All presented idempotency solutions share similar principles—preventing duplicate processing—but each has its own strengths and drawbacks. Selecting the appropriate method depends on the specific business requirements, performance constraints, and existing infrastructure.