How to Ensure Idempotency in Distributed Systems: Strategies and Code Examples

Background

In many real systems, operations should produce the same effect or return the same result no matter how many times they are executed. Typical scenarios include:

Frontend duplicate submissions should result in only one backend response.

A payment request should deduct the user's account only once, even if the network or a bug causes a resend.

Sending a message should be performed only once to avoid spamming the user.

Creating a business order should succeed only once; duplicate orders cause serious problems.

All these cases require idempotent behavior.

Idempotency Concept

Idempotence (idempotent, idempotence) originates from mathematics and abstract algebra. In programming, an idempotent operation yields the same effect and result regardless of how many times it is executed with the same parameters. An idempotent function does not change system state after the first execution, e.g., getUsername() or setTrue(). Complex idempotent guarantees are often implemented using a unique transaction number (serial number).

Technical Solutions

1. Query Operations – Queries are naturally idempotent; the same SELECT on unchanged data returns identical results.

2. Delete Operations – Deleting the same record multiple times is idempotent (subsequent deletions may return zero rows).

3. Unique Indexes – Adding a unique (or composite) index prevents duplicate records, e.g., a user can have only one financial account.

4. Token Mechanism to Prevent Duplicate Submissions

Business requirement: a page's data should be submitted only once. Causes of duplicate submissions include repeated clicks, network retries, or proxy resends. Solution: generate a token stored in Redis (or JVM memory) before submission; validate and delete the token on the backend, then issue a new token.

Request a token from the service before submitting data; store it in Redis/JVM with an expiration.

After submission, the backend validates the token, deletes it, and returns a new token.

Token characteristics: single‑use, can be used for rate limiting. Deleting the token is the validation step; using SELECT + DELETE can cause concurrency issues and is discouraged.

5. Pessimistic Lock – Lock rows during data retrieval, e.g., SELECT * FROM table_xxx WHERE id='xxx' FOR UPDATE; The locked column must be a primary key or unique index to avoid table‑wide locks.

6. Optimistic Lock – Lock only at update time, offering higher efficiency. Implementations include version numbers or conditional updates.

update table_xxx set name=#name#, version=version+1 where version=#version#

Another example using a conditional check:

update tablexxx set avaiamount=avaiamount-#subAmount# where avaiamount-#subAmount# >= 0

Use primary key or unique index in the WHERE clause to ensure row‑level locking.

7. Distributed Lock – In distributed systems, introduce a lock via third‑party services like Redis or Zookeeper before inserting or updating data, then release the lock after the operation.

8. Select + Insert Pattern – For low‑concurrency jobs, first query to check if the operation has already been performed, then proceed if not. Not suitable for high‑traffic core flows.

9. State‑Machine Idempotency – Design business processes as finite state machines; if the entity is already in the target state, further transitions are ignored, ensuring idempotence.

10. API Idempotency – Require callers to provide a source identifier and a sequence number (source+seq). Create a unique composite index on these fields to prevent duplicate processing.

Summary

Idempotency is a fundamental trait for competent developers. When designing systems—especially financial platforms like payment gateways—ensuring that operations are idempotent guarantees accuracy, prevents duplicate charges or messages, and improves user experience.