Mastering Idempotence: Techniques to Ensure Safe Operations in Backend Systems

Idempotent Concept

Idempotence (idempotent, idempotence) is a mathematical and computer science concept commonly found in abstract algebra.

In programming, an idempotent operation yields the same effect regardless of how many times it is executed. An idempotent function can be called repeatedly with the same parameters and always return the same result, without altering system state. For example, getUsername() and setTrue() are idempotent.

In plain terms: performing an operation any number of times produces the same effect or return value.

Examples

Repeated form submissions on the frontend should generate only one result on the backend.

A payment request should deduct the user's account only once, even if the request is resent due to network issues.

Sending a message should occur only once; duplicate SMS would overwhelm the user.

Creating a business order should result in a single order, not multiple duplicates.

Many similar scenarios rely on idempotent behavior.

Technical Solutions for Achieving Idempotence

Query Operations

When data does not change, repeated queries return identical results; SELECT is naturally idempotent.

Delete Operations

Delete is also idempotent: deleting once or multiple times results in the data being removed (though the returned count may differ).

Unique Index to Prevent Dirty Data

For example, each user should have only one financial account. Adding a unique index on the user ID ensures that only one insert succeeds; subsequent attempts raise a DuplicateKeyException.

org.springframework.dao.DuplicateKeyException

Token Mechanism to Prevent Duplicate Submissions

Requirement: Page data should be submitted only once. Cause: Repeated clicks, network retries, or Nginx resends can cause duplicate submissions. Solution:

Cluster environment: use token + Redis.

Single‑JVM environment: use token + Redis or token + JVM memory.

Before submitting data, request a token from the service; store the token in Redis or JVM memory with an expiration.

After submission, the backend validates and deletes the token, then returns a new token. Tokens are one‑time, can be used for rate limiting.

Note: Use Redis DELETE to verify a token; a successful delete means validation passed. Do not use SELECT+DELETE due to concurrency issues.

Pessimistic Lock

Lock the data when retrieving it.

SELECT * FROM table_xxx WHERE id='xxx' FOR UPDATE;

Note: The id must be a primary key or unique index; otherwise, the lock may affect the whole table. Pessimistic locks are usually used within a transaction and may hold the lock for a long time.

Optimistic Lock

Optimistic lock only locks at the moment of update, offering higher efficiency than pessimistic lock.

Implementation can use a version field or other conditional checks:

1. Version Number

UPDATE table_xxx SET name = #{name}, version = version + 1 WHERE version = #{version}

2. Conditional Check

UPDATE table_xxx SET avai_amount = avai_amount - #{subAmount} WHERE avai_amount - #{subAmount} >= 0

Requirement: avai_amount - #{subAmount} >= 0 – suitable for inventory models where a simple check ensures data safety without a version number.

Note: Optimistic‑lock updates should use a primary key or unique index to avoid full‑table locks.

UPDATE table_xxx SET name = #{name}, version = version + 1 WHERE id = #{id} AND version = #{version}

UPDATE table_xxx SET avai_amount = avai_amount - #{subAmount} WHERE id = #{id} AND avai_amount - #{subAmount} >= 0

Distributed Lock

When a global unique index is hard to define in a distributed system, a distributed lock (via Redis or Zookeeper) can be used to serialize insert or update operations.

SELECT + INSERT

For low‑concurrency back‑ends or batch jobs, a simple approach is to query key data first, check if the operation has already been performed, and then proceed.

Note: Do not use this method for high‑concurrency critical paths.

State‑Machine Idempotence

When designing documents or tasks that follow a finite state machine, if the entity is already in the target state, further requests to move it back should be rejected, ensuring idempotence.

Note: Understanding state machines is crucial for designing robust business workflows.

API Idempotence for External Interfaces

Example: A payment API (e.g., UnionPay) requires the caller to provide source and seq. These two fields are uniquely indexed in the provider’s database to prevent duplicate payments.

When a third‑party calls the API, the provider first checks whether a record with the same source and seq exists. If it does, the previous result is returned; otherwise, the request is processed and the result stored.

Final Summary

Idempotence should be a core principle for competent programmers; it must be considered when designing systems, especially in payment platforms, banking, and internet finance, where efficiency and data accuracy are critical to avoid duplicate charges and ensure a good user experience.