Mastering Idempotency in Distributed Java Services: Strategies & Code

1. Idempotency Overview

1.1 Understanding Idempotency

In computing, idempotence means that executing an operation multiple times yields the same result without additional side effects. In Java backend development, idempotency is achieved by ensuring method or service call results are deterministic, so repeated calls produce predictable outcomes.

While this definition matches most literature, real‑world internet services often encounter conflicts between the theoretical definition and business logic.

For example, a query call from system A to system B may fail due to a bug; after fixing the bug, a retry with the same parameters could still return the same failure, which is idempotent but not aligned with business expectations. Similarly, a payment request that initially fails due to insufficient balance will return the same error after the user tops up, again satisfying the definition but violating business logic.

Therefore, idempotency solutions should focus on preventing undesirable side effects of duplicate requests—especially for write operations such as duplicate charges or refunds—rather than merely adhering to the strict definition.

1.2 Why Idempotency Is Needed

In microservice and distributed architectures, a single request often involves multiple services. Network jitter and system anomalies make 100 % success impossible, so retries are common, which inevitably cause duplicate requests.

Idempotent design is created to handle duplicate requests, ensuring expected results without side effects. The following scenarios generate duplicate requests:

Unreliable user : Users may accidentally or intentionally click repeatedly, causing rapid duplicate submissions.

Unreliable network : Network jitter or gateway instability can trigger retry mechanisms, especially when delivering messages via queues.

Unreliable service : In data‑consistency scenarios, downstream service timeouts often lead to retries.

In my Spring Cloud microservice column “Using RocketMQ for Distributed Transactions”, a bug caused by missing idempotent handling led to data‑consistency issues when the broker did not receive an ACK and retried.

1.3 Idempotency and Concurrency

Concurrent write scenarios typically require idempotent handling. For instance, multiple rapid form submissions or simultaneous submissions using special techniques constitute a classic concurrent case that needs idempotency.

However, idempotency is not exclusive to concurrency. It is needed whenever a duplicate request involves a write operation, even if the requests are spaced apart.

In the internet domain, idempotency and concurrency are closely related, leading some to think that solving idempotency solves high concurrency.

2. Idempotency Key Design

An idempotency key is a unique identifier (e.g., token or business serial number) agreed upon in advance to ensure that the same request is processed only once.

The key must satisfy three properties: uniqueness, immutability, and transitivity.

Two design approaches exist:

Non‑business idempotency key : Uses a UUID, timestamp, or business serial number. Both caller and callee must persist the key to trace request‑key relationships.

Business idempotency key : Combines business elements such as “userId+activityId”. The callee can derive the key from request parameters without separate persistence.

3. Idempotency Implementation Schemes

Idempotency is achieved by ensuring that identical requests are processed only once. Six common schemes are presented: unique index, token, pessimistic lock, optimistic lock, distributed lock, and state machine.

3.1 Unique Index Scheme

The unique index scheme relies on a database table that disallows duplicate rows with the same index value. In high‑concurrency scenarios, only one thread can insert the duplicate record successfully; others receive a unique‑constraint exception.

A typical business‑flow table includes the following core fields: id (bigint) – primary key. gmt_create (datetime) – creation time. gmt_modified (datetime) – last modification time. user_id (varchar(32)) – user identifier, also usable for sharding. out_biz_no (varchar(64)) – external business serial number, i.e., the idempotency key. biz_no (varchar(64)) – internal business serial number for tracing. status (char(1)) – execution status.

Typically, user_id and out_biz_no form a composite unique index to prevent duplicate inserts under concurrency.

3.2 Token Mechanism

The token mechanism prevents duplicate client submissions, especially for order‑creation forms. Workflow:

1) When the user accesses the form page, the client requests a unique token (UUID or global ID) from the server, which stores the token in Redis or a database.

2) On the first form submission, the token is sent together with the form. The server validates the token's existence, executes business logic, and then destroys the token.

3) Subsequent submissions carry the same token, but since it has been destroyed, the server rejects the duplicate request.

3.3 Pessimistic Lock Mechanism

Pessimistic lock uses the database's locking mechanism combined with transactions to serialize access.

// 1. Begin transaction
begin;

// 2. Query by idempotency key for update
record = select * from tbl_xxx where out_biz_no = 'xxx' for update;

// 3. Decision based on status
if (record.getStatus() != expectedStatus) {
    return;
}

// 4. Update record
update tbl_xxx set status = 'targetStatus' where out_biz_no = 'xxx';

// 5. Commit transaction
commit;

Pessimistic lock is suitable for update scenarios but may cause long wait times under high concurrency.

3.4 Optimistic Lock Mechanism

Optimistic lock relies on conditional updates with a version field.

// 1. Retrieve object with version
select * from tablename where id = xxx;

// 2. Update with version check
update tableName set sq = sq-#{quantity}, version = #{version}+1
where id = xxx and version = #{version};

This ensures that concurrent updates do not interfere with each other.

3.5 Distributed Lock Mechanism

Distributed lock works similarly to pessimistic lock but is lighter weight. The process attempts to acquire a lock; if successful, it executes business logic, otherwise it rejects the request.

Because lock acquisition does not guarantee successful business execution, distributed lock should be combined with transaction and retry mechanisms for a complete idempotent solution.

3.6 State Machine Mechanism

Many business documents have a finite set of states with a fixed transition order. If a record is already in the next state, re‑applying the previous state change has no effect, ensuring idempotency.

For example, inventory status may transition through “reserved”, “deducted”, “occupied”, and “released”. If the deduct interface is called again while the status is already “deducted”, the service can simply return success.

State machines can be combined with optimistic locks:

update tableName set sq = sq-#{quantity}, status = #{update_status}
where id = #{id} and status = #{status};

3.7 Summary

The six idempotency schemes can be grouped into three technical routes: unique index, unique data (pessimistic/optimistic/distributed locks), and state‑machine constraints. In practice, these schemes should be combined—for example, locks alone do not handle retries, so they must be paired with unique indexes and transactions.

4. Code Implementation

In the DailyMart project, five idempotency schemes (excluding pessimistic lock) are implemented in a starter component dailymart-idempotent-spring-boot-starter. Adding the starter as a Maven dependency enables the functionality.

<dependency>
  <groupId>com.jianzh5</groupId>
  <artifactId>dailymart-idempotent-spring-boot-starter</artifactId>
  <version>${project.version}</version>
</dependency>

The core uses Spring AOP. Annotate methods with @Idempotent and specify IdempotentTypeEnum to select the scheme.

Distributed‑lock schemes rely on Redis, so Redis configuration must be added to the Spring Boot properties.

spring:
  data:
    redis:
      host: xxx.xx.xx.xx
      port: 29359
dailymart:
  cache:
    redis:
      prefix: "inventory:"
  idempotent:
    token:
      prefix: "token-"
      timeout: 30000

4.1 Unique Index Implementation

When a user places an order, the inventory‑pre‑deduction interface can use a unique index on the business‑flow field transactionId together with a transaction to guarantee idempotency.

4.2 Optimistic Lock Implementation

For payment‑related inventory deduction, optimistic lock can be applied either via native SQL:

public interface InventoryItemMapper extends BaseMapper<InventoryItemDO> {
    @Update("UPDATE inventory_item SET sellable_quantity = #{sellableQuantity}, withhold...")
    void updateByVersion(InventoryItemDO inventoryItemDO);
}

or using MyBatis‑Plus’s @Version annotation on the version field and adding the OptimisticLockerInnerInterceptor bean.

4.3 State Machine Implementation

During a return, the inventory release interface checks the current state and returns early if already released.

@Override
@Transactional
public void releaseInventory(Long transactionId) {
    // if already released, return
    if (inventoryRecord.getState() == InventoryRecordStateEnum.RELEASE.code()) {
        return;
    }
    // ...
}

4.4 Token Implementation

The client obtains a token via /token, which the server stores in Redis. The order‑creation endpoint is annotated with @Idempotent(type = IdempotentTypeEnum.TOKEN, ...) to enforce token validation.

@PostMapping("/api/order/create")
@Idempotent(type = IdempotentTypeEnum.TOKEN, message = "Order is being created, please do not submit repeatedly")
public void create(@RequestBody OrderDTO orderDTO) {
    orderService.save(orderDTO);
}

4.5 Distributed‑Lock Implementation

Two ways to use distributed locks: (1) IdempotentTypeEnum.PARAM hashes the entire request parameters as the lock key; (2) IdempotentTypeEnum.SpEL uses a SpEL expression to select specific fields as the lock key.

@Idempotent(type = IdempotentTypeEnum.PARAM, message = "Order is being created, please do not submit repeatedly")
@PostMapping("/api/order/create")
public void create(@RequestBody OrderDTO orderDTO) {
    orderService.create(orderDTO);
}

@Idempotent(key = "#lockRequest.transactionId", type = IdempotentTypeEnum.SpEL)
@PostMapping("/api/order/update")
public void update(@RequestBody OrderDTO orderDTO) {
    orderService.update(orderDTO);
}

5. Final Summary

The article details various idempotency solutions for distributed systems and clarifies the distinction between idempotency and concurrency. It emphasizes that a single technique rarely suffices; combining multiple approaches yields a robust solution.