General Idempotent Implementation for Backend Services

Background

Answering a community question about generic solutions and practices for idempotency. The article assumes readers already understand the concept of idempotency and have encountered related problems in their own systems.

In most business systems, idempotency is handled individually, but a unified framework can simplify development, especially for non‑core scenarios such as network retries or repeated user clicks.

Simple Idempotent Implementation

The simplest approach uses a database record to determine whether an operation has already been performed.

Database Record Check

For example, an order should be paid only once. Before processing payment, the system checks a dedicated table for a previous payment record; if found, the request is treated as already successful.

This requires an extra table to store performed actions.

Concurrency Problem Solution

When two requests for the same order query the table simultaneously, both may see no record and proceed, causing duplicate payments.

Common solutions include using a unique index in the database or applying a distributed lock to ensure only one request proceeds at a time.

General Idempotent Implementation

A reusable component can encapsulate idempotent handling, allowing developers to focus on business logic.

Design Scheme

General Storage

The component first queries a storage layer and locks the resource to avoid concurrent conflicts.

Ease of Use

Inject the component into business code; it hides locking and record‑checking details.

Annotation Support

Developers can add an annotation to a method to enable idempotency without writing explicit code.

Multi‑Level Storage

Level‑1 storage (e.g., Redis) provides fast, short‑term caching for most cases; Level‑2 storage (e.g., MySQL, MongoDB) handles long‑term or permanent records. The storage type can be configured, and a strategy pattern is recommended.

Concurrent Read/Write

Two modes are supported: sequential (write level‑1 then level‑2) and parallel (simultaneous writes) to improve performance.

Idempotent Execution Flow

Idempotent Interface

Interface definition

public interface DistributedIdempotent {
    /**
     * Idempotent execution
     * @param key Idempotent key
     * @param lockExpireTime Lock expiration time
     * @param firstLevelExpireTime First‑level storage expiration
     * @param secondLevelExpireTime Second‑level storage expiration
     * @param timeUnit Time unit for expiration
     * @param readWriteType Read/write mode
     * @param execute Logic to execute on first request
     * @param fail Logic to execute on duplicate request
     * @return Result of execution
     */
    <T> T execute(String key, int lockExpireTime, int firstLevelExpireTime, int secondLevelExpireTime, TimeUnit timeUnit, ReadWriteTypeEnum readWriteType, Supplier<T> execute, Supplier<T> fail);
}

Usage example (code‑based idempotency with return value)

/**
 * Code‑based idempotent method with return value
 */
public String idempotentCode(String key) {
    return distributedIdempotent.execute(key, 10, 10, 50, TimeUnit.SECONDS, ReadWriteTypeEnum.ORDER, () -> {
        System.out.println("Processing...");
        return "success";
    }, () -> {
        System.out.println("Duplicate request...");
        return "fail";
    });
}

Idempotent Annotation

Annotations simplify usage; adding @Idempotent to a business method automatically applies the idempotent logic.

/**
 * Annotation‑based idempotent method
 */
@Idempotent(spelKey = "#key", idempotentHandler = "idempotentHandler", readWriteType = ReadWriteTypeEnum.PARALLEL, secondLevelExpireTime = 60)
public void idempotent(String key) {
    System.out.println("Processing...");
}

public void idempotentHandler(String key, IdempotentException e) {
    System.out.println(key + ": idempotentHandler already executed.");
}

Automatic Duplicate Request Differentiation

When using code‑based idempotency, the developer supplies a unique key; with annotations, the key can be generated automatically via SPEL or an MD5 hash of request URL, parameters, body, and headers.

Storage Structure

Redis: store the idempotent key as a String with a default value of 1.

MySQL: create a table to persist records (expired data can be cleaned periodically).

CREATE TABLE `idempotent_record` (
  `id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'Primary key',
  `key` varchar(50) NULL DEFAULT '',
  `value` varchar(50) NOT NULL DEFAULT '',
  `expireTime` timestamp NOT NULL COMMENT 'Expiration time',
  `addTime` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'Creation time',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='Idempotent records';

MongoDB: same fields as MySQL, stored as JSON; collections are created automatically.

Source Code

Source repository: https://github.com/yinjihuan/kitty

Sample projects: https://github.com/yinjihuan/kitty-samples

If you find this useful, please give a three‑click support!