Idempotency Design for Write Services: 7 Patterns & Selection Guide

This article provides a comprehensive analysis of idempotency design patterns for write‑operation services, organized into backend‑only solutions and client‑server collaborative solutions, followed by a practical selection guide.

01 Backend‑Implemented Idempotency Solutions

Pattern 1: Business Unique Key with Database Unique Constraint

Core principle: A business unique key identifies "the same operation"; a database unique constraint guarantees that this identity can be persisted only once. The key must remain stable across all retries of the same business action; generating a new auto‑increment ID or UUID per request breaks idempotency.

Applicable scenario: Payment platform receiving duplicate notifications for the same payment. Example: channel A, merchant M1, payment serial P1001 for order 911001, amount 3980 cents. The composite key (channel, merchant_id, payment_no) uniquely identifies the payment.

Implementation: Create a unique index on the payment record table:

UNIQUE KEY uk_payment (channel, merchant_id, payment_no)

On first callback, insert the payment record, update the order, and save necessary results in the same transaction. On duplicate, the unique constraint violation triggers a verification: compare order ID and amount with the stored record; if they match, return the original result; if they differ, return a parameter conflict error.

Best practices: After a unique‑key conflict, roll back the failed transaction, then read the existing record in a new transaction. Verify permissions and parameters before returning the original result. In sharded databases, ensure the same business key routes to the same shard. Remote operations (coupon issuance, refunds) require downstream deduplication; the local unique constraint cannot protect them.

Pattern 2: State Machine with Conditional Update

Core principle: Encode the allowed pre‑state in the UPDATE WHERE clause. A successful transition leaves that pre‑state, so repeated requests cannot satisfy the condition again.

Applicable scenario: User repeatedly cancels a pending‑payment order (order 911001). The order can only move from PENDING_PAYMENT to CANCELLED; subsequent cancellations find zero rows updated.

Implementation: Use a conditional UPDATE:

UPDATE lab_order
SET order_state = 'CANCELLED'
WHERE order_id = 911001
  AND order_state = 'PENDING_PAYMENT';

Only when exactly one row is updated proceed to write cancellation log and release local reserved inventory within the same transaction. If zero rows updated, commit the write transaction, then check current state: if already CANCELLED return success per contract; if PAID reject direct cancellation; if not found or unauthorized handle per API spec.

Notes: If a rejected request is resubmitted and returns to a pre‑state (e.g., back to PENDING_APPROVAL), add an approval round or execution record to distinguish the two rounds. Remote resource release cannot join the local transaction and must be guaranteed separately.

Pattern 3: Business Execution Record with Local Transaction

Core principle: Persist a unique execution record for each business action, committing the record, business modifications, and result together. A duplicate request hits the existing record, verifies parameters, and returns the saved result without re‑executing business logic.

Applicable scenario: Warehouse retries shipping registration for order 911001 with operation ID S001 (carrier + tracking number). Need to confirm exactly which shipping action completed and retain its parameters and result for later retries.

Implementation: Unique key on execution record:

UNIQUE KEY uk_shipping_action (tenant_id, shipping_action_no)

Flow: 1. Insert execution record with S001 (tenant, action_no, parameters, result placeholder). 2. Conditional UPDATE order status from PENDING_SHIPMENT to SHIPPED. 3. If UPDATE affects 1 row, save shipping details and final result, commit. 4. If UPDATE affects 0 rows, rollback, then check order state: if another action already shipped, return order‑already‑shipped; do not create new shipping details. 5. On unique‑key conflict, rollback, read original record in new transaction; if parameters match return original result, else return parameter conflict.

Key guarantees: Unique constraint identifies the same operation; local transaction ensures execution record and business changes are atomic; state condition ensures shipping only occurs in allowed phase. If record commits before business logic, a mid‑failure could leave a "record exists but not shipped" false success. Remote logistics calls (create tracking number) cannot roll back with local transaction and need separate reliable invocation and deduplication.

If both business data and execution record reside in Redis, combine check‑modify‑save in a single Lua script (keys must be in same hash slot). Script errors do not auto‑rollback; handle partial writes and define retention/failure recovery.

Pattern 4: Naturally Idempotent Operations

Core principle: Express the operation as setting a deterministic target value. Repeated execution yields the same target state (e.g., "set to OFF"). Toggle operations are not naturally idempotent.

Applicable scenario: User disables email notifications. Request explicitly sets email_enabled = 0. Retries keep setting to 0; no extra business effect.

Implementation: Simple UPDATE with target value:

UPDATE user_preference
SET email_enabled = 0
WHERE user_id = ?;

First request changes 1→0; retries set 0→0. Confirm target state satisfied and return success. Do not implement as toggle (would re‑enable on second call). On timeout, query target state or retry with original target. If business must protect later user changes, include version precondition. Check side effects: if each disable also issues a coupon, coupon issuance still needs separate deduplication.

Pattern 5: Request Fingerprint with Time Window

Core principle: Generate a fingerprint from normalized request content; within an agreed time window, treat identical fingerprints as a single request. Business must accept this merging rule.

Applicable scenario: User clicks "Export monthly orders" twice within 3 seconds. Business agrees to merge exports within 5 seconds into one task. After window expires, same request may create a new task. Not suitable for operations requiring precise intent distinction (e.g., placing orders).

Implementation: Normalize parameters (fixed date range, tenant, user, action, filters, format) → fingerprint F. Pre‑allocate task ID T1. Use Redis atomic SET NX EX 5 to register fingerprint for 5 seconds:

SET repeat:F T1 NX EX 5

Duplicate request reads T1 from Redis, then must query task store to confirm task creation (first request may have only set Redis placeholder). Fingerprint retains business‑meaning parameters, normalizes JSON field order, excludes per‑attempt trace IDs. On confirmed creation failure, atomically compare‑and‑delete placeholder; on unknown result, continue verifying via original task ID. If Redis result unknown, check placeholder and task; if placeholder expired and no task ID, retry registration per window rules.

02 Client‑Server Collaborative Idempotency Solutions

Pattern 1: Frontend Generates Idempotency Key, Backend Saves Result

Core principle: Frontend generates an idempotency key for one business intent; backend uses it to detect duplicates and persist the result. Same content can be purchased twice, but retries of the same purchase reuse the original key.

Applicable scenario: User buys SKU 9001 quantity 2; frontend generates key purchase-001. Backend creates order 911001, response lost, frontend retries with same key → returns order 911001. A new purchase uses purchase-002.

Frontend flow:

Confirm new purchase → generate and save purchase-001 + payload.

First submit → send purchase-001.

Timeout or refresh → reuse purchase-001.

Explicit new purchase → generate purchase-002.

HTTP request header includes Idempotency-Key: purchase-001.

POST /orders HTTP/1.1
Content-Type: application/json
Idempotency-Key: purchase-001

{"items":[{"skuId":9001,"quantity":2}]}

Backend processing: Extract tenant/user from trusted identity, scope key. Use database unique constraint on idempotency record (tenant, user, action, key). In transaction: 1. Validate identity, permissions, request format. 2. Begin transaction. 3. Insert idempotency record (unique constraint acquires operation identity). 4. Validate business conditions for new order. 5. Create order, save details and necessary logs. 6. Save first business response. 7. Commit on full success; rollback on any failure. 8. Return order result after commit.

Duplicate handling (conflict on unique constraint): Rollback failed transaction, start new transaction, verify permissions, parameters, and stored result. Return original result only if parameters match; else return parameter conflict. Normalize irrelevant differences (JSON field order). On timeout or empty query, continue confirming via original key; must hit primary DB, re‑enter full flow with unique constraint if needed; never switch keys. Retain records at least until agreed retry window ends. Distinguish HTTP requestId (per attempt) from business idempotency key (per purchase). Cleaning records too early may allow late retries to recreate; extend retention or rely on downstream business identity for longer protection. Remote executions require each service to implement its own deduplication.

Pattern 2: Backend Issues Submission Token, Frontend Carries and Reuses

Core principle: Backend pre‑creates a submission credential (token) bound to a prepared settlement; frontend presents the token; backend uses conditional update to ensure token is consumed exactly once. Token and business result commit together.

Applicable scenario: Backend calculates quote for SKU 9001 qty 2 → settlement record Q1 (price 3980 cents, 5‑minute TTL) and issues token T1 bound to Q1, user, CREATE_ORDER action. User confirms with T1; backend creates order per Q1. Retry with T1 returns original order.

Implementation: On quote preparation, save settlement details (items, price, version) and create token T1 with state UNUSED, expiry 5 minutes, bound to user, Q1, action. Return T1 to frontend. Frontend stores T1; refresh/retry reuse T1.

Backend consumes token with conditional UPDATE:

UPDATE submission_token
SET state = 'USED'
WHERE token = ?
  AND actor_id = ?
  AND state = 'UNUSED'
  AND expires_at > CURRENT_TIMESTAMP;

On UPDATE success (1 row), still verify Q1 price/inventory in business transaction, create order, save token consumption, order, parameter summary, result together. Rollback on any failure; retry within TTL using original T1. Token does not guarantee inventory lock.

On UPDATE 0 rows: end write transaction, check committed records in new transaction. If original transaction still uncertain, wait or return pending per contract; do not assume token invalid.

Token state handling:

UNUSED & within TTL → compete via conditional update, then execute business.

USED & parameters match → return original result within result confirmation period.

USED & parameters differ → parameter conflict.

UNUSED but TTL expired → reject new consumption; require re‑quote.

Not exist or not owned by user/action → reject per API spec.

Separate submission TTL (limits new consumption) from result confirmation period (limits how long completed results are queryable). A consumed token past its TTL but within confirmation period still allows result verification. After timeout, do not grab a new token T2; restore original token and confirm result first. Token consumption and order write must share the same local transaction; splitting across Redis (delete token) and MySQL (write order) risks "token consumed but order not created" and requires cross‑store recovery design. If one settlement Q1 permits only one order, enforce via unique constraint on order→Q1 link; switching tokens must not bypass this. If Q1 expires uncommitted, re‑quote and reconfirm; if old submission result unknown, verify via T1 first.

03 Selection Guide in Real Projects

Choose by examining existing business identifiers in the interface:

Payment duplicate notification → Unique key + unique constraint (stable payment identity from channel/merchant/serial).

Repeat cancel pending order → State machine + conditional update (clear pre‑state, atomic state change + resource release).

Warehouse retry shipping registration → Execution record + local transaction (need to know which action completed, retain parameters/result).

Repeat disable email notification → Naturally idempotent (explicit target value; side effects like coupon need separate deduplication).

Short‑term duplicate report export → Request fingerprint + time window (business allows merging within window; not for long‑term exactly‑once).

Direct purchase with retry → Frontend idempotency key (no pre‑existing backend identity; frontend distinguishes retry vs new purchase).

Quote then confirm order → Backend submission token (existing backend preparation step; token binds settlement to submission right).

Patterns can be combined. Shipping example uses operation‑no unique constraint, state condition, and local transaction. If only state transition guarantee needed, conditional update suffices; add execution record when original parameters and result must be retrievable. Reuse existing shipping record table if it already serves that purpose.

Frontend key vs backend token: choose based on whether a backend preparation step exists. Simple purchase → frontend key simpler; quote‑then‑confirm → backend token natural. Both require retries to reuse original identity; same identity with different business parameters → parameter conflict, never overwrite original result.

Final check: timeout handling. On definite failure, rollback and retry per contract; on unknown commit result, query via original business key, operation ID, or token. Only when the interface can identify duplicates, avoid duplicate side effects, and let the caller confirm the processing result is the idempotency flow complete.