Ensuring Idempotency in Distributed Services: Unique IDs, UUID, Snowflake, and Shared Storage

Why Idempotency is Needed

Idempotency means that executing an operation multiple times has the same effect as executing it once; designing idempotent APIs prevents duplicate orders, double inventory deduction, or charging users twice, especially when service calls may timeout and be retried.

Unique ID

To achieve idempotency, each transaction should carry a unique identifier, which can be allocated by a centralized ID service or generated by the upstream service, though the latter must avoid collisions.

UUID

UUID (Universally Unique Identifier) is a 128‑bit value; while not absolutely collision‑free, the probability is negligible. It is large, not human‑readable, and not sequential, which may be unsuitable for some ordering requirements.

Snowflake

Snowflake, an open‑source algorithm from Twitter, generates a 64‑bit long ID composed of a 41‑bit timestamp (≈69.7 years), a 10‑bit machine identifier (up to 1024 nodes), and a 12‑bit sequence per millisecond (up to 4096 IDs).

Implementations exist in many languages; Redis‑based generators follow similar principles, using bits for time, logical shard ID, and auto‑incrementing node ID.

Shared Storage

When the idempotency service is distributed, the unique IDs must be stored in shared storage (e.g., MySQL or Redis) so that each instance remains stateless.

Avoid Unnecessary Queries

Not every request is a duplicate; checking the existence of an ID for every call can waste resources. With MySQL, an INSERT … ON DUPLICATE KEY UPDATE can detect duplicates; with Redis, SETNX or SETEX can be used similarly.

insert into ... values ... on DUPLICATE KEY UPDATE ...