Distributed System Unique ID: Characteristics and Implementation Solutions

In complex distributed systems, a large amount of data and messages need globally unique identifiers. Traditional auto‑increment IDs cannot meet the requirements of high‑growth, sharded databases, making a global ID generation system essential.

Characteristics of Distributed System Unique IDs

Key requirements include:

Global uniqueness – no duplicate IDs.

Trend‑increasing – ordered IDs improve write performance in B‑tree indexes.

Monotonic increase – each new ID must be greater than the previous one.

Security – non‑sequential IDs prevent easy enumeration and data scraping.

Additionally, the ID generation service must have low average and 99.9th‑percentile latency, five‑nines availability, and support high QPS.

Implementation Schemes

1. UUID

UUID (Universally Unique Identifier) is a 128‑bit value represented as 36 characters (8‑4‑4‑4‑12). It can be generated locally without network latency.

Advantages:

Very high performance due to local generation.

Disadvantages:

Large storage footprint (16 bytes, usually stored as a 36‑character string).

Potential security concerns when generated from MAC addresses.

Not ideal as a primary key in many databases.

2. Database‑Generated ID

Using MySQL auto_increment_increment and auto_increment_offset to ensure unique IDs across instances.

Advantages:

Simple to implement using existing DB features; low cost.

Monotonically increasing IDs satisfy ordering requirements.

Disadvantages:

Strong dependency on the database; DB failures can cripple the entire system.

Performance limited by a single DB node’s read/write capacity.

3. Redis‑Generated ID

Redis, being single‑threaded, can generate global IDs using atomic INCR/INCRBY commands, suitable for daily sequence numbers (e.g., date + increment).

Advantages:

No DB dependency; higher performance.

Numeric IDs are naturally ordered, aiding pagination and sorting.

Disadvantages:

Introduces a new component if Redis is not already used, increasing system complexity.

Requires additional coding and configuration effort.

4. Zookeeper‑Generated ID

Zookeeper can generate 32‑bit or 64‑bit sequential numbers via znode version numbers, but it is rarely used due to multi‑step API calls and the need for distributed locks under high contention.

5. Snowflake Algorithm

Snowflake allocates a 64‑bit ID split into timestamp, data‑center, machine, and sequence bits (e.g., 41‑bit time, 10‑bit machine, 12‑bit sequence). This yields 69 years of timestamps, up to 1024 machines, and 409.6 w/s QPS.

Advantages:

Trend‑increasing IDs due to timestamp in high bits.

Independent of databases; high stability and performance.

Flexible bit allocation to suit business needs.

Disadvantages:

Strong reliance on accurate system clocks; clock rollback can cause duplicate IDs or service downtime.

For a deeper dive, the author offers a 300k‑word collection of architecture articles and a comprehensive Java interview Q&A set, available via the linked WeChat public account.