How to Choose the Right Global Unique ID Strategy for Distributed Systems

In distributed systems, many scenarios require globally unique IDs, such as payment transaction numbers, sharding keys, transaction version numbers, or distributed tracing.

Good global unique IDs should be:

Globally unique – never duplicate.

Monotonically increasing when required, which also benefits database indexing.

Highly available to handle massive request volumes.

Secure – avoid exposing sequential patterns that could leak information.

Compact to reduce storage pressure.

Common solutions for generating unique IDs in distributed environments include:

01. Using the Database

The simplest approach is to let the database generate auto‑increment primary keys and expose them via a service. For small systems with modest data volume and concurrency, this is sufficient. To reduce database load, batches of IDs can be pre‑generated and cached.

Advantages: Easy to understand and implement.

Disadvantages: Database‑specific implementations make migration difficult; performance becomes a bottleneck under high concurrency; IDs contain little information and are sequential.

02. Using Other Components / Middleware

Components such as Redis, MongoDB, or Zookeeper can generate IDs. Redis uses INCR or INCRBY; MongoDB provides ObjectId; Zookeeper can use znode version numbers.

Example of a MongoDB ObjectId: {"_id": ObjectId("5d47ca7528021724ac19f745")} MongoDB ObjectId is 12 bytes. Versions before 3.2 consist of a 4‑byte timestamp, 3‑byte machine identifier, 2‑byte process ID, and 3‑byte counter. Versions after 3.2 use a 4‑byte timestamp, 5‑byte random value, and 3‑byte incrementing counter.

Advantages: Higher performance than a database, can be clustered, and the ID carries useful information such as a timestamp.

Disadvantages: Introduces additional components and complexity; the ID service becomes a single point of failure.

03. UUID

Universally Unique Identifiers are widely used. They combine MAC address, timestamp, namespace, and random or pseudo‑random values. Different versions exist:

Version 1: Time‑based, includes timestamp, random number, and MAC address (or IP address).

Version 2: DCE security, replaces part of the timestamp with POSIX UID/GID.

Version 3: Name‑based using MD5 hash.

Version 4: Randomly generated, with a small chance of collision.

Version 5: Name‑based using SHA‑1 hash.

Java example:

public class CreateUUID {
    public static void main(String[] args) {
        String uuid = UUID.randomUUID().toString();
        System.out.println("uuid : " + uuid);
        uuid = UUID.randomUUID().toString().replaceAll("-", "");
        System.out.println("uuid : " + uuid);
    }
}

Advantages: Generated locally without network overhead; no third‑party component needed; simple and fast.

Disadvantages: Long length, not storage‑friendly, and lack of ordering can affect database performance.

04. Snowflake

Twitter’s Snowflake algorithm generates 64‑bit IDs locally, avoiding the single‑point risk of external services. The 64‑bit layout is:

1 bit – unused (always 0).

41 bits – timestamp in milliseconds (covers about 69 years).

10 bits – machine identifier (5‑bit datacenter ID + 5‑bit worker ID).

12 bits – sequence number within the same millisecond.

In Java, Snowflake IDs fit into a long type.

Advantages: Local generation, no network cost, no single point of failure, roughly ordered by time, high performance.

Disadvantages: Relies on synchronized clocks; clock rollback can cause duplicate IDs.

Many companies also open‑source ID generation frameworks, such as Meituan’s Leaf and Baidu’s UidGenerator.