Mastering Distributed ID Generation: 9 Proven Methods and Their Trade‑offs

Why Use Distributed IDs?

Before diving into specific implementations, we first examine why distributed IDs are needed and what characteristics they must satisfy.

What is a Distributed ID?

Consider a simple MySQL database. When data volume is small, a single database table can handle the workload, and master‑slave replication may suffice. As data grows, sharding becomes necessary, and a globally unique identifier is required because auto‑increment IDs can no longer guarantee uniqueness across shards. This globally unique identifier is called a

distributed ID

.

Requirements for Distributed IDs

Global uniqueness: IDs must be unique across the entire system.

High performance: ID generation should be fast and low‑latency to avoid becoming a bottleneck.

High availability: Availability should be as close to 100% as possible.

Easy integration: The design should allow plug‑and‑play usage.

Trend increment (optional): Preferably IDs increase monotonically, though this depends on business needs.

Generation Methods

We analyze nine common distributed ID generation approaches and discuss their pros and cons.

UUID

Database auto‑increment ID

Database multi‑master mode

Segment (range) mode

Redis

Snowflake algorithm

TinyID (Didi)

Uid‑generator (Baidu)

Leaf (Meituan)

1. UUID

In Java, the simplest way to obtain a unique ID is using

UUID

. While UUIDs are globally unique, they are not recommended for most business scenarios because they are long, unordered strings that lack business meaning and degrade database performance.

Advantages:

Simple to generate locally without network overhead.

Disadvantages:

Unordered string, no monotonic increment.

No business semantics.

36‑character string consumes more storage and slows queries; MySQL recommends short primary keys.

<code>public static void main(String[] args) {
    String uuid = UUID.randomUUID().toString().replaceAll("-", "");
    System.out.println(uuid);
}
</code>

2. Database Auto‑Increment ID

Using a dedicated MySQL instance to generate IDs via

auto_increment

is straightforward. The table schema might look like:

<code>CREATE DATABASE `SEQ_ID`;
CREATE TABLE SEQID.SEQUENCE_ID (
    id BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
    value CHAR(10) NOT NULL DEFAULT '',
    PRIMARY KEY (id)
) ENGINE=MyISAM;
</code>

Insert a row to obtain a new ID:

<code>INSERT INTO SEQUENCE_ID(value) VALUES ('values');
</code>

Advantages:

Simple implementation; IDs are monotonic integers with fast query performance.

Disadvantages:

Single‑point failure; MySQL becomes a bottleneck under high concurrency.

3. Database Multi‑Master Mode

To improve availability, a master‑slave cluster can be upgraded to a dual‑master setup, each generating auto‑increment IDs. By configuring different

auto_increment_offset

and

auto_increment_increment

values, duplicate IDs are avoided.

<code>-- MySQL_1 configuration
SET @@auto_increment_offset = 1;   -- start value
SET @@auto_increment_increment = 2; -- step

-- MySQL_2 configuration
SET @@auto_increment_offset = 2;
SET @@auto_increment_increment = 2;
</code>

Resulting ID sequences:

1,3,5,7,9 … and 2,4,6,8,10 …

While this solves the single‑point issue, scaling beyond two nodes requires manual reconfiguration of offsets and steps, which is cumbersome.

4. Segment (Range) Mode

Segment mode fetches a range of IDs from the database in batches, reducing database load. The table schema typically includes fields for the current maximum ID, step size, business type, and version for optimistic locking.

<code>CREATE TABLE id_generator (
    id BIGINT NOT NULL,
    biz_type INT NOT NULL,
    max_id BIGINT NOT NULL COMMENT 'current max ID',
    step INT NOT NULL COMMENT 'range length',
    version INT NOT NULL COMMENT 'optimistic lock version',
    PRIMARY KEY (id)
) ENGINE=InnoDB;
</code>

When a service exhausts its local range, it updates the

max_id

atomically:

<code>UPDATE id_generator
SET max_id = max_id + step, version = version + 1
WHERE version = #{version} AND biz_type = #{biz_type};
</code>

This approach minimizes database hits and works well under high concurrency.

5. Redis

Redis can generate IDs atomically using the

INCR

command. Example:

<code>127.0.0.1:6379> SET seq_id 1   // initialize
OK
127.0.0.1:6379> INCR seq_id   // increment and return
(integer) 2
</code>

When using Redis, persistence mode must be considered.

RDB

snapshots may lose recent increments if Redis crashes, while

AOF

guarantees durability but can slow recovery due to the high frequency of

INCR

commands.

6. Snowflake Algorithm

Twitter's Snowflake generates 64‑bit IDs composed of a sign bit, 41‑bit timestamp, 5‑bit machine ID, 5‑bit data‑center ID, and 12‑bit sequence. The algorithm ensures monotonic, time‑ordered IDs.

Java implementation (simplified):

<code>public class SnowFlakeShortUrl {
    private static final long START_TIMESTAMP = 1480166465631L;
    private static final long SEQUENCE_BIT = 12L;
    private static final long MACHINE_BIT = 5L;
    private static final long DATA_CENTER_BIT = 5L;
    private static final long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT);
    private static final long MACHINE_LEFT = SEQUENCE_BIT;
    private static final long DATA_CENTER_LEFT = SEQUENCE_BIT + MACHINE_BIT;
    private static final long TIMESTAMP_LEFT = DATA_CENTER_LEFT + DATA_CENTER_BIT;
    private long dataCenterId;
    private long machineId;
    private long sequence = 0L;
    private long lastTimeStamp = -1L;
    // constructors, getters, and nextId() omitted for brevity
    public synchronized long nextId() {
        long currTimeStamp = System.currentTimeMillis();
        if (currTimeStamp < lastTimeStamp) {
            throw new RuntimeException("Clock moved backwards. Refusing to generate id");
        }
        if (currTimeStamp == lastTimeStamp) {
            sequence = (sequence + 1) & MAX_SEQUENCE;
            if (sequence == 0L) {
                currTimeStamp = waitNextMillis();
            }
        } else {
            sequence = 0L;
        }
        lastTimeStamp = currTimeStamp;
        return (currTimeStamp - START_TIMESTAMP) << TIMESTAMP_LEFT
                | dataCenterId << DATA_CENTER_LEFT
                | machineId << MACHINE_LEFT
                | sequence;
    }
    private long waitNextMillis() {
        long mill = System.currentTimeMillis();
        while (mill <= lastTimeStamp) {
            mill = System.currentTimeMillis();
        }
        return mill;
    }
    public static void main(String[] args) {
        SnowFlakeShortUrl sf = new SnowFlakeShortUrl(2, 3);
        for (int i = 0; i < 16; i++) {
            System.out.println(sf.nextId());
        }
    }
}
</code>

7. TinyID (Didi)

TinyID implements a segment‑based approach similar to Leaf. It provides both HTTP and Java client interfaces.

HTTP example:

<code>GET http://localhost:9999/tinyid/id/nextIdSimple?bizType=test&token=0f673adf80504e2eaa552f5d791b644c
Response: 3
</code>

Java client usage:

<code>// Obtain a single ID
Long id = TinyId.nextId("test");
// Obtain a batch of IDs
List<Long> ids = TinyId.nextId("test", 10);
</code>

8. Uid‑generator (Baidu)

Baidu's

uid‑generator

is based on Snowflake but allows custom bit allocations for timestamp, work‑ID, and sequence. It stores a

WORKER_NODE

table to assign a unique

workId

to each instance.

9. Leaf (Meituan)

Leaf supports both segment mode and Snowflake mode. In segment mode, it uses a MySQL table

leaf_alloc

to allocate ID ranges. Configuration example:

<code>leaf.name=com.sankuai.leaf.opensource.test
leaf.segment.enable=true
leaf.jdbc.url=jdbc:mysql://localhost:3306/leaf_test?useUnicode=true&characterEncoding=utf8
leaf.jdbc.username=root
leaf.jdbc.password=root
leaf.snowflake.enable=false
</code>

After starting

leaf-server

, the segment API can be accessed at

http://localhost:8080/api/segment/get/leaf-segment-test

.

Conclusion

This article provides a brief overview of various distributed ID generators, highlighting that each method has its own strengths and weaknesses. The choice of implementation should be driven by specific business requirements.