Why Distributed IDs Matter and 9 Efficient Generation Methods

1. Why Use Distributed IDs?

Before discussing specific implementations, we analyze why distributed IDs are needed and what characteristics they should have.

1. What is a distributed ID?

Example with MySQL: when data volume grows, a single database cannot meet the need for unique identifiers across sharded tables, so a globally unique ID system is required.

2. What requirements should a distributed ID meet?

Globally unique

High performance

High availability

Easy to integrate

Trend‑incremental (optional)

2. What are the generation methods for distributed IDs?

We analyze nine common generators and their pros and cons.

UUID

Database auto‑increment ID

Database multi‑master mode

Segment (range) mode

Redis

Snowflake algorithm

TinyID (Didi)

Uid‑generator (Baidu)

Leaf (Meituan)

1. UUID

In Java, UUID can be generated with one line of code, but it is not recommended for business IDs because it is a long unordered string, consumes storage, and slows queries.

Advantages:

Simple generation, no network cost, globally unique.

Disadvantages:

No trend‑incremental order.

No business meaning.

Length (36 characters) hurts MySQL performance as primary key.

2. Database auto‑increment ID

Using a dedicated MySQL instance to generate IDs via auto_increment. Example DDL and insert statements are provided.

Advantages:

Simple implementation, monotonic increasing numeric IDs are fast to query.

Disadvantages:

Single‑point failure and bottleneck under high concurrency.

3. Database cluster mode

Using master‑slave or dual‑master clusters with different auto_increment offsets and steps to avoid duplicate IDs.

Configuration examples for two MySQL instances are shown.

Advantages:

Solves single‑node DB issue.

Disadvantages:

Complicated expansion and still limited by DB performance.

4. Segment (range) mode

Fetch a range of IDs from the database and allocate them locally. Table schema and update logic with optimistic lock are described.

5. Redis

Use Redis INCR command for atomic increment. Persistence considerations (RDB vs AOF) are discussed.

6. Snowflake algorithm

Twitter's Snowflake generates 64‑bit IDs composed of sign bit, timestamp, machine ID, data center ID, and sequence. Java implementation code is provided.

/**
 * Twitter's SnowFlake algorithm, using SnowFlake algorithm to generate an integer, then convert to base‑62 as a short URL
 * https://github.com/beyondfengyu/SnowFlake
 */
public class SnowFlakeShortUrl {
    private static final long START_TIMESTAMP = 1480166465631L;
    private static final long SEQUENCE_BIT = 12L;   // sequence bits
    private static final long MACHINE_BIT = 5L;      // machine bits
    private static final long DATA_CENTER_BIT = 5L; // data center bits
    private static final long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT);
    private static final long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT);
    private static final long MAX_DATA_CENTER_NUM = -1L ^ (-1L << DATA_CENTER_BIT);
    private static final long MACHINE_LEFT = SEQUENCE_BIT;
    private static final long DATA_CENTER_LEFT = SEQUENCE_BIT + MACHINE_BIT;
    private static final long TIMESTAMP_LEFT = DATA_CENTER_LEFT + DATA_CENTER_BIT;
    private long dataCenterId;
    private long machineId;
    private long sequence = 0L;
    private long lastTimeStamp = -1L;
    private long getNextMill() {
        long mill = getNewTimeStamp();
        while (mill <= lastTimeStamp) {
            mill = getNewTimeStamp();
        }
        return mill;
    }
    private long getNewTimeStamp() {
        return System.currentTimeMillis();
    }
    public SnowFlakeShortUrl(long dataCenterId, long machineId) {
        if (dataCenterId > MAX_DATA_CENTER_NUM || dataCenterId < 0) {
            throw new IllegalArgumentException("DtaCenterId can't be greater than MAX_DATA_CENTER_NUM or less than 0！");
        }
        if (machineId > MAX_MACHINE_NUM || machineId < 0) {
            throw new IllegalArgumentException("MachineId can't be greater than MAX_MACHINE_NUM or less than 0！");
        }
        this.dataCenterId = dataCenterId;
        this.machineId = machineId;
    }
    public synchronized long nextId() {
        long currTimeStamp = getNewTimeStamp();
        if (currTimeStamp < lastTimeStamp) {
            throw new RuntimeException("Clock moved backwards.  Refusing to generate id");
        }
        if (currTimeStamp == lastTimeStamp) {
            sequence = (sequence + 1) & MAX_SEQUENCE;
            if (sequence == 0L) {
                currTimeStamp = getNextMill();
            }
        } else {
            sequence = 0L;
        }
        lastTimeStamp = currTimeStamp;
        return (currTimeStamp - START_TIMESTAMP) << TIMESTAMP_LEFT
                | dataCenterId << DATA_CENTER_LEFT
                | machineId << MACHINE_LEFT
                | sequence;
    }
    public static void main(String[] args) {
        SnowFlakeShortUrl snowFlake = new SnowFlakeShortUrl(2, 3);
        for (int i = 0; i < (1 << 4); i++) {
            System.out.println(snowFlake.nextId());
        }
    }
}

7. Baidu uid‑generator

Based on Snowflake but allows custom timestamp, workId, and sequence bits. Requires a WORKER_NODE table to obtain a workId for each machine.

8. Meituan Leaf

Supports both segment mode and Snowflake mode; configuration files and table definitions are shown.

9. Didi TinyID

Implements segment mode similar to Leaf. Provides HTTP and Java client integration with example SQL, configuration, and usage code.

Conclusion

The article gives a brief overview of each distributed ID generator, highlighting their advantages and drawbacks; the choice depends on specific business requirements.