Distributed ID Concepts, Implementation Schemes, and Open‑Source Solutions

In distributed systems a unique identifier (ID) must be globally unique, similar to a personal ID in the real world, and should support high concurrency, high availability, and high performance.

Common ID generation schemes are compared in the table below:

Scheme

Description

Advantages

Disadvantages

UUID

Universally Unique Identifier that does not require a central coordinator.

Reduces pressure on global nodes, fast primary‑key generation, globally unique, easy data merging across servers.

Consumes 16 characters, not ordered, causes random I/O and index inefficiency.

Database auto‑increment

MySQL auto‑increment primary key.

Small space (INT/BIGINT), sequential I/O, numeric queries faster than strings.

Limited concurrency, not suitable for sharding, reveals data volume.

Redis auto‑increment

Atomic counter in Redis.

In‑memory, excellent concurrency.

Potential data loss, also reveals data volume.

Snowflake algorithm

Classic distributed‑ID solution using bit fields.

No external dependencies, high performance.

Clock rollback issues.

The two most popular solutions today are the segment mode and the Snowflake algorithm .

Segment mode relies on a database but allocates a block of IDs (e.g., 100, 200, 300) at a time, greatly improving performance compared with simple auto‑increment.

Snowflake composes an ID from a sign bit (0 for positive), a timestamp (milliseconds), a datacenter ID (typically 5 bits), a machine ID (5 bits), and a sequence number (12 bits). The sign bit ensures the ID is positive.

Capacity analysis: timestamp bits (41) give a range of about 69 years; worker ID bits (10) allow 1,024 machines; sequence bits (12) allow 4,096 IDs per millisecond.

public class SnowFlake {

    /** Starting timestamp */
    private final static long START_STMP = 1480166465631L;

    /** Bits allocated to each part */
    private final static long SEQUENCE_BIT = 12; // sequence bits
    private final static long MACHINE_BIT = 5;   // machine id bits
    private final static long DATACENTER_BIT = 5; // datacenter bits

    /** Maximum values for each part */
    private final static long MAX_DATACENTER_NUM = -1L ^ (-1L << DATACENTER_BIT);
    private final static long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT);
    private final static long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT);

    /** Left shift amounts */
    private final static long MACHINE_LEFT = SEQUENCE_BIT;
    private final static long DATACENTER_LEFT = SEQUENCE_BIT + MACHINE_BIT;
    private final static long TIMESTMP_LEFT = DATACENTER_LEFT + DATACENTER_BIT;

    private long datacenterId; // datacenter
    private long machineId;    // machine
    private long sequence = 0L;
    private long lastStmp = -1L; // last timestamp

    public SnowFlake(long datacenterId, long machineId) {
        if (datacenterId > MAX_DATACENTER_NUM || datacenterId < 0) {
            throw new IllegalArgumentException("datacenterId can't be greater than MAX_DATACENTER_NUM or less than 0");
        }
        if (machineId > MAX_MACHINE_NUM || machineId < 0) {
            throw new IllegalArgumentException("machineId can't be greater than MAX_MACHINE_NUM or less than 0");
        }
        this.datacenterId = datacenterId;
        this.machineId = machineId;
    }

    /** Generate next ID */
    public synchronized long nextId() {
        long currStmp = getNewstmp();
        if (currStmp < lastStmp) {
            throw new RuntimeException("Clock moved backwards.  Refusing to generate id");
        }
        if (currStmp == lastStmp) {
            // same millisecond, increment sequence
            sequence = (sequence + 1) & MAX_SEQUENCE;
            if (sequence == 0L) {
                // sequence overflow, wait for next millisecond
                currStmp = getNextMill();
            }
        } else {
            // different millisecond, reset sequence
            sequence = 0L;
        }
        lastStmp = currStmp;
        return (currStmp - START_STMP) << TIMESTMP_LEFT // timestamp
                | datacenterId << DATACENTER_LEFT      // datacenter
                | machineId << MACHINE_LEFT            // machine
                | sequence;                            // sequence
    }

    private long getNextMill() {
        long mill = getNewstmp();
        while (mill <= lastStmp) {
            mill = getNewstmp();
        }
        return mill;
    }

    private long getNewstmp() {
        return System.currentTimeMillis();
    }

    public static void main(String[] args) {
        SnowFlake snowFlake = new SnowFlake(2, 3);
        for (int i = 0; i < (1 << 12); i++) {
            System.out.println(snowFlake.nextId());
        }
    }
}

When choosing an open‑source component, consider feature fit, compatibility, extensibility, team expertise, and community activity.

Meituan Leaf is a distributed ID service offering global uniqueness, monotonic increase, high availability, and low latency (over 50k QPS with <1 ms tail latency). It supports both segment mode and Snowflake and is widely used within Meituan.

Baidu UidGenerator implements Snowflake with custom worker‑id bits, uses a ring buffer to cache IDs, and achieves up to 6 million QPS on a single machine, but its repository has not been actively maintained for two years and only supports Snowflake.

Overall, Meituan Leaf is generally preferred due to its richer feature set and active maintenance.

The article ends by inviting readers to share other common distributed‑ID solutions they know.