Mastering Distributed ID Generation: Concepts, Algorithms, and Open‑Source Solutions

Distributed ID Concept

In distributed systems a unique identifier (ID) is required for massive data and message tracking. Unlike single‑node auto‑increment IDs, a distributed ID must support high concurrency, high availability, and high performance.

Implementation Schemes

Common approaches include:

UUID – globally unique without a central coordinator; fast key generation but 16‑byte size and non‑sequential, hurting index performance.

Database auto‑increment – small INT/BIGINT space and sequential I/O, but limited concurrency, complex sharding, and potential data leakage.

Redis atomic counter – in‑memory, high concurrency, but vulnerable to data loss.

Snowflake algorithm – decentralized, high‑performance IDs with configurable bit fields.

Snowflake Algorithm Structure

The 64‑bit ID consists of a sign bit (0 for positive), a timestamp in milliseconds, a 5‑bit data‑center ID, a 5‑bit machine ID, and a 12‑bit sequence number, allowing 4096 IDs per millisecond and a usable lifespan of about 69 years.

Java implementation of the Snowflake algorithm:

public class SnowFlake {
    private final static long START_STMP = 1480166465631L;
    private final static long SEQUENCE_BIT = 12; // sequence bits
    private final static long MACHINE_BIT = 5;   // machine bits
    private final static long DATACENTER_BIT = 5; // datacenter bits

    private final static long MAX_DATACENTER_NUM = -1L ^ (-1L << DATACENTER_BIT);
    private final static long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT);
    private final static long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT);

    private final static long MACHINE_LEFT = SEQUENCE_BIT;
    private final static long DATACENTER_LEFT = SEQUENCE_BIT + MACHINE_BIT;
    private final static long TIMESTMP_LEFT = DATACENTER_LEFT + DATACENTER_BIT;

    private long datacenterId;
    private long machineId;
    private long sequence = 0L;
    private long lastStmp = -1L;

    public SnowFlake(long datacenterId, long machineId) {
        if (datacenterId > MAX_DATACENTER_NUM || datacenterId < 0) {
            throw new IllegalArgumentException("datacenterId can't be greater than MAX_DATACENTER_NUM or less than 0");
        }
        if (machineId > MAX_MACHINE_NUM || machineId < 0) {
            throw new IllegalArgumentException("machineId can't be greater than MAX_MACHINE_NUM or less than 0");
        }
        this.datacenterId = datacenterId;
        this.machineId = machineId;
    }

    public synchronized long nextId() {
        long currStmp = getNewstmp();
        if (currStmp < lastStmp) {
            throw new RuntimeException("Clock moved backwards.  Refusing to generate id");
        }
        if (currStmp == lastStmp) {
            sequence = (sequence + 1) & MAX_SEQUENCE;
            if (sequence == 0L) {
                currStmp = getNextMill();
            }
        } else {
            sequence = 0L;
        }
        lastStmp = currStmp;
        return (currStmp - START_STMP) << TIMESTMP_LEFT
                | datacenterId << DATACENTER_LEFT
                | machineId << MACHINE_LEFT
                | sequence;
    }

    private long getNextMill() {
        long mill = getNewstmp();
        while (mill <= lastStmp) {
            mill = getNewstmp();
        }
        return mill;
    }

    private long getNewstmp() {
        return System.currentTimeMillis();
    }

    public static void main(String[] args) {
        SnowFlake snowFlake = new SnowFlake(2, 3);
        for (int i = 0; i < (1 << 12); i++) {
            System.out.println(snowFlake.nextId());
        }
    }
}

Popular Open‑Source Components

Meituan Leaf

Leaf is a distributed ID service from Meituan’s R&D platform, offering high reliability, low latency, and globally unique IDs. It supports both segment mode and Snowflake algorithm and is widely used across Meituan’s business units.

Baidu UidGenerator

UidGenerator is Baidu’s Java‑based Snowflake implementation. It uses future timestamps to avoid sequence contention and a RingBuffer to boost throughput, achieving up to 6 million QPS on a single node, but its maintenance has slowed.

Comparing the two, Leaf provides more features (segment + Snowflake) and more recent maintenance, making it the preferable choice for most projects.