Designing Distributed Global Unique ID Generation: Snowflake Algorithm and Practical Implementations

Preface

System unique IDs are a frequent design challenge; this article offers a distributed global unique ID generation scheme.

Problem

Why distributed global unique IDs are needed and their business requirements

In complex distributed systems like Meituan, Dianping, and other large‑scale services, massive data and messages require unique identifiers for orders, riders, coupons, etc., especially after sharding databases.

ID Generation Hard Requirements

Globally unique

Trend‑increasing

Monotonically increasing

Security (non‑predictable)

Contains timestamp

Usability Requirements for ID Generation System

High availability (99.999% success)

Low latency

High QPS (e.g., 100k concurrent requests)

Common Solutions

UUID

UUID.randomUUID() generates 36‑character strings with high performance locally, but they are unordered, large, and hurt MySQL InnoDB index performance.

Problems

Unordered, cannot provide incremental IDs

Long primary keys degrade insert performance

Index fragmentation in B‑tree structures

Database Auto‑Increment Primary Key

Single‑node

Uses REPLACE INTO to insert rows; the auto‑increment value increases, satisfying uniqueness, monotonicity, and incrementality for low‑concurrency distributed scenarios.

REPLACE into t_test(stub) values('b');
select LAST_INSERT_ID();

Clustered Distributed

Not ideal due to horizontal scaling difficulty and high database load for each ID request.

Redis‑Based Global ID Strategy

Single‑node

Redis is single‑threaded, guaranteeing atomic INCR and INCRBY operations.

Clustered

Set different step sizes per node and assign TTL to keys; e.g., five nodes with initial values 1‑5 and step 5 produce IDs like:

A: 1 6 11 16 21
B: 2 7 12 17 22
C: 3 8 13 18 23
D: 4 9 14 19 24
E: 5 10 15 20 25

Maintenance of Redis clusters can be complex.

Snowflake Algorithm

What It Is

Twitter's distributed self‑increment ID algorithm, generating up to 260,000 ordered IDs per second.

Structure

1 sign bit (always 0)

41‑bit timestamp (milliseconds, ~69.7 years)

10‑bit machine identifier (5‑bit datacenter + 5‑bit worker)

12‑bit sequence within the same millisecond (0‑4095)

Implementation (Java)

/**
 * twitter's snowflake algorithm -- java implementation
 */
public class SnowFlake {
    private static final long START_STMP = 1480166465631L;
    private static final long SEQUENCE_BIT = 12L;
    private static final long MACHINE_BIT = 5L;
    private static final long DATACENTER_BIT = 5L;
    private static final long MAX_DATACENTER_NUM = -1L ^ (-1L << DATACENTER_BIT);
    private static final long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT);
    private static final long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT);
    private static final long MACHINE_LEFT = SEQUENCE_BIT;
    private static final long DATACENTER_LEFT = SEQUENCE_BIT + MACHINE_BIT;
    private static final long TIMESTMP_LEFT = DATACENTER_LEFT + DATACENTER_BIT;
    private long datacenterId;
    private long machineId;
    private long sequence = 0L;
    private long lastStmp = -1L;
    public SnowFlake(long datacenterId, long machineId) { /* validation omitted */ }
    public synchronized long nextId() { /* core logic omitted */ }
    private long getNextMill() { return System.currentTimeMillis(); }
    public static void main(String[] args) { /* demo omitted */ }
}

Engineering Experience

hutool Utility Package

https://github.com/looly/hutool

SpringBoot Integration

<dependency>
    <groupId>cn.hutool</groupId>
    <artifactId>hutool-all</artifactId>
    <version>5.3.1</version>
</dependency>

/**
 * Snowflake demo using hutool
 */
public class SnowFlakeDemo {
    private long workerId = 0L;
    private long datacenterId = 1L;
    private Snowflake snowFlake = IdUtil.createSnowflake(workerId, datacenterId);
    @PostConstruct
    public void init() { /* convert IP to long */ }
    public synchronized long snowflakeId() { return this.snowFlake.nextId(); }
    public static void main(String[] args) { /* multithread test */ }
}

Pros

Timestamp in high bits, sequence in low bits → trend‑increasing IDs

No dependency on external databases; high performance as a service

Flexible bit allocation for different business needs

Cons

Relies on system clock; clock rollback can cause duplicate IDs

Across machines, global monotonicity is not guaranteed, only trend‑increasing

Other Alternatives

Baidu's open‑source UidGenerator

Meituan's Leaf ID generator