Choosing the Right Globally Unique ID Strategy: From Auto‑Increment to Snowflake

Globally unique IDs are a near‑universal requirement for systems, essential for search, data storage, and fast retrieval. Various strategies exist, each with trade‑offs regarding uniqueness, ordering, scalability, and performance.

Simple Requirement Analysis

The core business need is to generate a unique record identifier, often used as a primary key with a clustered index. To support pagination or sorting, a separate time field is usually indexed, but if IDs are generated in roughly time order, the extra index can be omitted.

Global uniqueness

Trend‑ordered (monotonically increasing) values

Pros and Cons of Common Generation Strategies

Method 1: Use Database auto_increment

Advantages:

Leverages built‑in database functionality, simple to use

Guarantees uniqueness

Guarantees incrementality

Step size is fixed and configurable

Disadvantages:

Availability issues in master‑slave or read/write split architectures; if the master fails, ID generation stops

Scalability limited by single‑node write throughput

Improvement:

Redundant masters to avoid a single write point

Horizontal sharding with distinct auto_increment offsets per shard

The diagram shows three write databases, each configured with a different auto_increment start value and the same step size, ensuring non‑overlapping ID ranges (e.g., DB01 generates 0,3,6,…; DB02 generates 1,4,7,…; DB03 generates 2,5,8,…).

While this improves availability, it sacrifices absolute incrementality and still places write pressure on the databases.

Method 2: Single‑Node Batch ID Service

Distributed systems lack a global clock, making absolute ordering difficult. A single service can use the local clock to provide ordered IDs, and batch fetching reduces database write load.

The architecture uses dual masters for high availability; the database stores only the current maximum ID. The service pulls a batch (e.g., 5 IDs), updates the max value, and then serves IDs locally without further DB hits, reducing pressure to roughly one‑sixth of the original load.

Advantages:

Provides absolute, ordered ID generation

Greatly reduces database load, enabling tens of thousands of IDs per second

Disadvantages:

Service remains a single point of failure

Restart may create gaps in the sequence

Performance ceiling prevents horizontal scaling

Improvement:

Introduce a standby (shadow) service that takes over automatically if the primary fails, typically using VIP + keepalived for transparent failover

Method 3: UUID / GUID

Local generation avoids remote calls, offering low latency and virtually unlimited scalability. UUID uuid = UUID.randomUUID(); Advantages:

Generated locally, no network latency

Excellent scalability, effectively no performance ceiling

Disadvantages:

Does not provide trend ordering

Long string representation leads to inefficient primary‑key indexes; common mitigations include storing as two uint64 values or using half‑length storage, which may affect uniqueness guarantees

Method 4: Use Current Millisecond Timestamp

Generating IDs from the current millisecond timestamp yields locally ordered, integer IDs that index efficiently.

Advantages:

Local generation, low latency

IDs are trend‑ordered

Integer IDs give high index performance

Disadvantages:

Concurrent generation above 1,000 per millisecond can cause collisions

Even using microseconds caps at 1,000,000 IDs per second and still cannot guarantee uniqueness under extreme load

Method 5: Use Redis to Generate ID

Redis’s single‑threaded nature and atomic INCR/INCRBY commands enable fast, globally unique ID generation, outperforming database‑based approaches.

Advantages:

Simple integration, higher performance than relational databases

Numeric IDs are naturally ordered, aiding pagination and sorting

Disadvantages:

Introducing Redis adds system complexity if it is not already present

Requires additional coding and configuration effort

Method 6: Twitter’s Snowflake Algorithm

Snowflake generates 64‑bit IDs composed of:

41 bits for milliseconds since a custom epoch (≈69 years)

10 bits for machine identification (5 bits data center, 5 bits worker)

12 bits for a per‑millisecond sequence (up to 4096 IDs per node per ms)

The algorithm can theoretically generate up to 4 million IDs per second per node, easily meeting most business demands.

package com;

public class SnowflakeIdGenerator {
    // Algorithm parameters
    private final long startTime = 1498608000000L; // 2017‑06‑28 00:00:00 UTC
    private final long workerIdBits = 5L;
    private final long dataCenterIdBits = 5L;
    private final long maxWorkerId = -1L ^ (-1L << workerIdBits);
    private final long maxDataCenterId = -1L ^ (-1L << dataCenterIdBits);
    private final long sequenceBits = 12L;
    private final long workerIdMoveBits = sequenceBits;
    private final long dataCenterIdMoveBits = sequenceBits + workerIdBits;
    private final long timestampMoveBits = sequenceBits + workerIdBits + dataCenterIdBits;
    private final long sequenceMask = -1L ^ (-1L << sequenceBits);

    // Runtime state
    private long workerId;
    private long dataCenterId;
    private long sequence = 0L;
    private long lastTimestamp = -1L;

    public SnowflakeIdGenerator(long workerId, long dataCenterId) {
        if (workerId > maxWorkerId || workerId < 0) {
            throw new IllegalArgumentException(String.format("Worker Id can't be greater than %d or less than 0", maxWorkerId));
        }
        if (dataCenterId > maxDataCenterId || dataCenterId < 0) {
            throw new IllegalArgumentException(String.format("DataCenter Id can't be greater than %d or less than 0", maxDataCenterId));
        }
        this.workerId = workerId;
        this.dataCenterId = dataCenterId;
    }

    public synchronized long nextId() {
        long timestamp = currentTime();
        if (timestamp < lastTimestamp) {
            throw new RuntimeException(String.format("Clock moved backwards. Refusing to generate id for %d milliseconds", lastTimestamp - timestamp));
        }
        if (lastTimestamp == timestamp) {
            sequence = (sequence + 1) & sequenceMask;
            if (sequence == 0) {
                timestamp = blockTillNextMillis(lastTimestamp);
            }
        } else {
            sequence = 0L;
        }
        lastTimestamp = timestamp;
        return ((timestamp - startTime) << timestampMoveBits)
                | (dataCenterId << dataCenterIdMoveBits)
                | (workerId << workerIdMoveBits)
                | sequence;
    }

    protected long blockTillNextMillis(long lastTimestamp) {
        long timestamp = currentTime();
        while (timestamp <= lastTimestamp) {
            timestamp = currentTime();
        }
        return timestamp;
    }

    protected long currentTime() {
        return System.currentTimeMillis();
    }

    public static void main(String[] args) {
        SnowflakeIdGenerator idWorker = new SnowflakeIdGenerator(0, 0);
        for (int i = 0; i < 100; i++) {
            long id = idWorker.nextId();
            System.out.println(id);
        }
    }
}

This Java implementation demonstrates how to generate thread‑safe, globally unique, time‑ordered IDs using Snowflake.