Choosing the Right Distributed ID Strategy: UUID, Snowflake, Segment, Redis, and More

Background

In complex distributed systems, unique identifiers are needed for large amounts of data, e.g., when sharding an order table the auto‑increment ID cannot be used. Requirements include trend‑increasing, monotonic increasing, and security.

Trend increasing: Use ordered primary keys for better write performance.

Monotonic increasing: Ensure each new ID is greater than the previous one.

Security: Avoid predictable sequential IDs that expose business volume.

The article introduces several distributed ID solutions, their pros, cons, use cases, and code examples.

1. UUID

UUID (Universally Unique Identifier) is generated from time, counter, and hardware MAC address, forming a 36‑character string (8‑4‑4‑4‑12). Java provides UUID.randomUUID() which outputs a 128‑bit value.

import java.util.UUID;

public class Test {
    public static void main(String[] args) {
        System.out.println(UUID.randomUUID());
    }
}

b0378f6a-eeb7-4779-bffe-2a9f3bc76380

Reasons UUID is rarely used in practice:

High storage cost: 16 bytes (36‑char string) is large for many scenarios.

Security risk: MAC‑based UUID can expose hardware information.

MySQL index impact: Unordered UUIDs cause frequent page splits and degrade performance.

2. Database Auto‑Increment ID

MySQL auto‑increment works per table but not across sharded tables. Two approaches are presented.

2.1 Primary‑Key Table

Create a dedicated table to generate global IDs.

CREATE TABLE `unique_id` (
  `id` bigint NOT NULL AUTO_INCREMENT,
  `biz` char(1) NOT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `biz` (`biz`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;

Insert a row and retrieve LAST_INSERT_ID() to obtain a unique ID.

BEGIN;
REPLACE INTO unique_id (biz) values ('o');
SELECT LAST_INSERT_ID();
COMMIT;

2.2 Auto‑Increment Step

Set different auto‑increment steps for each MySQL instance to avoid collisions.

Check the step with:

show variables like '%increment%';

High concurrency may affect scalability.

3. Segment Mode

Segment mode fetches a range of IDs from the database and caches it in memory; when exhausted, a new range is requested.

Table example:

CREATE TABLE id_generator (
  id int(10) NOT NULL,
  max_id bigint NOT NULL COMMENT 'current max id',
  step int(20) NOT NULL COMMENT 'segment length',
  biz_type int(20) NOT NULL COMMENT 'business type',
  version int(20) NOT NULL COMMENT 'optimistic lock version',
  PRIMARY KEY (`id`)
);

Segment mode reduces database load but suffers from discontinuity after server restarts.

4. Redis INCR

Redis INCR can generate globally unique IDs.

Example implementation:

@Component
public class RedisDistributedId {
    @Autowired
    private StringRedisTemplate redisTemplate;
    private static final long BEGIN_TIMESTAMP = 1659312000L;

    public long nextId(String item) {
        LocalDateTime now = LocalDateTime.now();
        long nowSecond = now.toEpochSecond(ZoneOffset.UTC);
        long timestamp = nowSecond - BEGIN_TIMESTAMP;
        String date = now.format(DateTimeFormatter.ofPattern("yyyy:MM:dd"));
        Long increment = redisTemplate.opsForValue().increment("id:" + item + ":" + date);
        return timestamp << 32 | increment;
    }
}

Redis persistence issues arise if the server crashes.

5. Snowflake Algorithm

Twitter’s Snowflake generates 64‑bit IDs composed of timestamp, datacenter ID, worker ID, and sequence.

Sign bit (1): unused.

Timestamp (41): ~69 years.

Machine ID (10): up to 1024 nodes.

Sequence (12): up to 4096 IDs per millisecond.

Java implementation (simplified):

public class SnowflakeIdWorker {
    private final long twepoch = 1604374294980L;
    private final long workerIdBits = 5L;
    private final long datacenterIdBits = 5L;
    private final long maxWorkerId = -1L ^ (-1L << workerIdBits);
    private final long maxDatacenterId = -1L ^ (-1L << datacenterIdBits);
    private final long sequenceBits = 12L;
    private final long workerIdShift = sequenceBits;
    private final long datacenterIdShift = sequenceBits + workerIdBits;
    private final long timestampLeftShift = sequenceBits + workerIdBits + datacenterIdBits;
    private final long sequenceMask = -1L ^ (-1L << sequenceBits);
    private long workerId;
    private long datacenterId;
    private long sequence = 0L;
    private long lastTimestamp = -1L;

    public synchronized long nextId() {
        long timestamp = timeGen();
        if (timestamp < lastTimestamp) {
            throw new RuntimeException("Clock moved backwards.");
        }
        if (lastTimestamp == timestamp) {
            sequence = (sequence + 1) & sequenceMask;
            if (sequence == 0) {
                timestamp = tilNextMillis(lastTimestamp);
            }
        } else {
            sequence = 0L;
        }
        lastTimestamp = timestamp;
        return ((timestamp - twepoch) << timestampLeftShift)
                | (datacenterId << datacenterIdShift)
                | (workerId << workerIdShift)
                | sequence;
    }
    // helper methods omitted for brevity
}

Clock rollback can cause duplicate IDs; usually mitigated by persisting the last timestamp.

6. Meituan Leaf

Open‑source project supporting both segment and Snowflake modes. Snowflake mode relies on ZooKeeper to assign work IDs.

7. Baidu UidGenerator

Java implementation based on Snowflake, optimized for higher QPS (over 6 M). Requires MySQL for worker ID allocation.

Time part uses 28 bits, giving ~8.5 years of range; can be re‑configured.

8. Didi TinyID

Built on Leaf’s segment algorithm, adds a TinyID client/server architecture. Supports only segment mode.

Comparison Summary

Each solution balances uniqueness, ordering, storage cost, performance, and operational complexity. Choose based on system requirements such as ID length, generation rate, fault tolerance, and infrastructure constraints.