Skip to content
Distributed ID Generation: 8 Strategies from UUID to Snowflake

Distributed ID Generation: 8 Strategies from UUID to Snowflake

This article compares eight distributed ID generation approaches—UUID, database auto-increment, segment mode, Redis, Snowflake, Baidu UidGenerator, Meituan Leaf, and Didi TinyID—detailing their mechanisms, pros, cons, and real-world implementations to help choose the right solution for scalable systems.

Architect's Guide
Architect's Guide
Architect's Guide

Why Distributed IDs Are Needed

In monolithic systems, primary keys often use auto-increment. After sharding databases and tables, auto-increment IDs can collide across shards, violating business uniqueness requirements.

Requirements for Distributed IDs

Global uniqueness : IDs must never repeat.

Trend increasing : Ordered IDs benefit MySQL InnoDB clustered index write performance.

Monotonic increasing : Each ID greater than the previous, useful for transaction versioning and sorting.

Information security : IDs should increase but not be consecutive, preventing malicious scraping of sequential data.

Eight Distributed ID Generation Solutions

3.1 UUID

Universally Unique Identifier, 32 hex digits in 8-4-4-4-12 format (36 chars). Total space: 2^128 ≈ 3.4×10^38.

Pros : High performance, local generation, no network dependency.

Cons : Storage inefficient (16 bytes, 36-char string); MAC-based UUID may leak MAC address; unordered nature causes frequent page splits and performance degradation.

3.2 Database Auto-increment (Flickr Ticket Servers)

Described by Flickr in 2010. Deploy N machines, each with unique initial value and step = N. Example: 2 machines, step=2. Server1 starts at 1 (1,3,5…), Server2 at 2 (2,4,6…).

Drawbacks : Scaling requires reconfiguring step and initial values on all machines; IDs only trend increasing, not strictly monotonic; every ID fetch hits the database, high pressure.

3.3 Segment Mode

Fetch a range (e.g., [1,1000]) from DB, cache in memory. When exhausted, fetch next segment.

CREATE TABLE id_generator (</code><code>  id int(10) NOT NULL,</code><code>  max_id bigint(20) NOT NULL COMMENT 'current max id',</code><code>  step int(20) NOT NULL COMMENT 'segment step',</code><code>  biz_type int(20) NOT NULL COMMENT 'business type',</code><code>  version int(20) NOT NULL COMMENT 'version for optimistic lock',</code><code>  PRIMARY KEY (`id`)</code><code>)

Update with optimistic locking:

update id_generator set max_id = #{max_id+step}, version = version + 1 where version = #{version} and biz_type = XXX

Pros : Mature implementations exist (Baidu UidGenerator, Meituan Leaf).

Cons : Depends on database availability.

3.4 Redis Implementation

Uses atomic INCR/INCRBY commands. Redis single-threaded nature guarantees uniqueness and order. For high concurrency, cluster with segmentation (step/range) similar to DB approach.

Pros : Good performance, ensures uniqueness and order.

Cons : Introduces Redis dependency.

3.5 Snowflake Algorithm

Twitter open-source 64-bit ID layout:

1 sign bit (always 0)

41-bit timestamp (milliseconds) → 2^41 ms ≈ 69 years

10-bit worker ID → 1024 machines

12-bit sequence → 4096 IDs per millisecond

Snowflake bit layout
Snowflake bit layout

Pros : Trend increasing, no third-party dependency, high throughput, flexible bit allocation.

Cons : Strong dependency on machine clock; clock rollback causes duplicate IDs or service unavailability.

3.6 Baidu UidGenerator

Java implementation based on Snowflake with improvements:

Customizable workerId bits and initialization for Docker/virtualized environments (auto-restart, drift).

Borrows future time to overcome sequence concurrency limit.

RingBuffer caches generated UIDs, parallelizes production/consumption, CacheLine padding avoids false sharing.

Single-machine QPS up to 6 million.

Bit allocation (differs from standard Snowflake):

sign (1 bit)

delta seconds (28 bits, relative to 2016-05-20, ~8.7 years)

worker id (22 bits, ~4.2M startups, allocated via DB at startup)

sequence (13 bits, 8192 per second)

Reference: GitHub README at https://github.com/baidu/uid-generator/blob/master/README.zh_cn.md

3.7 Meituan Leaf

Two modes: Leaf-segment (DB) and Leaf-snowflake.

Leaf-segment

Proxy server batch-fetches segments (step size), reducing DB pressure. biz_tag isolates different business ID sequences; scaling via biz_tag sharding.

+-------------+--------------+------+-----+-------------------+-----------------------------+</code><code>| Field       | Type         | Null | Key | Default           | Extra                       |</code><code>+-------------+--------------+------+-----+-------------------+-----------------------------+</code><code>| biz_tag     | varchar(128) | NO   | PRI |                   |                             |</code><code>| max_id      | bigint(20)   | NO   |     | 1                 |                             |</code><code>| step        | int(11)      | NO   |     | NULL              |                             |</code><code>| desc        | varchar(256) | YES  |     | NULL              |                             |</code><code>| update_time | timestamp    | NO   |     | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |</code><code>+-------------+--------------+------+-----+-------------------+-----------------------------+

Leaf-snowflake

Uses Zookeeper persistent sequential nodes for automatic workerID assignment:

Start Leaf-snowflake, connect ZK, check registration under leaf_forever parent node.

If registered, retrieve workerID (sequential node int ID), start service.

If not registered, create persistent sequential node under parent, use sequence number as workerID, start service.

This solves Snowflake's clock dependency. Architecture diagram:

Leaf-snowflake architecture
Leaf-snowflake architecture

Reference: Meituan Tech Blog at https://tech.meituan.com/2017/04/21/mt-leaf.html

3.8 Didi TinyID

Java distributed ID system based on DB segment algorithm. Extends leaf-segment with multi-DB support and tinyid-client.

Architecture diagram:

TinyID architecture
TinyID architecture

Pros : Easy integration, mature implementation.

Cons : Depends on DB stability; requires cluster master-slave for high availability.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

redisuuiddistributed-idsnowflake-algorithmsegment-modedatabase-auto-incrementmeituan-leafbaidu-uidgenerator
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.