Distributed ID Generation: 8 Strategies from UUID to Snowflake
This article compares eight distributed ID generation approaches—UUID, database auto-increment, segment mode, Redis, Snowflake, Baidu UidGenerator, Meituan Leaf, and Didi TinyID—detailing their mechanisms, pros, cons, and real-world implementations to help choose the right solution for scalable systems.
Why Distributed IDs Are Needed
In monolithic systems, primary keys often use auto-increment. After sharding databases and tables, auto-increment IDs can collide across shards, violating business uniqueness requirements.
Requirements for Distributed IDs
Global uniqueness : IDs must never repeat.
Trend increasing : Ordered IDs benefit MySQL InnoDB clustered index write performance.
Monotonic increasing : Each ID greater than the previous, useful for transaction versioning and sorting.
Information security : IDs should increase but not be consecutive, preventing malicious scraping of sequential data.
Eight Distributed ID Generation Solutions
3.1 UUID
Universally Unique Identifier, 32 hex digits in 8-4-4-4-12 format (36 chars). Total space: 2^128 ≈ 3.4×10^38.
Pros : High performance, local generation, no network dependency.
Cons : Storage inefficient (16 bytes, 36-char string); MAC-based UUID may leak MAC address; unordered nature causes frequent page splits and performance degradation.
3.2 Database Auto-increment (Flickr Ticket Servers)
Described by Flickr in 2010. Deploy N machines, each with unique initial value and step = N. Example: 2 machines, step=2. Server1 starts at 1 (1,3,5…), Server2 at 2 (2,4,6…).
Drawbacks : Scaling requires reconfiguring step and initial values on all machines; IDs only trend increasing, not strictly monotonic; every ID fetch hits the database, high pressure.
3.3 Segment Mode
Fetch a range (e.g., [1,1000]) from DB, cache in memory. When exhausted, fetch next segment.
CREATE TABLE id_generator (</code><code> id int(10) NOT NULL,</code><code> max_id bigint(20) NOT NULL COMMENT 'current max id',</code><code> step int(20) NOT NULL COMMENT 'segment step',</code><code> biz_type int(20) NOT NULL COMMENT 'business type',</code><code> version int(20) NOT NULL COMMENT 'version for optimistic lock',</code><code> PRIMARY KEY (`id`)</code><code>)Update with optimistic locking:
update id_generator set max_id = #{max_id+step}, version = version + 1 where version = #{version} and biz_type = XXXPros : Mature implementations exist (Baidu UidGenerator, Meituan Leaf).
Cons : Depends on database availability.
3.4 Redis Implementation
Uses atomic INCR/INCRBY commands. Redis single-threaded nature guarantees uniqueness and order. For high concurrency, cluster with segmentation (step/range) similar to DB approach.
Pros : Good performance, ensures uniqueness and order.
Cons : Introduces Redis dependency.
3.5 Snowflake Algorithm
Twitter open-source 64-bit ID layout:
1 sign bit (always 0)
41-bit timestamp (milliseconds) → 2^41 ms ≈ 69 years
10-bit worker ID → 1024 machines
12-bit sequence → 4096 IDs per millisecond
Pros : Trend increasing, no third-party dependency, high throughput, flexible bit allocation.
Cons : Strong dependency on machine clock; clock rollback causes duplicate IDs or service unavailability.
3.6 Baidu UidGenerator
Java implementation based on Snowflake with improvements:
Customizable workerId bits and initialization for Docker/virtualized environments (auto-restart, drift).
Borrows future time to overcome sequence concurrency limit.
RingBuffer caches generated UIDs, parallelizes production/consumption, CacheLine padding avoids false sharing.
Single-machine QPS up to 6 million.
Bit allocation (differs from standard Snowflake):
sign (1 bit)
delta seconds (28 bits, relative to 2016-05-20, ~8.7 years)
worker id (22 bits, ~4.2M startups, allocated via DB at startup)
sequence (13 bits, 8192 per second)
Reference: GitHub README at https://github.com/baidu/uid-generator/blob/master/README.zh_cn.md
3.7 Meituan Leaf
Two modes: Leaf-segment (DB) and Leaf-snowflake.
Leaf-segment
Proxy server batch-fetches segments (step size), reducing DB pressure. biz_tag isolates different business ID sequences; scaling via biz_tag sharding.
+-------------+--------------+------+-----+-------------------+-----------------------------+</code><code>| Field | Type | Null | Key | Default | Extra |</code><code>+-------------+--------------+------+-----+-------------------+-----------------------------+</code><code>| biz_tag | varchar(128) | NO | PRI | | |</code><code>| max_id | bigint(20) | NO | | 1 | |</code><code>| step | int(11) | NO | | NULL | |</code><code>| desc | varchar(256) | YES | | NULL | |</code><code>| update_time | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |</code><code>+-------------+--------------+------+-----+-------------------+-----------------------------+Leaf-snowflake
Uses Zookeeper persistent sequential nodes for automatic workerID assignment:
Start Leaf-snowflake, connect ZK, check registration under leaf_forever parent node.
If registered, retrieve workerID (sequential node int ID), start service.
If not registered, create persistent sequential node under parent, use sequence number as workerID, start service.
This solves Snowflake's clock dependency. Architecture diagram:
Reference: Meituan Tech Blog at https://tech.meituan.com/2017/04/21/mt-leaf.html
3.8 Didi TinyID
Java distributed ID system based on DB segment algorithm. Extends leaf-segment with multi-DB support and tinyid-client.
Architecture diagram:
Pros : Easy integration, mature implementation.
Cons : Depends on DB stability; requires cluster master-slave for high availability.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
