Distributed ID Generation: Requirements, Schemes, and Implementations

In monolithic systems primary keys are often generated by auto‑incrementing database columns, but this approach fails for distributed systems where data is sharded across multiple databases and machines. Duplicate IDs can appear, breaking business logic.

Business requirements for a distributed ID :

Global uniqueness : each ID must be unique across the entire system.

Trend‑increasing : ordered IDs improve write performance for InnoDB clustered indexes.

Monotonic increase : IDs should be larger than previous ones to support versioning and sorting.

Security : IDs should not be strictly sequential to avoid easy data scraping.

Common ID generation schemes :

UUID : 128‑bit universally unique identifier. Advantages – locally generated, no network dependency; disadvantages – large size, string representation, potential MAC address leakage, and poor insertion locality.

Database auto‑increment : each machine uses a distinct auto‑increment offset and step (e.g., step = number of machines). Works but requires manual reconfiguration when scaling and still puts load on the DB.

Segment mode : a range (segment) of IDs is fetched from a central DB and cached in memory. When the segment is exhausted, a new range is requested. CREATE TABLE id_generator ( id int(10) NOT NULL, max_id bigint(20) NOT NULL COMMENT 'current max id', step int(20) NOT NULL COMMENT 'segment length', biz_type int(20) NOT NULL COMMENT 'business type', version int(20) NOT NULL COMMENT 'optimistic lock version', PRIMARY KEY (`id`) ); UPDATE id_generator SET max_id = #{max_id+step}, version = version + 1 WHERE version = #{version} AND biz_type = #{biz_type}; Pros: mature solution, used by Baidu UidGenerator, Meituan Leaf. Cons: depends on the database.

Redis : uses atomic INCR/INCRBY commands. Guarantees uniqueness and ordering thanks to Redis' single‑threaded nature. Pros: high performance, simple. Cons: requires a Redis cluster for high concurrency.

Snowflake (Twitter) : 64‑bit ID composed of sign bit, timestamp (41 bits), machine ID (10 bits), and sequence (12 bits). Implemented in Java as a long . Pros: trend‑increasing, no external service, very fast. Cons: strong dependency on accurate system clock; clock rollback can cause duplicates.

Baidu UidGenerator : an enhanced Snowflake implementation with configurable worker‑id bits and optimistic‑lock versioning. Claims up to 6 M QPS per node.

Meituan Leaf : provides two modes – segment and Snowflake. Segment mode batches DB reads; Snowflake mode uses Zookeeper to allocate worker IDs automatically. +-------------+--------------+------+-----+-------------------+-----------------------------+ | Field | Type | Null | Key | Default | Extra | +-------------+--------------+------+-----+-------------------+-----------------------------+ | biz_tag | varchar(128) | NO | PRI | | | | max_id | bigint(20) | NO | | 1 | | | step | int(11) | NO | | NULL | | | desc | varchar(256) | YES | | NULL | | | update_time| timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP | +-------------+--------------+------+-----+-------------------+-----------------------------+

Didi TinyID : Java‑based ID service built on the segment algorithm, supporting multiple DBs and a client library.

The article concludes with a call for discussion, provides QR‑code links for interview questions, and includes promotional material for the author’s WeChat group.