Designing a High‑Performance Distributed ID Generator with Vesta

Preface

In a previous article the author described a periodic reconciliation mode that guarantees eventual consistency by passing a unified trace ID between systems. The service that generates unique trace IDs is often called a "ID generator"; Twitter's Snowflake is a popular open‑source implementation, while Slowfake (a Scala version) lacks documentation and maintenance.

To provide Java developers with a ready‑to‑use ID service, the author presents Vesta, a self‑developed multi‑scenario distributed ID generator written in Java. Vesta can be embedded as a JAR, deployed as a central server, or exposed via a REST API, offering simplicity, flexibility, and high efficiency.

1. How to Think and Design

1.1 Current Problems

Many business systems rely on database auto‑increment columns for IDs, which creates difficulties during database migration, scaling, data cleaning, and sharding.

1.2 Why Not Use UUID

UUIDs guarantee uniqueness but lack time‑based rough ordering, reversibility, and manufacturability. They are long, consume more storage, and degrade B‑tree index performance due to random writes.

1.3 Requirement Analysis

Global uniqueness

Rough ordering (second‑level or millisecond‑level)

Reversibility (ID can be decoded to obtain generation time and other info)

Manufacturability (IDs can be regenerated after failures)

High performance (target >10,000 TPS per node)

High availability (peer‑to‑peer cluster with failover)

Scalability (support for growing number of servers)

1.4 Design and Implementation

1.4.1 Publishing Modes

Embedded mode: a native JAR for Java clients, requires pre‑configured machine ID.

Central server mode: a client JAR that calls a central ID service.

REST mode: a RESTful service usable by non‑Java clients.

The publishing mode is encoded in the generated ID.

1.4.2 ID Types

Two types are supported:

Maximum‑peak type: second‑level ordering, 30‑bit timestamp, 20‑bit sequence.

Minimum‑granularity type: millisecond‑level ordering, 40‑bit timestamp, 10‑bit sequence.

1.4.3 Data Structure

Machine ID : 10 bits (supports up to 1024 servers).

Sequence (max‑peak): 20 bits – up to 1,048,576 IDs per second.

Sequence (minimum‑granularity): 10 bits – up to 1,024 IDs per millisecond.

Timestamp (max‑peak): 30 bits – about 34 years of seconds.

Timestamp (minimum‑granularity): 40 bits – about 34 years of milliseconds.

Publish mode : 2 bits (00 embedded, 01 central, 02 REST, 03 reserved).

ID type : 1 bit (0 max‑peak, 1 minimum‑granularity).

Version : 1 bit for future extensions.

1.4.4 Concurrency

For central server and REST modes, ID generation is CPU‑bound; network I/O is the bottleneck. A ReentrantLock protects the time and sequence fields, offering better performance than synchronized.

1.4.5 Machine ID Allocation

Machine IDs 0‑923 are reserved for embedded mode, 924‑1023 for central/REST modes. The ranges can be adjusted dynamically.

1.4.6 Zookeeper Integration

Embedded mode can obtain a machine ID from Zookeeper; if the pool is exhausted, startup fails. A default static machine ID can be used as an alternative.

1.4.7 Time Synchronization

Linux crontab with NTP ensures clock accuracy. Adjusting the clock may temporarily halt ID generation or cause duplicate timestamps if the clock is set backward.

1.4.8 Design Validation

Construct an ID with global uniqueness, reversibility, and manufacturability.

Ensure rough ordering via second‑ or millisecond‑level timestamps.

Use multithreading with ReentrantLock (CAS) to achieve >10,000 TPS per node.

Provide fallback between publishing modes and between Zookeeper and static IDs for high availability.

Support up to 1024 servers with dynamic partitioning for scalability.

2. How to Meet Performance Requirements

The target is >10,000 TPS per server. Tests were performed on a laptop (dual‑core i3, 4 GB RAM) with 100 concurrent threads.

2.3 Embedded Mode Benchmark

2.4 Central Server Mode Benchmark

2.5 REST Mode (Netty) Benchmark

2.6 REST Mode (Spring Boot + Tomcat) Benchmark

2.7 Performance Summary

Netty achieves ~11,000 QPS, while Tomcat reaches ~5,000 QPS. Embedded mode can exceed 400,000 QPS, indicating that network I/O is the primary bottleneck. Central server mode (Dubbo) performed poorly (<2,000 QPS) and requires further optimization.

3. Quick Start Guide

3.1 Installation and Startup

Download the latest REST package (e.g., vesta-rest-netty-0.0.1-bin.tar.gz).

Extract the archive: tar xzvf vesta-rest-netty-0.0.1-bin.tar.gz.

Edit vesta-rest-netty-0.0.1/conf/vesta-rest-netty.properties: vesta.machine=1022<br/>vesta.genMethod=2<br/>vesta.type=0 Machine ID must be unique per host; genMethod=2 selects embedded mode; type=0 selects max‑peak type.

Change the server port in vesta-rest-netty/target/vesta-rest-netty-0.0.1/bin/server.sh (default 8088, e.g., set port=10010).

Make scripts executable: chmod 755 * in the bin directory.

Start the service: ./start.sh.

Successful start prints:

apppath: /home/robert/vesta/vesta-rest-netty-0.0.1<br/>Vesta Rest Netty Server is started.

3.2 Test the REST Service

Generate an ID: curl http://localhost:10010/genid → returns a numeric ID.

Decode the ID: curl http://localhost:10010/expid?id=1138729511026688 → returns JSON with fields such as genMethod, machine, seq, time, etc.

Convert the timestamp: curl http://localhost:10010/transtime?time=12235264 → returns a human‑readable date.

Re‑encode data to fabricate an ID (demonstrated with the same endpoint).

4. Conclusion

The ID generator is a fundamental component of distributed service architectures, ensuring consistency and high availability. Vesta provides a ready‑to‑use, high‑performance solution with multiple deployment options, and its design principles can guide developers building their own ID services.