How to Implement Distributed ID Generation with Segment Mode? A Double‑Buffer Issuer in Practice

When each request generates an ID by updating a single row, the row‑level lock on biz_tag becomes a bottleneck. The author proposes a segment (range) mode where the database allocates a batch of IDs in one UPDATE id_alloc SET max_id = max_id + step WHERE biz_tag = ? statement and the application serves IDs from this range in memory.

1. Batch Allocation and Interval Definition

The step size (e.g., 1000) determines how many IDs are fetched per batch. The range is expressed as [startInclusive, endExclusive) (left‑closed, right‑open) to avoid off‑by‑one errors. The length of a range is simply endExclusive - startInclusive. A MySQL table id_alloc_tutorial stores biz_tag, max_id (the right boundary), and step with a primary key on biz_tag and a check constraint ensuring step > 0.

CREATE TABLE id_alloc_tutorial (
  biz_tag   VARCHAR(64) NOT NULL PRIMARY KEY,
  max_id   BIGINT      NOT NULL,
  step     INT         NOT NULL CHECK (step > 0),
  update_time TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);

2. Java Range Validation

A

record SegmentRange(long startInclusive, long endExclusive, int step)

validates that step > 0, startInclusive >= 0, endExclusive > startInclusive, and that the interval length matches step. Invalid arguments raise IllegalArgumentException.

3. JDBC Segment Allocator

The JdbcSegmentAllocator implements SegmentAllocator. Its allocate(String bizTag) method runs the UPDATE … SET max_id = max_id + step inside a @Transactional block, then reads the new max_id and step with SELECT … FOR UPDATE to keep the write lock until the read completes. This guarantees that each batch is allocated atomically and that no other instance can interleave.

4. Single‑Buffer Issuer

A LocalSegment holds the current range and an AtomicLong cursor. tryNext() atomically increments the cursor; if the value is still < endExclusive it returns the ID, otherwise it returns OptionalLong.empty() signalling exhaustion. When the buffer is exhausted the caller must fetch a new segment, which introduces a latency spike.

5. Double‑Buffer Design

To hide the latency, each bizTag owns a SegmentBuffer containing volatile LocalSegment current and volatile LocalSegment next. A ReentrantLock switchLock protects state changes, while an AtomicBoolean loading guarantees that only one preload task runs per buffer. When current.shouldPreload(ratio) becomes true, a background task loads the next range via the allocator and stores it in next. If the preload succeeds, the fast path swaps current = next; if it fails, the loading flag is cleared and waiting threads are notified via a Condition loadFinished.

6. Failure and Back‑Pressure Handling

If the thread‑pool rejects a preload task, the code resets loading, signals all waiters, and logs a warning. When the background load throws an exception, the same cleanup occurs and the caller falls back to a synchronous allocate call, ensuring the system never stays stuck waiting for a nonexistent segment.

7. Testing Strategy

Unit tests verify the left‑closed/right‑open semantics ( LocalSegmentTest), concurrency safety with a FakeSegmentAllocator that mimics independent watermarks per bizTag, and that 100 000 parallel requests produce exactly 100 000 unique IDs. Preload behavior is tested by forcing the allocator to allocate a second segment after the threshold is reached and asserting that IDs 10 and 11 are emitted without a third allocation.

8. Parameter Estimation and Configuration

The step size is estimated as peakQPS × desiredIntervalSec × safetyFactor. For example, 8000 QPS, a 5 s interval, and a safety factor of 1.5 yield step ≈ 60000. The preload threshold must cover the time needed to fetch a segment: thresholdTime ≈ (step × preloadRatio) / peakQPS. Configuration values (preload ratio, thread count, queue capacity, datasource timeouts) are shown in a YAML snippet.

9. Decision Guidance

The author recommends segment allocation when a numeric, monotonically increasing ID is required and the write‑through rate is high. For low traffic, a simple auto‑increment column is cheaper; for strictly consecutive numbers, a transactional sequence is needed; and when the database may be unavailable, Snowflake‑style or TSID solutions are preferable.

Overall, the double‑buffer issuer reduces lock contention, smooths latency spikes, and provides a clear testing and configuration framework for reliable distributed ID generation.