How DBLE Maps Logical Shards to Physical Partitions Using a Hash Modulo

Algorithm Overview

DBLE determines the target physical shard in two steps: it first applies a modulo operation on the shard index to obtain a logical shard number, then uses a pre‑computed mapping table to translate that logical shard into a physical shard.

Configuration Steps

Define two arrays partitionLength[] and partitionCount[] in rule.xml.

During DBLE startup, compute the dot product of the two arrays; the result is the modulus and also the total number of logical shards.

Generate a mapping table by cross‑multiplying the arrays; the total number of physical shards equals the sum of all elements in partitionCount[].

At runtime, DBLE extracts the shard index from the WHERE clause, applies the modulo to get the logical shard number.

Lookup the mapping table to obtain the corresponding physical shard number.

Comparison with MyCat

When partitionLength is set to 1 and partitionCount to N , DBLE behaves like MyCat’s simple N‑modulo sharding algorithm. DBLE, however, allows the dot product of the two arrays to be any value in the range [1, 2880], providing more flexible shard distributions.

Development Considerations

Shard index must be an integer or an integer‑formatted string (negative values are allowed).

The sum of partitionCount[] must not exceed 2880.

Configuration examples:

<property name="partitionLength">1</property>

and

<property name="partitionCount">2880</property>

The order of elements in the two arrays matters; swapping values changes the mapping.

Data distribution depends on the continuity of the shard key. If the shard key is sequential (e.g., transaction IDs), distribution tends to be even, but range queries may span multiple shards.

Operational Guidelines

Scaling up : Pre‑allocate extra logical shards without changing the dot product of the two arrays to avoid data rebalancing; only migrate the affected data.

Scaling down : Use the same pre‑allocation strategy; if the dot product must change, a full data rebalancing is required.

Both expansion and contraction require careful migration of data between physical shards.

Configuration Details

In rule.xml the two configurable properties are:

<property name="partitionLength">512,256</property>

<property name="partitionCount">1,2</property>

This configuration means that one physical shard holds 512 logical shards and the next physical shard holds 256 logical shards. The mapping is order‑sensitive, and the dot product of the two arrays must stay within [1, 2880]. When all elements of partitionLength[] are 1, the algorithm reduces to a simple modulo operation, making logical and physical shards one‑to‑one.