Deep Dive into XXL-JOB: Time Wheel Scheduling, Consistent Hash Routing, and Sharding Implementation

Introduction: The author, a senior architect, introduces XXL‑JOB, a lightweight distributed task scheduling platform designed for rapid development, simplicity, and extensibility.

XXL‑JOB Overview: It consists of a scheduling center, executors, and a data center. The scheduler manages tasks, logs, and executor registration.

Time‑Wheel Scheduling: The article explains how XXL‑JOB implements a “class‑time‑wheel” similar to Netty’s HashedWheelTimer, using buckets and a rotating pointer to trigger tasks. An example with a 1‑second bucket and an 8‑second wheel is illustrated.

private volatile static Map
> ringData = new ConcurrentHashMap<>();
int ringSecond = (int)((jobInfo.getTriggerNextTime()/1000)%60);
private void pushTimeRing(int ringSecond, int jobId) {
    List
ringItemData = ringData.get(ringSecond);
    if (ringItemData == null) {
        ringItemData = new ArrayList
();
        ringData.put(ringSecond, ringItemData);
    }
    ringItemData.add(jobId);
}

Task Execution: Two threads, scheduleThread and ringThread , read upcoming tasks and execute due tasks from the current bucket.

List
ringItemData = new ArrayList<>();
int nowSecond = Calendar.getInstance().get(Calendar.SECOND);
for (int i = 0; i < 2; i++) {
    List
tmpData = ringData.remove((nowSecond+60-i)%60);
    if (tmpData != null) {
        ringItemData.addAll(tmpData);
    }
}
for (int jobId : ringItemData) {
    JobTriggerPoolHelper.trigger(jobId, TriggerTypeEnum.CRON, -1, null, null);
}

Consistent‑Hash Routing: The article describes the consistent‑hash strategy used for executor selection, the need for virtual nodes, and presents the MD5‑based hash function.

byte[] digest = md5.digest();
long hashCode = ((long)(digest[3] & 0xFF) << 24)
              | ((long)(digest[2] & 0xFF) << 16)
              | ((long)(digest[1] & 0xFF) << 8)
              | (digest[0] & 0xFF);
long truncateHashCode = hashCode & 0xffffffffL;

Sharding Implementation: XXL‑JOB’s sharding broadcast strategy distributes a job across multiple executors. The ShardingUtil class stores sharding parameters in an inheritable thread‑local, and ShardingJobHandler retrieves them to process only the assigned slice.

public class ShardingUtil {
    private static InheritableThreadLocal
contextHolder = new InheritableThreadLocal<>();
    public static class ShardingVO {
        private int index;
        private int total;
        // getters and setters omitted
    }
    public static void setShardingVo(ShardingVO shardingVo) {
        contextHolder.set(shardingVo);
    }
    public static ShardingVO getShardingVo() {
        return contextHolder.get();
    }
}

Observations: The author reflects on XXL‑JOB’s design goals, the custom RPC layer, the advantages of the time‑wheel and consistent‑hash approaches, and notes areas for further exploration such as fault tolerance and logging.