Unveiling the Core Architecture of Xxl-Job: A Lightweight Distributed Scheduler

Core Concepts

The Xxl‑Job platform consists of three main entities:

Scheduler (admin center) : a standalone web service that stores task definitions in a shared database and triggers task execution via HTTP.

Executor : a service instance that runs the actual job logic; each service instance maps to one executor instance and is identified by an appname.

Task : the unit of work defined in the scheduler and executed by an executor.

The design separates trigger logic from execution, making tasks flexible and callable by different scheduling rules.

Demo Setup

1. Clone the source code from https://github.com/xuxueli/xxl-job.git and configure the database connection.

2. Run the SQL scripts under /doc/db to create the required tables.

3. Start the admin console (default URL http://localhost:8080/xxl-job-admin/toLogin, credentials admin/123456).

4. Add an executor named sanyou-xxljob-demo and a task named TestJob with a cron expression that fires every second.

5. Implement the task using the @XxlJob("TestJob") annotation; the executor will log "TestJob任务执行了。。。" each second.

Executor Startup – XxlJobSpringExecutor

The class XxlJobSpringExecutor implements SmartInitializingSingleton, so its afterSingletonsInstantiated() method runs during bean initialization. It performs three key actions:

Initialize JobHandler : scans the Spring context for beans annotated with @XxlJob, wraps each method into a MethodJobHandler, and stores them in a local Map keyed by task name.

Create an HTTP server : launches a Netty‑based server on the configured port (e.g., 9999) to receive scheduler requests.

Register to the scheduler : starts a registration thread that posts the executor’s appname, IP and port to the admin center, allowing the scheduler to locate the executor.

JobHandler Types

MethodJobHandler – invokes a Java method via reflection.

GlueJobHandler – allows dynamic Java code editing through the GLUE IDE without restarting the service.

ScriptJobHandler – executes scripts (e.g., Groovy, Shell) in BEAN or GLUE mode.

Task Trigger Mechanism

The scheduler runs a dedicated schedule thread that queries the xxl_job_info table for tasks whose next fire time is within the next 5 seconds (the hard‑coded pre‑read window). It classifies tasks into three buckets:

Already overdue by more than 5 s – handled according to the task’s expiration strategy (ignore or fire immediately).

Overdue by less than 5 s – fired immediately and then placed into a time wheel for the next cycle.

Not yet due but within 5 s – placed into the time wheel awaiting the exact trigger time.

Before processing, the scheduler acquires a distributed lock using the SQL statement:

select * from xxl_job_lock where lock_name = 'schedule_lock' for update

Only the instance that obtains the lock proceeds, ensuring a single scheduler in a cluster performs the trigger calculation.

Fast/Slow Thread‑Pool Optimization

Triggering a task involves an HTTP call to the executor, which can be time‑consuming. To avoid blocking the schedule thread, Xxl‑Job hands the trigger to an asynchronous thread pool. It maintains two pools:

Fast pool – handles tasks whose HTTP latency is below 500 ms.

Slow pool – receives tasks that have exceeded 500 ms latency more than 10 times within a minute, preventing long‑running triggers from starving other tasks.

Executor Selection (Routing Strategies)

When multiple executor instances exist, the scheduler chooses one based on the configured routing strategy. Supported strategies include:

First, Last, Round‑Robin, Random – simple deterministic algorithms.

Consistent Hash – distributes tasks based on a hash of the task key.

Least Frequently Used (LFU) – selects the instance with the fewest recent executions (cache resets every 24 h).

Least Recently Used (LRU) – uses a LinkedHashMap to pick the least recently accessed instance.

Failover – picks the first responsive executor.

Busy‑Transfer – avoids executors currently processing the same task.

Sharding Broadcast – assigns a shard index to each executor and broadcasts the task to all, each executor handling its own slice of data.

Task Execution Flow on the Executor

Upon receiving a trigger request, ExecutorBizImpl.run() creates a dedicated JobThread for the task. The thread places the job into an in‑memory queue, from which a worker thread fetches and runs the job. This design isolates tasks and enables queuing.

If the selected executor is already busy with the same task, the scheduler applies a blocking strategy defined per task:

Single‑machine serial – queue the new trigger, preserving FIFO order.

Discard later triggers – ignore the new request.

Cover previous trigger – interrupt the existing JobThread, discard its queued work, and start a fresh thread.

Note that blocking strategies affect only the specific executor instance; other instances running the same logical task are unaffected.

Result Callback

After a job finishes, its JobThread pushes the result into an internal queue. A dedicated TriggerCallbackThread continuously polls this queue and batches results back to the scheduler. The scheduler then updates the task’s execution status and handles retries or downstream sub‑tasks.

Architecture Overview

The official diagram (though slightly outdated) shows the admin center, executors, HTTP communication, and a now‑replaced custom RPC layer that has been swapped for plain HTTP to improve cross‑language compatibility.

Overall, Xxl‑Job provides a lightweight yet extensible framework for distributed task scheduling, with clear separation of concerns, pluggable routing, and robust handling of concurrency and failure scenarios.