Choosing the Right Distributed Scheduler: Elastic‑Job vs X‑Job vs Quartz

1. Introduction

We first consider several business scenarios that require a task at a specific moment:

Payment system runs a batch at 1 am daily for daily settlement and on the 1st of each month for monthly settlement.

E‑commerce flash‑sale price discount starts at 8:00.

12306 ticketing system recovers orders not paid within 30 minutes.

After a product is shipped, an SMS reminder is sent to the customer.

How do we solve many similar scenarios?

Many business scenarios need a task at a particular moment; scheduled tasks address this. Generally, a system can replace some scheduled tasks with message passing, and the two have many similarities and can be interchangeable.

For example, after successful shipment we can send an MQ message and consume it to send an SMS.

However, they cannot be swapped in some cases:

Time‑driven vs event‑driven: internal systems can be time‑driven, but when external systems are involved only time‑driven works, e.g., crawling external website prices hourly.

Batch vs single processing: batch processing of accumulated data is more efficient when real‑time is not required, and some business logic can only be batch, e.g., monthly mobile‑fee settlement.

Real‑time vs non‑real‑time: message middleware can process data in real time, but some cases do not need real‑time, e.g., VIP upgrade.

System internal vs system decoupling: scheduled‑task scheduler is usually internal, while message middleware can be used between systems.

2. Scheduled‑Task Frameworks

Standalone

Timer : a timer class; TimerTask implements Runnable; drawback: unchecked exceptions stop the thread.

ScheduledExecutorService : schedules with relative delay or period; drawback: cannot specify absolute date/time.

Spring scheduling : simple configuration, many features; suitable when the system is single‑node.

Distributed

Quartz : Java’s de‑facto standard for scheduled tasks; focuses on timing, not data; can achieve high availability via DB but lacks distributed parallel scheduling.

TBSchedule : early Alibaba open‑source distributed scheduler; uses Timer instead of thread pool; code is outdated, limited job types, documentation missing.

elastic‑job : Dangdang’s elastic distributed scheduler; rich features, uses Zookeeper for coordination, supports cloud deployment (v2.15).

Saturn : Vipshop’s self‑developed distributed scheduler, based on elastic‑job v1, deployable to Docker.

xxl‑job : Dazhong Dianping’s lightweight distributed scheduler; aims for rapid development, easy learning, lightweight, extensible.

3. Comparison of Distributed Schedulers

We compare elastic‑job (E‑Job) and xxl‑job (X‑Job).

Project background and community

X‑Job: developed by three contributors from Dazhong Dianping; 2470 stars, 1015 forks; six QQ groups; over 40 companies use it; documentation complete.

E‑Job: open‑source from Dangdang; 17 contributors; 2524 stars, 1015 forks; one QQ group and one source‑code group; over 50 companies use it; documentation complete; clear development roadmap.

Cluster deployment support

X‑Job: cluster deployment requires identical configuration on each node; scheduler distinguishes clusters via DB config.

Executor cluster deployment improves availability and processing capacity; all executors must share the same admin address configuration.

E‑Job: rewrites Quartz’s DB‑based distributed feature, uses Zookeeper as registry; supports high availability via multiple scheduler instances with leader election.

Task duplication in multi‑node deployment

X‑Job: uses Quartz’s DB‑based distributed feature.

E‑Job: splits a job into N items; each server executes its assigned shards; when a new server joins or leaves, elastic‑job re‑shards before the next run.

Log traceability

X‑Job: provides a log query UI.

E‑Job: records important events via event subscription, stored in relational DB.

Monitoring and alerts

X‑Job: triggers failure alerts (e.g., email) on scheduling failure.

E‑Job: allows custom alert implementation via event subscription.

Job run‑status monitoring, executor liveness, recent success/failure metrics, and threshold‑based alerts are supported.

Elastic scaling

X‑Job: uses Quartz’s DB‑based distributed feature; many servers increase DB load.

E‑Job: uses Zookeeper for registration, control, and coordination.

Parallel scheduling

X‑Job: multi‑threaded (default 10 threads) to ensure precise execution.

E‑Job: uses task sharding; each shard runs on a different server in parallel.

High‑availability strategy

X‑Job: DB lock ensures only one scheduler triggers a task in a cluster.

E‑Job: multiple scheduler instances point to the same Zookeeper cluster; leader election provides failover.

Failure handling

X‑Job: failure alarm (default) and retry.

E‑Job: re‑sharding before next run; during execution, failed servers’ shards can be taken over by idle servers (failover), at the cost of performance.

Dynamic sharding

X‑Job: broadcast sharding per executor; supports dynamic executor scaling.

E‑Job: supports multiple sharding strategies, including average allocation, hash‑based IP ordering, and round‑robin; custom strategies possible. Sharding is driven by Zookeeper and triggers on node join/leave or leader election.

4. Comparison with Quartz

API‑driven task manipulation is not user‑friendly.

Persisting QuartzJobBean into the database is highly invasive.

Scheduling logic coupled with business logic limits performance as task count grows.

Quartz focuses on timing, not data; lacks distributed parallel scheduling.

5. Overall Comparison

6. Summary and Conclusion

Common points : Both E‑Job and X‑Job have large user bases, complete documentation, and satisfy basic scheduled‑task requirements.

Differences :

X‑Job emphasizes simplicity and easy management; low learning curve; rich failure and routing strategies; recommended for scenarios with relatively few users and a limited number of servers.

E‑Job focuses on data‑driven design, elastic scaling, and sharding to fully utilize distributed resources; higher learning cost; recommended for large data volumes and many servers.

That’s all for today; hope it helps!