What Makes ElasticJob the Next‑Generation Distributed Scheduler?

ElasticJob, launched in 2015, is a distributed scheduling solution that combines Quartz's scheduling capabilities with ZooKeeper's coordination, offering both lightweight (ElasticJob Lite) and cloud‑native (ElasticJob Cloud) variants.

Guide: Scheduling in computing allocates resources at the right moment to execute predefined tasks, a critical concern in both single‑machine and distributed environments. It hinges on resource governance and trigger timing.

Background

ElasticJob was created to fill the gap of distributed job scheduling frameworks, complementing existing timers like Quartz by providing a one‑stop automated operations console for distributed jobs.

It adopts proven technologies—Quartz for scheduling semantics and ZooKeeper for distributed coordination—resulting in a stable, scalable distributed scheduling framework.

What Is ElasticJob?

ElasticJob consists of two independent projects: ElasticJob Lite, a lightweight, decentralized solution delivered as a JAR, and ElasticJob Cloud, which builds on a custom Mesos framework to add resource governance, application distribution, and process isolation.

It enables developers to focus on business logic while relieving both developers and operators from concerns about throughput, availability, and management by simply adding service nodes.

ElasticJob Scheduling Model

ElasticJob separates scheduling into process‑internal (thread‑level) scheduling via Lite and process‑level scheduling via Cloud.

Process‑Internal Scheduling

ElasticJob Lite provides thread‑level scheduling within a process, integrates smoothly with Spring, Dubbo, and other Java frameworks, and allows beans to be injected into jobs.

It runs embedded with the business application, sharing its lifecycle, and requires only a registration center (ZooKeeper or ETCD) for coordination.

Architecture:

The distributed job nodes elect a master node for sharding; after sharding, both master and slave nodes execute tasks autonomously.

Process‑Level Scheduling

ElasticJob Cloud adds process‑level scheduling, supporting resident (Lite‑like) and transient jobs. Transient jobs launch a new process for each execution, leveraging Mesos for resource control.

Cloud uses a centralized scheduler managed by Mesos, providing high availability.

Beyond Lite's capabilities, Cloud offers full resource allocation and job distribution, forming a true job‑cloud scheduling system.

ElasticJob Feature List

Key features include elastic scheduling, resource allocation, job governance, and a visual management console.

Elastic Scheduling

ElasticJob enables horizontal scaling via task sharding; each server runs only its assigned shards, and the system dynamically re‑balances shards when servers are added or fail.

When new servers join, ElasticJob detects them via temporary nodes in the registry and redistributes shards accordingly.

If a server crashes, remaining nodes take over its shards, ensuring high availability.

Resource Allocation

ElasticJob leverages Mesos to allocate CPU and memory resources for jobs, isolating them before execution. Future integration with Kubernetes is planned.

Job Governance

Provides high‑availability, failover, and missed‑execution handling for distributed jobs.

Visual Management Console

Offers CRUD operations for jobs, execution history, and configuration management.

Typical Use Cases

ElasticJob addresses complex tasks, resource‑intensive jobs, and business‑application jobs.

Complex Tasks

Massive data migration can be accelerated by elastic sharding.

Resource‑Oriented Tasks

Heavy reporting jobs benefit from transient job types that acquire resources on demand and release them after completion.

Business Applications

Order‑pull jobs can be integrated seamlessly with existing services via Spring injection.

New Version Design Philosophy (3.0.0‑alpha)

Upcoming 3.x release focuses on modularity, plug‑in architecture, and expanded job APIs based on Apache ShardingSphere.

Job Ecosystem

Introduces a flexible, plug‑in job API allowing custom job types (script, HTTP, big‑data, file, etc.) via SPI.

Diverse Schedulers

Adds a one‑time scheduling API alongside the existing cron‑based scheduler.

Micro‑Kernel & Ecosystem Separation

Separates core job execution from auxiliary modules, opening persistence to multiple storage backends.

Future Roadmap

Job Dependencies

Support DAG‑based job dependencies at both job and shard levels.

Scheduler‑Executor Separation

Enable independent deployment of schedulers and executors for both Lite and Cloud modes.

More User‑Friendly Cloud Management

Extend Cloud support to both Mesos and Kubernetes, and provide a standalone job control platform without resource management.

Plug‑in Ecosystem

Build on ShardingSphere’s plug‑in architecture to allow custom job types, registry centers, and persistence layers.

About the ElasticJob Community

The community was revived after a period of inactivity and is now an active Apache ShardingSphere sub‑project, with many contributions on GitHub.

ElasticJob aims to become an independent Apache top‑level project and the scheduling backbone for ShardingSphere’s elastic migration.