PowerJob: A Next‑Generation Distributed Task Scheduling and Computing Framework – Introduction and Quick‑Start Guide

Hello everyone, I am Chen~

PowerJob is a new‑generation distributed task scheduling and computing framework that supports CRON, API, fixed‑frequency, and fixed‑delay scheduling strategies, provides workflows to orchestrate task dependencies, and enables easy job scheduling and complex distributed computations.

Why Choose PowerJob?

Popular job scheduling frameworks on the market include the legacy Quartz, the Quartz‑based elastic‑job, and the formerly Quartz‑based xxl‑job. Below are some shortcomings of these frameworks.

Quartz, as the first‑generation scheduling framework, can be considered the “ancestor” of all existing distributed schedulers. Due to historical reasons, it does not provide a web UI and can only be configured via APIs, making it less convenient and flexible; moreover, it only supports single‑node execution and cannot fully utilize a cluster’s computing power.

xxl‑job, regarded as the second‑generation scheduler, addressed many of Quartz’s limitations and has been an excellent framework in recent years, but it still has several drawbacks:

Database support is single: only MySQL is supported; using other databases requires custom code modifications.

Limited distributed computing capability: only static sharding is supported, making complex task computation difficult.

No workflow support: task dependencies cannot be configured, unsuitable for DAG scenarios.

In today’s era of growing data volumes and increasingly complex business logic, a more powerful scheduling framework is needed, and PowerJob was created to meet this demand.

PowerJob can be considered the third‑generation scheduling framework, adding distributed computing and workflow features on top of basic scheduling. Its main characteristics are:

Ease of use: provides a front‑end web UI for visual management of tasks (create, read, update, delete), monitoring of task status, and log viewing.

Comprehensive timing strategies: supports CRON expressions, fixed frequency, fixed delay, and API‑based scheduling.

Rich execution modes: supports standalone, broadcast, Map, and MapReduce modes; the Map/MapReduce processors enable cluster‑wide distributed computation with minimal code.

DAG workflow support: allows online configuration of task dependencies, visual composition, and data passing between upstream and downstream tasks.

Broad processor support: supports Spring Bean, built‑in/external Java classes, Shell, Python, etc.

Operations convenience: online log feature displays executor logs in real time on the web console, reducing debugging cost and improving development efficiency.

Minimal dependencies: only requires a relational database (MySQL/PostgreSQL/Oracle/MS SQLServer…) and supports all databases compatible with Spring Data JPA.

High availability & performance: the scheduler server is lock‑free, allowing multiple servers to be deployed for horizontal scaling and high availability.

Failover & recovery: failed tasks can be retried according to configured retry policies as long as the executor cluster has sufficient nodes.

Product Comparison

Applicable Scenarios

Business scenarios requiring timed execution, such as nightly full data sync or report generation; scenarios needing all machines to execute simultaneously, like clearing cluster logs via broadcast mode.

Scenarios requiring distributed processing, for example updating a large dataset where single‑node execution is too slow; Map/MapReduce processors can distribute the workload across the cluster.

Overall Architecture

Quick Start

PowerJob consists of a scheduler server (powerjob‑server) and an executor (powerjob‑worker). The server provides web services and schedules tasks, while the worker executes user‑written task code and offers distributed computing capabilities.

Project Initialization

Spring Boot basics are omitted.

git clone https://github.com/KFCFans/PowerJob.git

Import the project into an IDE; the source structure is shown below. Start the scheduler server (powerjob‑server) and write your processor code in the samples module.

Start Scheduler Server

Create the database powerjob-daily .

Modify the configuration file (the official documentation provides detailed explanations). Update the following properties:

spring.datasource.core.jdbc-url , spring.datasource.core.username , and spring.datasource.core.password . If you use MongoDB, also update spring.data.mongodb.uri for a full‑featured experience.

oms.env=DAILY
logging.config=classpath:logback-dev.xml

####### Database Configuration #######
spring.datasource.core.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.core.jdbc-url=jdbc:mysql://remotehost:3306/powerjob-daily?useUnicode=true&characterEncoding=UTF-8
spring.datasource.core.username=root
spring.datasource.core.password=No1Bug2Please3!
spring.datasource.core.hikari.maximum-pool-size=20
spring.datasource.core.hikari.minimum-idle=5

####### MongoDB Configuration (optional) #######
spring.data.mongodb.uri=mongodb://remotehost:27017/powerjob-daily

####### Mail Configuration (if email alerts are needed) #######
spring.mail.host=smtp.163.com
spring.mail.username=zqq
spring.mail.password=qqz
spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.starttls.enable=true
spring.mail.properties.mail.smtp.starttls.required=true

####### Resource Cleanup Configuration #######
oms.log.retention.local=1
oms.log.retention.remote=1
oms.container.retention.local=1
oms.container.retention.remote=-1
oms.instanceinfo.retention=1

####### Cache Configuration #######
oms.instance.metadata.cache.size=1024

After editing the configuration, start the server via the main class com.github.kfcfans.powerjob.server.OhMyApplication . Check the startup logs to confirm success, then open http://127.0.0.1:7700/ . If the web UI appears, the scheduler is running.

Register an application by clicking the “Register Application” button on the homepage, entering oms-test and a console password.

Write Sample Code

Enter the sample project ( powerjob-worker-samples ) and modify the configuration to connect to the scheduler, then implement your processor.

Update the startup configuration class com.github.kfcfans.powerjob.samples.OhMySchedulerConfig , setting the AppName to the name you registered.

@Configuration
public class OhMySchedulerConfig {
    @Bean
    public OhMyWorker initOMS() throws Exception {
        // Server HTTP address (port is server.port, not ActorSystem port)
        List
serverAddress = Lists.newArrayList("127.0.0.1:7700");

        // 1. Create configuration
        OhMyConfig config = new OhMyConfig();
        config.setPort(27777);
        config.setAppName("oms-test");
        config.setServerAddress(serverAddress);
        // If you have no heavy Map/MapReduce needs, use in‑memory store for speed
        config.setStoreStrategy(StoreStrategy.MEMORY);

        // 2. Create Worker and set config
        OhMyWorker ohMyWorker = new OhMyWorker();
        ohMyWorker.setConfig(config);
        return ohMyWorker;
    }
}

Implement a processor by creating a new class that extends the desired processor type. Below is a simple example using the standalone BasicProcessor :

@Slf4j
@Component
public class StandaloneProcessorDemo implements BasicProcessor {

    @Override
    public ProcessResult process(TaskContext context) throws Exception {
        OmsLogger omsLogger = context.getOmsLogger();
        omsLogger.info("StandaloneProcessorDemo start process, context is {}.", context);
        System.out.println("jobParams is " + context.getJobParams());
        return new ProcessResult(true, "process successfully~");
    }
}

Run the sample application by executing the main class com.github.kfcfans.powerjob.samples.SampleApplication and verify the console output.

Task Configuration and Execution

After both the scheduler server and sample project are running, return to the web UI at http://127.0.0.1:7700/ to configure and run tasks.

Enter the registered application name on the homepage to access the management console.

Click “Task Management → New Task” (top‑right) to create a task.

After creating the task, you can see it in the console. To run it immediately, click the “Run” button.

View the task’s runtime status and online logs in the sidebar.

This basic tutorial concludes here; additional features such as advanced workflows, MapReduce, and containerized tasks are documented in the official guide.

Open‑source repository: https://github.com/PowerJob/PowerJob

Final Note (Please Support)

If this article helped or inspired you, please like, view, share, and bookmark—it fuels my continued effort!

My knowledge community is also open; reply with the keyword “Knowledge Planet” to receive a ¥30 discount coupon (full price ¥99). The community offers premium series on Spring, massive data sharding, DDD micro‑services, source‑code deep dives, and more, with each additional series increasing the price by ¥20.

Follow the public account “Code Monkey Technical Column” for fan benefits; reply “join group” to get into the tech discussion group.