Apache DolphinScheduler 3.0.0 Release Highlights and New Features

On August 10, 2022, after a series of alpha and beta iterations, Apache DolphinScheduler officially released its third major version, 3.0.0, which brings the most extensive changes since the project's inception.

The release is summarized by four keywords: faster, stronger, more modern, and easier to maintain.

Faster: The UI has been rebuilt, delivering tens‑fold faster user response and hundreds‑fold faster developer build times.

Stronger: New capabilities such as data‑quality assurance, custom time zones, additional task types, and multiple alert plugins have been added.

More Modern: Both page layout and icon styles have been refreshed for a contemporary look.

Easier to Maintain: Backend services have been split to align with container‑native and micro‑service architectures, clarifying responsibilities and simplifying upkeep.

New Features and Enhancements

All‑new UI

AWS support

Service splitting

All services can be started or stopped using the bin/dolphinscheduler-daemon.sh script.

Data Quality Assurance

Since version 2.0.0 users have awaited native data‑quality features. Version 3.0.0 finally provides built‑in data‑quality monitoring, allowing users to configure thresholds for record counts, weekly/monthly fluctuations, and receive alerts when data quality deviates.

This ensures that workflow execution results are accurate without requiring additional development effort.

Support for Flink Task Type

The release extends DolphinScheduler to support Flink SQL tasks, submitted via sql-client.sh. Previously only Flink CLI submissions were possible, which required manual resource handling. The new approach enables users to write SQL directly in the task definition, improving transparency and ease of use.

Implementation classes include:

Core source code for the Flink task is shown below:

public class FlinkTask extends AbstractYarnTask {
    /**
     * flink parameters
     */
    private FlinkParameters flinkParameters;

    /**
     * taskExecutionContext
     */
    private TaskExecutionContext taskExecutionContext;

    public FlinkTask(TaskExecutionContext taskExecutionContext) {
        super(taskExecutionContext);
        this.taskExecutionContext = taskExecutionContext;
    }

    @Override
    public void init() {
        logger.info("flink task params {}", taskExecutionContext.getTaskParams());
        flinkParameters = JSONUtils.parseObject(taskExecutionContext.getTaskParams(), FlinkParameters.class);
        if (flinkParameters == null || !flinkParameters.checkParameters()) {
            throw new RuntimeException("flink task params is not valid");
        }
        flinkParameters.setQueue(taskExecutionContext.getQueue());
        if (ProgramType.SQL != flinkParameters.getProgramType()) {
            setMainJarName();
        }
    }
    // ... (remaining methods omitted for brevity)
}

Release Note

Source code and release assets are available on GitHub at https://github.com/apache/dolphinscheduler/releases/tag/3.0.0 .

Download the binaries from the official Apache site: https://dolphinscheduler.apache.org/en-us/download/download.html .

For readers who find this article helpful, remember to "watch", "like", and "bookmark" the post.