DolphinScheduler

DataFunTalk

Apr 4, 2023 · Big Data

Compass: An Open‑Source Big Data Task Diagnosis Platform for DolphinScheduler, Airflow and Spark

Compass is an open‑source big‑data diagnostic platform developed by OPPO that provides non‑intrusive, real‑time monitoring and root‑cause analysis for offline and streaming tasks on schedulers such as DolphinScheduler and Airflow, covering workflow‑level failures, Spark engine anomalies, resource usage, and offering one‑click reports and extensible rule‑based diagnostics.

DolphinSchedulerSparkTask Diagnosis

0 likes · 13 min read

Compass: An Open‑Source Big Data Task Diagnosis Platform for DolphinScheduler, Airflow and Spark

DataFunSummit

Mar 6, 2023 · Big Data

Building a Unified Scheduling Center with Apache DolphinScheduler: Lenovo’s Practice

This article details Lenovo’s implementation of a unified scheduling center using Apache DolphinScheduler, covering background requirements, reasons for choosing the platform, architectural evolution, feature enhancements, and practical deployments such as HTTP task parameter passing, Java task plugins, global parameters, and future roadmap.

BigDataDolphinSchedulerLenovo

0 likes · 19 min read

Building a Unified Scheduling Center with Apache DolphinScheduler: Lenovo’s Practice

Big Data Technology Architecture

Sep 18, 2022 · Backend Development

Design and Source Code Analysis of Apache DolphinScheduler

This article provides an in‑depth technical overview of Apache DolphinScheduler, covering its distributed design strategies, fault‑tolerance mechanisms, remote log access, source‑code module breakdown, API interfaces, Quartz integration, master‑worker execution flows, RPC communication, load‑balancing algorithms, logging services, and community contribution guidelines.

DolphinSchedulerQuartzRPC

0 likes · 47 min read

Design and Source Code Analysis of Apache DolphinScheduler

DataFunTalk

May 31, 2022 · Artificial Intelligence

Using DolphinScheduler OpenMLDB Task for End‑to‑End MLOps Workflow

This article introduces the DolphinScheduler OpenMLDB Task, explains how it integrates OpenMLDB's feature platform into DolphinScheduler workflows to create a complete MLOps pipeline, and provides a step‑by‑step demonstration using the TalkingData ad‑fraud detection dataset from Kaggle.

DolphinSchedulerOpenMLDBfeature engineering

0 likes · 7 min read

Using DolphinScheduler OpenMLDB Task for End‑to‑End MLOps Workflow

DataFunTalk

Mar 3, 2022 · Big Data

Youzan Data Platform and DP Data Development Platform: Architecture, Core Modules, and Scheduling System Upgrade

This article presents an in‑depth overview of Youzan's data platform, introduces the DP data development platform with its key features and workflow, details the core module architecture—including service, scheduling, and component layers—and explains the migration from Airflow to DolphinScheduler to improve performance, stability, and scalability.

Data DevelopmentDolphinSchedulerScheduling

0 likes · 14 min read

Youzan Data Platform and DP Data Development Platform: Architecture, Core Modules, and Scheduling System Upgrade

政采云技术

Feb 24, 2022 · Backend Development

Implementation of Workflow Versioning in DolphinScheduler

This article explains how DolphinScheduler (DS) implements workflow versioning by introducing integer‑based versions, describes the core principles of workflow and task relationships, outlines the key database tables and architecture, and provides detailed Java code for creating, updating, and managing workflow definitions, tasks, and their relations.

BackendDatabaseDolphinScheduler

0 likes · 13 min read

Implementation of Workflow Versioning in DolphinScheduler