This article explains how a distributed delayed‑execution component solves business scenarios that require timed actions, ensuring high availability and data consistency by leveraging Redis sorted sets and RabbitMQ dead‑letter queues, while providing a simple Java API and optional DB‑backed reliability.
Aloha is a Scala‑implemented distributed scheduling framework built on Spark that provides extensible plugins, high‑availability master/worker architecture, REST submission, custom application interfaces, event listeners, and a Scala‑based RPC system for managing long‑running tasks such as Spark, Flink, and ETL jobs.
The article details JD.com’s large‑scale big‑data system overhaul before the 11.11 shopping festival, highlighting a multi‑region Hydra Scheduler, intelligent storage policies, full containerization of the streaming platform, enhanced log reporting, and blockchain‑based traceability that together dramatically improve performance, stability, and user experience.
This article explains how to design and implement a high‑throughput, second‑level accurate coupon push notification service using Redis Sorted Sets, a distributed task cluster, dynamic queue configuration, and load‑balancing to handle millions of users within a minute.
This article describes how JD built a multi‑regional, ten‑thousand‑node Hadoop ecosystem, unified resource management with YARN, introduced a three‑level Router scheduling layer, optimized performance, and integrated deep‑learning frameworks to achieve high availability, cost efficiency, and scalable big‑data processing.
This article examines the principles, features, and implementation details of distributed scheduling for big‑data ETL pipelines, covering decentralised schedulers, host selection strategies, fault‑tolerance, operator abstraction, elasticity, trigger mechanisms, visual monitoring, alarm handling, data fan‑in/fan‑out, parameter consistency, real‑time quality checks, lineage tracking, and field‑level traceability.
Juice is an open‑source Mesos‑based distributed task scheduling framework that abstracts Docker tasks, provides REST APIs, leverages Redis queues, and implements custom resource allocation, offering high resource utilization and fault‑tolerant execution for audio‑video processing workloads.
Elastic-Job is a Java-based, decentralized distributed job scheduling framework that addresses limitations of existing solutions by providing features such as distributed coordination via Zookeeper, parallel task execution, elastic scaling, centralized management, customizable workflow tasks, and robust non‑functional requirements, with future plans for multi‑language support and enhanced monitoring.
This article introduces Elastic-Job, a Java‑based distributed job scheduling framework from Dangdang, covering its origins, core features such as sharding and elastic scaling, deployment with Zookeeper, best‑practice usage, open‑source development philosophy, future improvements, and detailed Q&A.
This article summarizes a China Mobile cloud computing talk covering the company’s vision for private and public clouds, core technologies such as lightweight virtualization, distributed scheduling and coordination, the evolution from “small cloud” to “big cloud”, practical reflections on implementation, and strategic recommendations for future development.
