BestHub
Sign in
Tagged articles

distributed scheduling

108 articles · Page 2 of 2
JD Tech
JD Tech
Nov 29, 2018 · Big Data

JD.com’s Big Data System Upgrade for the 11.11 Shopping Festival: Multi‑Region Scheduler, Intelligent Storage, Containerized Streaming, and Blockchain Traceability

The article details JD.com’s large‑scale big‑data system overhaul before the 11.11 shopping festival, highlighting a multi‑region Hydra Scheduler, intelligent storage policies, full containerization of the streaming platform, enhanced log reporting, and blockchain‑based traceability that together dramatically improve performance, stability, and user experience.

Intelligent Storagecontainerizationdistributed scheduling
0 likes · 7 min read
JD.com’s Big Data System Upgrade for the 11.11 Shopping Festival: Multi‑Region Scheduler, Intelligent Storage, Containerized Streaming, and Blockchain Traceability
JD Tech
JD Tech
Jul 9, 2018 · Big Data

JD's Large‑Scale Hadoop Cluster Resource Management and Scheduling Architecture

This article describes how JD built a multi‑regional, ten‑thousand‑node Hadoop ecosystem, unified resource management with YARN, introduced a three‑level Router scheduling layer, optimized performance, and integrated deep‑learning frameworks to achieve high availability, cost efficiency, and scalable big‑data processing.

HadoopJD.comYARN
0 likes · 12 min read
JD's Large‑Scale Hadoop Cluster Resource Management and Scheduling Architecture
Architecture Digest
Architecture Digest
Sep 2, 2017 · Big Data

Designing a High‑Availability, High‑Efficiency Distributed Scheduling Platform for Big Data

This article examines the principles, features, and implementation details of distributed scheduling for big‑data ETL pipelines, covering decentralised schedulers, host selection strategies, fault‑tolerance, operator abstraction, elasticity, trigger mechanisms, visual monitoring, alarm handling, data fan‑in/fan‑out, parameter consistency, real‑time quality checks, lineage tracking, and field‑level traceability.

Big DataData PipelineETL
0 likes · 23 min read
Designing a High‑Availability, High‑Efficiency Distributed Scheduling Platform for Big Data
Qunar Tech Salon
Qunar Tech Salon
Dec 25, 2015 · Backend Development

Design and Implementation of Elastic-Job: A Distributed Job Scheduling Framework

Elastic-Job is a Java-based, decentralized distributed job scheduling framework that addresses limitations of existing solutions by providing features such as distributed coordination via Zookeeper, parallel task execution, elastic scaling, centralized management, customizable workflow tasks, and robust non‑functional requirements, with future plans for multi‑language support and enhanced monitoring.

Elastic ScalingZookeeperdistributed scheduling
0 likes · 14 min read
Design and Implementation of Elastic-Job: A Distributed Job Scheduling Framework
High Availability Architecture
High Availability Architecture
Nov 5, 2015 · Backend Development

Elastic-Job: Overview of a Distributed Job Scheduling Framework

This article introduces Elastic-Job, a Java‑based distributed job scheduling framework from Dangdang, covering its origins, core features such as sharding and elastic scaling, deployment with Zookeeper, best‑practice usage, open‑source development philosophy, future improvements, and detailed Q&A.

Elastic-JobJob ShardingZookeeper
0 likes · 17 min read
Elastic-Job: Overview of a Distributed Job Scheduling Framework
Efficient Ops
Efficient Ops
Jun 18, 2015 · Cloud Computing

What Drives China Mobile’s Cloud Computing Strategy? Insights and Lessons

This article summarizes a China Mobile cloud computing talk covering the company’s vision for private and public clouds, core technologies such as lightweight virtualization, distributed scheduling and coordination, the evolution from “small cloud” to “big cloud”, practical reflections on implementation, and strategic recommendations for future development.

EPaaSPaaSVirtualization
0 likes · 18 min read
What Drives China Mobile’s Cloud Computing Strategy? Insights and Lessons