Tagged articles

task scheduling

211 articles · Page 3 of 3

May 5, 2017 · Backend Development

WeChat MQ 2.0: Enhanced Asynchronous Queue Design and Optimizations

The article introduces WeChat's self‑developed MQ 2.0 asynchronous queue, detailing its architecture, cross‑machine consumption model, improved task scheduling, efficient processing frameworks—including a MapReduce‑style engine and streaming tasks—and robust overload protection mechanisms that together boost reliability and performance for large‑scale backend services.

MapReduceStreaming Tasksdistributed systems

0 likes · 12 min read

WeChat MQ 2.0: Enhanced Asynchronous Queue Design and Optimizations

Hujiang Technology

Feb 6, 2017 · Backend Development

Design and Evolution of Juice: A Mesos‑Based Asynchronous Task Scheduling System

The article describes how Hujiang built and iterated a three‑generation asynchronous task scheduling platform—Juice—using Mesos and Docker to achieve on‑demand resource allocation, horizontal scalability, and efficient handling of large multimedia processing workloads.

Mesosasynchronous-processingbackend architecture

0 likes · 10 min read

Design and Evolution of Juice: A Mesos‑Based Asynchronous Task Scheduling System

Weidian Tech Team

Nov 28, 2016 · Big Data

How We Built the Mars Big Data Platform to Boost Development Efficiency

The article explains why Weidian needed a new big data development platform, outlines the functional features of the Mars system, describes its architecture, scheduling mechanisms, task execution flow, and discusses remaining challenges and future enhancements.

HadoopPlatform Architecturedistributed systems

0 likes · 11 min read

How We Built the Mars Big Data Platform to Boost Development Efficiency

Ctrip Technology

Oct 14, 2016 · Operations

Qunar Network Device Operations Platform: Architecture, Features, and Continuous Optimization

This article presents the design, implementation, and ongoing improvements of Qunar's network device operations platform, detailing its background, optimization strategies, permission model, automated tasks, monitoring capabilities, and how it enhances operational efficiency while reducing risk.

Access ControlAuditPlatform

0 likes · 7 min read

Qunar Network Device Operations Platform: Architecture, Features, and Continuous Optimization

Meituan Technology Team

Sep 23, 2016 · Databases

Database Automation Platform at Meituan: Architecture, Practices, and Lessons

Meituan’s Database Automation Platform evolved from a simple Django‑based 1.0 system into a modular 2.0 architecture using CMDB, RabbitMQ, and Celery, standardizing metadata, providing self‑service APIs, and automating tasks such as cluster provisioning, online schema changes, and high‑availability failover, now handling hundreds of daily operations for over a thousand developers while planning further component‑library refactoring.

CMDBDatabase AutomationDevOps

0 likes · 27 min read

Database Automation Platform at Meituan: Architecture, Practices, and Lessons

Liulishuo Tech Team

Jun 17, 2016 · Big Data

Building a Scalable Big Data Platform on AWS: Architecture and Execution Service Design

This article details the architectural design and implementation of a scalable big data platform built on AWS services, highlighting the transition from HDFS to S3 for storage, the use of EMR for elastic compute, and a custom Execution Service integrated with Consul and Airflow for automated cluster management and task scheduling.

AWS EMRAirflowBig Data Architecture

0 likes · 11 min read

Building a Scalable Big Data Platform on AWS: Architecture and Execution Service Design

High Availability Architecture

Mar 18, 2016 · Backend Development

High‑Availability Architecture of Upyun Image Processing Service

The article details Upyun's high‑availability image processing architecture, covering workload‑aware system design, custom GmServer implementation, task scheduling, current strengths and limitations, and future directions such as a ServiceServer‑based queue and Docker‑driven dynamic scaling.

Dockerbackend architecturecloud services

0 likes · 15 min read

High‑Availability Architecture of Upyun Image Processing Service

Java High-Performance Architecture

Nov 21, 2015 · Operations

Prevent Duplicate Cron Jobs with Linux flock: A Step‑by‑Step Guide

This article demonstrates how cron jobs can overlap when a script runs longer than its schedule, and shows how to use Linux's flock file lock to ensure only one instance of the script runs at a time, preventing execution conflicts.

CronFile LockLinux

0 likes · 3 min read

Prevent Duplicate Cron Jobs with Linux flock: A Step‑by‑Step Guide

MaGe Linux Operations

Oct 21, 2015 · Operations

How JobCenter Transforms Distributed Task Scheduling in E‑Commerce

JobCenter is a distributed task coordination platform that replaces crontab with a unified scheduling, monitoring, and alerting system, enabling e‑commerce teams to manage thousands of web‑service‑based jobs, ensure reliable execution, and gain clear visibility into task performance.

automationdistributed systemsmonitoring

0 likes · 7 min read

How JobCenter Transforms Distributed Task Scheduling in E‑Commerce

MaGe Linux Operations

May 21, 2015 · Operations

Master Linux Task Scheduling: at, batch, crontab, and anacron Explained

This guide walks you through Linux's task‑scheduling tools—at, batch, crontab, and anacron—covering one‑time and periodic job setup, command syntax, permission files, and practical examples to automate system administration tasks.

CronLinuxat

0 likes · 11 min read

Master Linux Task Scheduling: at, batch, crontab, and anacron Explained

MaGe Linux Operations

Jul 10, 2014 · Operations

Mastering Linux Cron: How to Schedule Tasks Efficiently

This guide explains what the Linux cron service is, how to configure it via /var/spool/cron and /etc/crontab, the syntax of cron expressions, environment variables, and provides practical examples for scheduling recurring tasks.

Cronautomationtask scheduling

0 likes · 9 min read

Mastering Linux Cron: How to Schedule Tasks Efficiently