How Cainiao Ark’s Elastic Scheduling Boosts Resource Efficiency and Cuts Costs

Introduction

Cainiao Ark is a resource‑management and operations platform for all Cainiao R&D, responsible for controlling infrastructure resources to support daily operations and large‑scale promotional events. Elastic scheduling is a core feature that dynamically adjusts resources based on business load.

Why Elastic Scheduling?

Before elastic scheduling, Cainiao’s resource utilization was low because capacity was estimated manually through single‑machine performance tests and experience‑based traffic forecasts, leaving large safety margins. Scaling actions were infrequent, causing waste during off‑peak periods and insufficient resources during traffic spikes. Elastic scheduling enables timely expansion when pressure rises and releases resources when it falls, maximizing efficiency while maintaining stability.

Why Cainiao Is Ideal for Elastic Scheduling

Business characteristics involve high‑frequency, short‑lived traffic peaks across merchants, CPs, and consumers, making a pulse‑type scaling scenario rare.

Since early 2017 Cainiao has fully containerized its services and adopted a hybrid‑cloud architecture, shifting resource management from “machine‑centric” to “application‑centric”. The Ark platform proved its stability during major sales events (e.g., 618, Double‑11).

The majority of core applications are stateless online services with clear peak‑valley patterns, providing abundant scaling opportunities.

Uniform technology stacks and standardized monitoring tools (Alimonitor, EagleEye, Alimetrics) give elastic scheduling a solid data foundation.

Comparison with Similar Products

Unlike domain‑specific elastic schedulers or public‑cloud auto‑scaling services, Cainiao’s solution must handle a wide variety of un‑homogeneous applications without strong business assumptions. It therefore offers highly configurable policies, a unified decision engine, and a “one‑click” onboarding experience that abstracts complexity away from users.

Current Deployment Status

To date, Ark can elastically manage application groups with more than 15 containers each, achieving an average CPU utilization above 20% across groups and scaling over 3,000 containers daily. During the 2017 Double‑11 event, elastic scheduling reduced the ratio of physical CPU cores to package count significantly.

Elastic Scheduling Model

Elastic scheduling follows a closed‑loop feedback model: monitoring data from each application group feeds strategy calculations, which produce scaling actions; the container operation service adjusts the cluster size; the resulting cluster behavior is fed back for further analysis. Historical data are stored offline for periodic automatic policy updates.

Three‑Layer Decision Architecture

Strategy Layer : multiple independent strategies (resource safety, resource optimization, time‑based, service safety, etc.) each output a scaling action and quantity.

Aggregation Layer : merges all strategy results, preferring expansion over contraction and selecting the largest expansion or smallest contraction quantity.

Execution Layer : applies final decisions, considering rules such as data‑center balancing, current scaling state, manual‑approval mode, and min/max protection.

This separation ensures stateless, idempotent calculations and high availability.

Key Decision Strategies

Resource Safety : monitors CPU, load‑1, and process‑running queues; triggers expansion when any metric exceeds configurable thresholds.

Resource Optimization : triggers contraction when all three metrics fall below lower thresholds, unless other strategies demand expansion.

Time Strategy : pre‑schedules capacity for predictable periodic traffic spikes.

Service Safety : evaluates response‑time (rt) and success‑rate for message‑queue consumers, RPC, and HTTP services; most expansions originate from this strategy.

Multi‑Service Voting : only expands when a sufficient proportion of services in a group violate rt thresholds, reducing false positives.

Downstream Analysis : checks whether rt violations stem from downstream dependencies; expands only if the root service itself is at fault.

Handling Spikes and Noise

All calculations use sliding time windows (5‑minute for expansion‑oriented strategies, 10‑minute for contraction‑oriented strategies). Within each window the maximum and minimum values are discarded, and the average of the remaining data is used to filter out spikes.

Scaling Quantity Calculation

Contraction reduces the current container count by 10%. Expansion is limited to 50% of the current count; the exact number is derived from a sigmoid function applied to the degree of threshold violation, with separate tuning for CPU (bounded) and rt (unbounded) metrics.

Big‑Event Support

During major promotions, Ark’s “container plan” allows owners to request capacity in advance. Elastic scheduling switches to a manual‑approval mode, scaling containers to the planned amount. After the event, non‑elastic groups shrink immediately, while elastic groups gradually release resources via normal strategies.

Future Directions

The system is still evolving; further improvements are expected in scheduling accuracy and coverage. Contributions on container, scheduling, and middleware technologies are welcomed.