How Kube Queue Optimizes Batch Job Scheduling in Kubernetes

Batch processing workloads such as data pipelines, scientific simulations, and AI model training consume large amounts of compute resources, requiring intelligent queuing based on priority and available resources to maximize cluster utilization.

Problems with the Default Kubernetes Scheduler

Lack of automated queuing : Pending Pods accumulate when resources are insufficient, slowing the scheduler and impacting online services.

Limited queuing strategies : Only priority and creation order are considered, which cannot handle diverse workload requirements.

No multi‑queue capability : Single‑queue design leads to resource starvation for other users or tenants.

Heterogeneous task types : Different workloads (ML, HPC, big‑data, offline workflows) need distinct resource‑scoring logic, increasing maintenance complexity.

Because of these limitations, many cloud providers avoid using the native scheduler for complex batch scenarios.

Queue Types in Kubernetes

Kubernetes distinguishes two categories of queues:

Decoupled Queue : Operates at a layer separate from the scheduler, handling task ordering and fairness without direct awareness of node topology. Examples include Kube Queue and the open‑source Kueue.

Coupled Queue : Tightly integrated with the scheduler; tasks are dequeued only when they can be scheduled. Representative implementations are Volcano and YuniKorn.

Kube Queue Overview

Kube Queue is a core component of Alibaba Cloud Container Service (ACK) AI Suite, designed to address the shortcomings of the default scheduler for AI and batch workloads. It works together with the Arena component and ACK’s Elastic Quota feature to provide multi‑tenant quota management and automated queuing.

The architecture consists of:

Kube Queue Controller : Central controller that creates and manages Queue objects based on quota configuration.

Operator Extension : Pluggable extensions that add custom support for different job types.

Work Mechanism

Queue

A Queue object encapsulates scheduling policies, quota limits, and queuing parameters. Kube Queue automatically creates Queue objects for each configured quota (ResourceQuota, ElasticQuota, ElasticQuotaTree), so users only need to define the upper‑level quota.

QueueUnit

A QueueUnit represents a normalized task entity that strips away non‑queuing parameters, reducing the number of objects the controller must watch. Any workload can integrate by implementing the Extension’s queuing interface, making the system extensible to new job types.

Scheduling Workflow

Job submission and enqueuing : When a job that Kube Queue recognizes is created (usually with .spec.suspend: true), the Job Operator and its Extension receive the creation event and evaluate resources and priority, then create a QueueUnit in the Enqueued state.

Queueing and waiting for scheduling : The controller assigns the QueueUnit to a specific Queue based on policy, orders it by priority and creation time, and repeatedly extracts the head of each queue for a scheduling attempt.

Dequeue and execution : Once a QueueUnit is dequeued, its state changes to Dequeued, the Extension removes the suspend flag, and the scheduler creates the pod. If the pod fails to schedule within a configurable timeout, the QueueUnit is re‑enqueued to avoid head‑of‑line blockage. Successful execution updates the state to Running.

Queue Strategies

Kube Queue offers three configurable strategies, each applicable per‑queue:

Round‑Robin (default) : Mirrors the kube‑scheduler’s round‑robin mechanism; tasks are tried sequentially and fall back to an Unschedulable queue on failure. Suitable when maximizing overall quota utilization is the priority.

Blocking : Only the front‑most task in a queue is considered for scheduling, allowing large tasks to obtain resources even when many small tasks are present. Use this to guarantee task priority.

Strict Priority : After a running task finishes, the scheduler attempts the highest‑priority pending task first, providing a compromise between round‑robin and blocking strategies.

Getting Started with Kube Queue

Install the kube-queue component from the ACK AI Suite page (one‑click installation). After installation, the kube-queue namespace contains the controller and various extension controllers.

By default, the controller sets oversellrate to 2, allowing the queue to schedule tasks using up to twice the requested resources; this can be adjusted to 1 in the Deployment spec.

To enable queuing between two Jobs, first create an ElasticQuotaTree that limits the queue to 1 CPU and 1 GiB memory and binds the default namespace:

apiVersion: scheduling.sigs.k8s.io/v1beta1
kind: ElasticQuotaTree
metadata:
  name: elasticquotatree
  namespace: kube-system
spec:
  root:
    name: root
    max:
      cpu: 1
      memory: 1Gi
    min:
      cpu: 1
      memory: 1Gi
    children:
    - name: child-1
      max:
        cpu: 1
        memory: 1Gi
      namespaces:
      - default

Submit two Jobs with the .spec.suspend field set to true so they enter the queue:

apiVersion: batch/v1
kind: Job
metadata:
  generateName: pi-
spec:
  suspend: true
  completions: 1
  parallelism: 1
  template:
    spec:
      containers:
      - name: pi
        image: perl:5.34.0
        command: ["sleep", "1m"]
        resources:
          requests:
            cpu: 1
          limits:
            cpu: 1
      restartPolicy: Never

After submission, only one Job runs while the other remains pending. When the first Job completes, the second is automatically dequeued and executed, demonstrating end‑to‑end automatic control.

What’s Next

The article outlines the importance of queue systems in the cloud‑native era and details how ACK’s Kube Queue fits into the Kubernetes ecosystem. Future posts will explore building a full‑featured task management platform on top of ElasticQuotaTree and advanced scheduling features.