How to Extend the Kubernetes Scheduler: Four Approaches

In a Kubernetes cluster there are four primary mechanisms to extend the scheduler’s capabilities.

Scheduler Extender

The original extensibility mechanism, called a scheduler extender, is invoked by kube‑scheduler via HTTP and receives the full list of Nodes for each scheduling cycle. Because the entire node list is transferred over the network, this approach incurs high scheduling latency. Integration requires adding an extenders section to the KubeSchedulerConfiguration as shown below.

apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
clientConnection:
  kubeconfig: "/etc/kubernetes/scheduler.conf"
leaderElection:
  leaseDuration: 15s
  renewDeadline: 10s
extenders:
- urlPrefix: "{scheduler-svc}"
  filterVerb: filter
  bindVerb: bind
  nodeCacheCapable: true
  weight: 1
  httpTimeout: 30s
  enableHTTPS: true
  tlsConfig:
    insecure: true
  managedResources:
  - name: "foot.io/gpu"
    ignoredByScheduler: false

This method is easy to integrate multiple vendor plugins into a single scheduling flow, but the high latency remains a drawback.

Scheduler Framework

Introduced in Kubernetes 1.16, the scheduler framework refactors the scheduler into two main phases—Filter and Bind—each further divided into multiple extension points. Developers can write plugins for these points, achieving higher scheduling efficiency because the calls are in‑process. However, any change requires recompiling the scheduler binary, making it a code‑level modification.

Multi‑Schedulers

Deploying multiple independent schedulers within the same cluster allows specialized scheduling for different resource types, such as CPU‑focused or GPU‑focused schedulers. This isolation prevents interference between schedulers, but it can waste resources and prevents combined policies—for example, a pod that needs both CPU and GPU scheduling cannot be satisfied by separate schedulers.

WebAssembly Extension

A newer approach is to write scheduler plugins in WebAssembly (Wasm) and load them directly in the scheduler process. This enables flexible, plug‑and‑play extensions without recompiling the scheduler. The technique is still under development but is already used in projects such as Istio, Envoy, and Dapr. The author suggests that achieving the same capabilities as the other three methods via Wasm would be the preferred solution.

The article also notes that Go’s plugin system is unsuitable for this purpose, prompting the shift to Wasm.

References

WebAssembly: https://github.com/kubernetes-sigs/kube-scheduler-wasm-extension