Resize Kubernetes Pods On‑The‑Fly: In‑Place Vertical Scaling in v1.27

Background

Changing the CPU or memory resources of a pod traditionally required deleting and recreating the pod, causing a disruptive restart. Kubernetes v1.27 adds an alpha feature that allows live adjustment of these resources without restarting the container.

New Alpha Feature in v1.27

The API now permits patching the resources field of a running pod’s spec to change CPU and memory requests or limits. The change is reflected in the pod’s status, not the original spec, so monitoring tools must read the new status fields.

New Fields

restartPolicy – controls container behavior during a resource adjustment.

allocatedResources – appears in containerStatuses and shows the node resources allocated to each container.

resources – added to the container status and mirrors the actual requests and limits reported by the container runtime (CRI).

resize – indicates the last resize request’s state and can be Proposed, InProgress, Deferred, or Infeasible.

Resize Status Values

Proposed : The request has been validated and recorded.

InProgress : The node has accepted the request and is applying it to the pod’s containers.

Deferred : The node cannot approve the request now; it will retry later when resources become available.

Infeasible : The requested resources exceed what the node can provide.

When to Use

Running pods have resource limits that are too high or too low.

Over‑provisioned pods occupy nodes, causing scheduling failures for other workloads.

Evicting high‑resource stateful pods is costly; instead, shrink lower‑priority pods or move them.

How to Enable and Use

Enable the InPlacePodVerticalScaling feature gate and start a local cluster with the flag:

FEATURE_GATES=InPlacePodVerticalScaling=true ./hack/local-up-cluster.sh

After the cluster is up, set the kubeconfig and use kubectl to patch a pod’s resources. Example commands:

export KUBECONFIG=/var/run/kubernetes/admin.kubeconfig
cluster/kubectl.sh
# or configure the default kubeconfig
export KUBERNETES_PROVIDER=local
cluster/kubectl.sh config set-cluster local --server=https://localhost:6443 --certificate-authority=/var/run/kubernetes/server-ca.crt
cluster/kubectl.sh config set-credentials myself --client-key=/var/run/kubernetes/client-admin.key --client-certificate=/var/run/kubernetes/client-admin.crt
cluster/kubectl.sh config set-context local --cluster=local --user=myself
cluster/kubectl.sh config use-context local
cluster/kubectl.sh

Once the cluster is running, you can schedule pods with specific resource requests and later adjust them with kubectl patch or kubectl edit.

Example Use Cases

1. Cloud Development Environment

Developers write code locally but build and test in a pod that mirrors production resources. The pod starts with minimal resources and is scaled up during builds or tests, then scaled down, avoiding OOM kills.

2. Java Process Initialization CPU Requirement

Some Java applications need higher CPU during startup than during steady‑state operation. You can request extra CPU for the init phase, then lower the request after the application is ready.

Known Issues

containerd versions < 1.6.9 do not support the required CRI features; resize requests may remain stuck in InProgress and the resources field will not update.

Resize operations can conflict with other pod updates, causing delays.

It may take time for the adjusted resources to appear in the pod’s status.

The feature is incompatible with static CPU management policies.

Acknowledgments

The feature is the result of extensive collaboration within the Kubernetes community. Thanks to contributors such as @thockin, @derekwaynecarr, @dchen1107, @ruiwen-zhao, @wangchen615, @bobbypage, and many others for API design, code reviews, testing, and documentation.