Why Updating a Kubernetes Resource via API Fails While kubectl Apply Succeeds – Understanding Update vs Patch

OpenKruise is an Alibaba Cloud open‑source large‑scale application automation engine that extends native Kubernetes controllers such as Deployment and StatefulSet with features like in‑place upgrade, priority‑based rollout, multi‑AZ workload abstraction, and unified sidecar injection.

In many Alibaba Cloud environments, OpenKruise is used for pod deployment and release management. A customer tried to update an Advanced StatefulSet by modifying the image field in a YAML file and sending an update request via the Kubernetes API, but received an error about missing metadata.resourceVersion. The same YAML applied with kubectl apply succeeded.

Update Mechanism

Kubernetes assigns every resource a globally unique metadata.resourceVersion. The version changes on every modification. An update request must include the current resourceVersion of the object; otherwise the apiserver rejects the request with a Conflict.

Fetch the existing object (e.g., via kubectl get or an informer).

Modify the desired fields on the fetched object.

Submit the modified object with an update request.

The apiserver validates that the submitted resourceVersion matches the latest one; if not, it returns a Conflict.

The diagram below shows the conflict handling when multiple users attempt concurrent updates.

Patch Mechanism

Patch requests send only the changed fields. Kubernetes supports four patch types: JSON Patch, Merge Patch, Strategic Merge Patch, and Apply Patch (server‑side apply). The default is Strategic Merge Patch for native resources.

Examples:

JSON Patch to add a new container:

kubectl patch deployment/foo --type='json' -p '[{"op":"add","path":"/spec/template/spec/containers/1","value":{"name":"nginx","image":"nginx:alpine"}}]'

JSON Patch to replace an existing container image:

kubectl patch deployment/foo --type='json' -p '[{"op":"replace","path":"/spec/template/spec/containers/0/image","value":"app-image:v2"}]'

Merge Patch requires sending the whole containers list:

kubectl patch deployment/foo --type='merge' -p '{"spec":{"template":{"spec":{"containers":[{"name":"app","image":"app-image:v2"},{"name":"nginx","image":"nginx:alpine"}]}}}}'

Strategic Merge Patch uses patchStrategy:"merge" and patchMergeKey:"name" annotations in the resource definition, allowing updates by specifying the container name instead of an index.

kubectl patch deployment/foo -p '{"spec":{"template":{"spec":{"containers":[{"name":"nginx","image":"nginx:mainline"}]}}}}'

Strategic Merge Patch works only for native resources and aggregated‑API CRDs; generic CRDs fall back to Merge Patch.

kubectl Apply and Edit

kubectl apply

implements a client‑side apply workflow: it stores the last applied configuration in an annotation, computes a diff against the current object, and sends the diff as a Strategic Merge Patch. Server‑side apply (available since K8s 1.14) performs similar logic on the server. kubectl edit fetches the current object, opens it in an editor, and after saving computes a diff and submits it as a patch, avoiding direct update to reduce conflict risk.

Guidance

If only your own controller modifies a field (e.g., custom labels), use patch for simplicity.

If the field may be modified by other components (e.g., replicas affected by HPA), prefer update to avoid unintended overwrites.

In the original case, the customer switched to fetching the object first, modifying it, and then issuing an update, which succeeded in performing an in‑place upgrade of the Advanced StatefulSet.