Kubernetes v1.31 Introduces Reliable PersistentVolume Reclaim Policy to Prevent Storage Leaks

Have you ever noticed that after deleting a PVC the storage space does not increase? Kubernetes v1.31 finally fixes this bug and promotes it as a major new feature.

The release highlights an improvement to the PersistentVolume (PV) reclaim policy, now marked as "Always Honor PersistentVolume Reclaim Policy" and entering the Beta stage. Historically, the reclaim policy should delete the underlying storage when a PVC is removed, but earlier Kubernetes versions sometimes ignored the policy, leading to storage leaks.

For example, a PV with the "Delete" reclaim policy bound to a PVC should be removed together with its storage when the PVC is deleted. In early versions, if the PV’s metadata changed manually or a brief network partition occurred before the PVC deletion, the delete event might not trigger the reclaim logic, leaving the storage orphaned.

Additionally, race conditions in the controller could cause PVC and PV status updates to become inconsistent when multiple controllers attempted to handle the same resource simultaneously.

The core of the improvement introduces a new mechanism that guarantees the PV reclaim policy is strictly applied regardless of the order of PVC and PV deletions, especially ensuring that a "Delete" policy always results in the removal of both the PV and its underlying storage.

Implementation details include:

PV and PVC state synchronization improvement: a stricter sync check ensures the PV status is updated promptly when a PVC is deleted, allowing the reclaim policy to take effect.

Controller event‑driven trigger mechanism: the PV controller now reliably reacts to PVC deletion events, even under high load or abnormal conditions.

Enhanced garbage‑collection logic: the controller performs multiple checks of the PV state and its reclaim policy, guaranteeing correct cleanup despite network glitches or node failures.

Idempotency guarantee: the new logic is idempotent, so repeated executions do not cause side effects, which is crucial for recovery after partitions or node restarts.

This mechanism was first introduced as an Alpha feature in Kubernetes v1.23 and promoted to Beta in v1.31, enabled by default. From v1.31 onward, users do not need extra configuration to benefit from the improved reclaim behavior.

Upgrading to v1.31 is recommended for anyone who has experienced incomplete PV deletions, as it ensures proper cleanup of PVs, PVCs, and underlying storage, while also bringing other enhancements such as better node health checks, controller performance optimizations, and security/stability upgrades.

For clusters still running older Kubernetes versions, the article suggests remedial actions: carefully manage the deletion order of PVCs and PVs, manually verify and clean any residual storage after PVC deletion, use idempotent scripts to periodically check for orphaned PVs, and enable the HonorPVReclaimPolicy feature gate if the version supports it.

The takeaway is that fixing a stubborn bug can be celebrated as a new feature, highlighting the importance of reliable resource reclamation in cloud‑native environments.