How Open-Local Simplifies Kubernetes Local Storage in ACK Distro

Background and Motivation

In cloud‑native environments, stateful applications often need persistent local storage. Compared with distributed storage, local disks offer lower cost, higher performance, and easier management, but Kubernetes lacks native awareness of local storage resources, leading to manual labeling, scheduling complexity, and stability issues.

open‑local Architecture

open‑local addresses these gaps with four components:

scheduler‑extender : Extends kube‑scheduler to recognize local storage attributes such as capacity, disk count, and media type, enabling mixed‑resource scheduling.

csi‑plugin : Implements the CSI standard for local disks, providing create/delete/expand volume, snapshot, and metrics capabilities.

agent : Runs on each node, discovers and reports local devices, and applies configuration from the controller.

controller : Distributes initialization configuration to agents and creates the NodeLocalStorage CRD instances.

open‑local also defines two CRDs: NodeLocalStorage: Represents discovered storage devices on a node; created by the controller and updated by agents. NodeLocalStorageInitConfig: Holds global and node‑specific initialization settings, such as allowed VolumeGroups (VGs) and device lists.

Usage Scenarios

Applications requiring capacity isolation to avoid log‑driven disk exhaustion.

Stateful workloads (e.g., HBase, etcd, ZooKeeper, Elasticsearch) that need node‑affinity and high‑performance local disks.

Clusters with many local disks where automated scheduling of stateful apps is desired.

Database workloads that benefit from snapshot‑based backup.

Step‑by‑Step Guide

1. Initialization

Ensure lvm tools are installed. Edit the NodeLocalStorageInitConfig to define VGs and devices:

apiVersion: csi.aliyun.com/v1alpha1
kind: NodeLocalStorageInitConfig
metadata:
  name: open-local
spec:
  globalConfig:
    listConfig:
      vgs:
        include:
        - open-local-pool-[0-9]+
        - your-vg-name
  resourceToBeInited:
    vgs:
    - devices:
      - /dev/vdc
      name: open-local-pool-0

After saving, the controller and agents populate NodeLocalStorage resources on each node.

2. Dynamic Volume Provisioning

open‑local provides storage classes such as open-local-lvm, open-local-lvm-xfs, and open-local-lvm-io-throttling:

# kubectl get sc
NAME                     PROVISIONER            RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
open-local-lvm           local.csi.aliyun.com   Delete          WaitForFirstConsumer   true                  8d
open-local-lvm-xfs       local.csi.aliyun.com   Delete          WaitForFirstConsumer   true                  6h56m
open-local-lvm-io-throttling local.csi.aliyun.com Delete          WaitForFirstConsumer   true                  -

Create a StatefulSet using open-local-lvm (ext4) or open-local-lvm-xfs (xfs):

# kubectl apply -f https://raw.githubusercontent.com/alibaba/open-local/main/example/lvm/sts-nginx.yaml

Verify the PVC, PV, and Pod status to confirm successful provisioning.

3. Volume Expansion

Patch the PVC to request a larger size:

# kubectl patch pvc html-nginx-lvm-0 -p '{"spec":{"resources":{"requests":{"storage":"20Gi"}}}}'

Check the PVC and PV to see the updated capacity.

4. Snapshot Support

open‑local defines a VolumeSnapshotClass named open-local-lvm. Create a snapshot:

# kubectl apply -f https://raw.githubusercontent.com/alibaba/open-local/main/example/lvm/snapshot.yaml
# kubectl get volumesnapshot
NAME          READYTOUSE   SOURCEPVC          RESTORESIZE   SNAPSHOTCLASS
new-snapshot-test   true   html-nginx-lvm-0   5Gi   open-local-lvm

Deploy a new Pod from the snapshot to verify data consistency.

5. Native Block Device

Deploy a StatefulSet that mounts the volume as a block device (e.g., /dev/sdd):

# kubectl apply -f https://raw.githubusercontent.com/alibaba/open-local/main/example/lvm/sts-block.yaml

Inspect the PVC to see volumeMode: Block.

6. IO Throttling

Define a storage class with IOPS and throughput limits:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: open-local-lvm-io-throttling
provisioner: local.csi.aliyun.com
parameters:
  csi.storage.k8s.io/fstype: ext4
  volumeType: "LVM"
  bps: "1048576"   # ~1 MiB/s
  iops: "1024"
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

Deploy a StatefulSet using this class, then exec into the pod and run fio to observe the throttled throughput (≈1024 KiB/s).

7. Temporary Volumes

open‑local can create CSI temporary volumes whose lifecycle matches the Pod, similar to emptyDir: # kubectl apply -f ./example/lvm/ephemeral.yaml Describe the pod to see the temporary volume details.

8. Monitoring Dashboard

open‑local ships with a Grafana dashboard that visualizes node storage devices, volumes, and usage. The dashboard can be accessed through the ACK Distro monitoring UI.

Conclusion

By integrating open‑local, operators reduce manual effort and improve cluster stability, while developers gain high‑performance local storage with advanced features such as snapshots, IO throttling, and block‑device access, making stateful cloud‑native workloads on ACK Distro more reliable and efficient.