Master LVM Online Expansion: Zero‑Downtime Disk Scaling on Linux

Overview

LVM adds an abstraction layer between physical storage and filesystems, allowing online expansion of logical volumes without service interruption. This eliminates the downtime, data‑risk and inflexibility of traditional partition‑based resizing.

Key concepts

Physical Volume (PV) : a whole disk or partition initialized for LVM.

Volume Group (VG) : a pool of one or more PVs from which logical volumes are allocated.

Logical Volume (LV) : the block device presented to the OS; its size can be changed at any time.

Physical Extent (PE) : the smallest allocation unit (default 4 MiB).

Prerequisites for online expansion

Filesystem must support online growth (e.g., ext4, xfs).

Kernel version that includes LVM2 (RHEL 7+/Ubuntu 16.04+).

At least one unused block device or free space in the VG.

No active snapshots on the target LV.

Step‑by‑step expansion example

Expand a 100 GiB LV /dev/data_vg/data_lv mounted at /data to 200 GiB by adding a new 100 GiB disk /dev/sdc.

1. Detect the new disk

# lsblk
# for host in /sys/class/scsi_host/host*; do echo "- - -" > ${host}/scan; done
# lsblk

2. Create a physical volume

# pvcreate /dev/sdc
# pvs

3. Extend the volume group

# vgextend data_vg /dev/sdc
# vgs

4. Extend the logical volume

# lvextend -l +100%FREE /dev/data_vg/data_lv
# lvs

5. Grow the filesystem

For XFS: # xfs_growfs /data For EXT4: # resize2fs /dev/data_vg/data_lv Verify:

# df -h /data

One‑command expansion (LVM 2.02.98+)

# lvextend -r -l +100%FREE /dev/data_vg/data_lv

The -r flag automatically resizes the filesystem after the LV is extended.

Alternative scenarios

Expanding an existing virtual disk : after resizing the virtual disk in the hypervisor, run echo 1 > /sys/block/sdb/device/rescan then pvresize /dev/sdb and continue with lvextend -r.

Partitioned PV : grow the partition with growpart /dev/sdb 1 or parted /dev/sdb resizepart 1 100%, run partprobe, then pvresize /dev/sdb1 before extending the LV.

Root LV : add a new PV, extend the root VG, then lvextend -r -l +100%FREE /dev/centos/root. For XFS use xfs_growfs /.

Thin provisioning : first extend the thin pool ( lvextend -l +100%FREE data_vg/data_pool) then the thin LV ( lvextend -L 500G data_vg/data_thin_lv) and finally grow the filesystem.

Using unused space on the same disk : create a new LVM‑type partition, pvcreate it, vgextend the VG, and lvextend -r -l +100%FREE the target LV.

Practical cases

MySQL data directory expansion

# df -h /var/lib/mysql
# pvcreate /dev/sdc
# vgextend mysql_vg /dev/sdc
# lvextend -r -l +100%FREE /dev/mysql_vg/mysql_lv
# df -h /var/lib/mysql

The MySQL service remained online throughout the operation.

Batch expansion script

#!/bin/bash
set -e
NEW_DISK=$1
VG_NAME=$2
LV_NAME=$3
pvcreate "$NEW_DISK"
vgextend "$VG_NAME" "$NEW_DISK"
lvextend -r -l +100%FREE "/dev/$VG_NAME/$LV_NAME"

Ansible automation

---
- name: LVM online expansion
  hosts: all
  become: yes
  vars:
    new_disk: "{{ new_disk }}"
    vg_name: "{{ vg_name }}"
    lv_name: "{{ lv_name }}"
  tasks:
    - name: Scan SCSI bus
      shell: for host in /sys/class/scsi_host/host*; do echo "- - -" > ${host}/scan; done
      changed_when: false
    - name: Wait for new disk
      wait_for:
        path: "{{ new_disk }}"
        state: present
        timeout: 30
    - name: Create PV and extend VG
      lvg:
        vg: "{{ vg_name }}"
        pvs: "{{ new_disk }}"
        state: present
    - name: Extend LV and resize FS
      lvol:
        vg: "{{ vg_name }}"
        lv: "{{ lv_name }}"
        size: +100%FREE
        resizefs: yes

Kubernetes PVC expansion (LVM‑backed storage)

# Ensure the StorageClass allows expansion
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-lvm
provisioner: kubernetes.io/no-provisioner
allowVolumeExpansion: true

# Patch the PVC size
kubectl patch pvc mysql-data -p '{"spec":{"resources":{"requests":{"storage":"200Gi"}}}}'

Best practices

Capacity planning

Start expansion when usage exceeds 70 % and act immediately at 85 %.

Set multi‑level alerts (70 %, 80 %, 90 %).

Expand in 50‑100 % increments to avoid frequent operations.

Operational guidelines

Take a snapshot or backup before any change.

Prefer low‑traffic windows for non‑critical expansions.

Log every command and its output (e.g., script /var/log/lvm_extend_$(date +%Y%m%d_%H%M%S).log).

Verify the result after each step.

Architecture design

Separate data LV from the OS LV.

Use meaningful VG/LV names (e.g., mysql_vg, data_lv).

Leave 10‑20 % free space in each VG for rapid future growth.

Use thin provisioning cautiously and monitor thin‑pool usage.

Multi‑path and RAID environments

Operate on multipath devices ( /dev/mapper/mpatha) rather than raw paths.

For hardware RAID, add disks at the controller before LVM steps.

For software RAID ( mdadm), extend the md device first, then run pvresize.

Cloud‑specific considerations

EBS/云盘: expand in the console, then run LVM steps after the cloud‑side resize completes.

During VM live‑migration, pause LVM operations.

NVMe devices require different scan commands, e.g. nvme ns-rescan /dev/nvme0n1 or echo 1 > /sys/class/nvme/nvme0/rescan_controller.

Troubleshooting

New disk not detected : run a SCSI scan, check dmesg, verify /dev entries, and ensure the hypervisor saved the configuration.

pvcreate filter excludes device : clear existing signatures with wipefs -a /dev/sdc or adjust /etc/lvm/lvm.conf filter.

vgextend reports PV already in VG : confirm with pvs; if already in the target VG, skip vgextend and proceed.

LV size changed but filesystem unchanged : run the appropriate filesystem grow command ( xfs_growfs or resize2fs) or use lvextend -r.

resize2fs error “On‑line shrinking not supported” : ext4 cannot be shrunk online; perform shrinking offline after unmounting.

xfs_growfs error “is not a mounted XFS filesystem” : provide the mount point, not the device path.

I/O errors during LVM operations : check dmesg, SMART data, multipath status, and storage‑array logs for hardware issues.

Data loss after expansion : verify device names before each command, keep backups, and ensure the filesystem grow step is executed.

LVM volumes fail to activate on boot : run pvscan, vgchange -ay, and check /etc/fstab and initramfs.

LVM metadata corruption : restore from backup with vgcfgrestore or remove missing PVs with vgreduce --removemissing --force.

Advanced topics

LVM cache : SSD cache can accelerate HDD pools; when extending a cached LV, extend the cache LV if needed ( lvs -a -o +devices,cache_mode).

LVM mirroring (RAID‑1) : mirrored LVs replicate new space automatically ( lvextend -l +100%FREE /dev/vg/mirrored_lv).

VDO (Virtual Data Optimizer) : expand VDO first ( vdo growLogical --name=vdo_data --vdoLogicalSize=2T) then the LV.

Stratis : a next‑gen storage manager; expansion is a single command ( stratis pool add-data my_pool /dev/sdc).

Monitoring & automation

Prometheus alert rules (node_exporter)

groups:
- name: lvm
  rules:
  - alert: LVMVolumeHighUsage
    expr: 100 - (node_filesystem_avail_bytes{mountpoint=~"/data.*"} / node_filesystem_size_bytes{mountpoint=~"/data.*"} * 100) > 80
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "LVM volume usage high"
      description: "{{ $labels.mountpoint }} usage is {{ $value }}%"
  - alert: LVMVolumeAlmostFull
    expr: 100 - (node_filesystem_avail_bytes{mountpoint=~"/data.*"} / node_filesystem_size_bytes{mountpoint=~"/data.*"} * 100) > 90
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "LVM volume almost full"
      description: "{{ $labels.mountpoint }} usage is {{ $value }}%"

Automatic expansion script (caution advised)

#!/bin/bash
THRESHOLD=85
VG_NAME="data_vg"
LV_NAME="data_lv"
MOUNT_POINT="/data"
USAGE=$(df -h "$MOUNT_POINT" | awk 'NR==2 {print $5}' | tr -d '%')
if [ "$USAGE" -gt $THRESHOLD ]; then
  FREE_PE=$(vgs --noheadings -o vg_free_count "$VG_NAME" | tr -d ' ')
  if [ "$FREE_PE" -gt 0 ]; then
    logger "Auto‑expanding $VG_NAME/$LV_NAME (usage $USAGE%)"
    lvextend -r -l +50%FREE "/dev/$VG_NAME/$LV_NAME"
  else
    logger "No free space in $VG_NAME for auto‑expansion"
  fi
fi

Run via cron after thorough testing; manual approval is recommended for production.

Performance considerations

Metadata updates are atomic and complete in milliseconds.

Filesystem grow commands cause a brief I/O spike but do not lock the filesystem.

Extending multi‑terabyte LVs is fast for LVM; the filesystem grow step may take minutes for ext4.

Thin‑pool expansion is slightly slower due to extra metadata handling.

Comparison with traditional partition expansion

Downtime : LVM online expansion – none; traditional – typically 1‑4 hours.

Data risk : LVM – very low; traditional – higher due to repartitioning.

Complexity : LVM – 3‑5 commands; traditional – backup, repartition, restore.

Cross‑disk expansion : LVM supports; traditional does not.

Shrink support : LVM (ext4 offline, xfs not); traditional generally not supported.

Snapshot capability : LVM supports; traditional does not.

Conclusion

LVM online expansion provides a reliable, low‑downtime method to grow storage on Linux systems. By mastering the PV → VG → LV workflow, using the lvextend -r shortcut, and following capacity‑planning and operational best practices, operations engineers can safely handle growth of data directories, root filesystems, thin‑provisioned volumes, and cloud‑native PVCs. Proper monitoring, logging, and cautious automation complete a robust storage‑scaling strategy.