Master LVM from Zero: Complete Guide to Logical Volume Management, Expansion, Snapshots & Migration

1. Problem Background

Disk full is a common production issue. When the partition uses LVM, expansion isn't simply "add disk, mkfs, mount" but requires: create PV → extend VG → extend LV → grow filesystem. Junior admins often struggle with PV/VG/LV relationships, fear of data loss, confusion between lvextend/resize2fs/xfs_growfs, shrinking risks, metadata backup/restore, and snapshot creation.

2. Applicable Scenarios

Server disk full, need expansion

Database directory needs more space

Merge multiple small disks into one large volume

Migrate single disk to new disk

Backup database via snapshots

Cross-host LVM migration

Resolve filesystem fragmentation

RAID + LVM combination

Cloud disk expansion

Capacity planning

Disaster recovery drills

3. Core Concepts

3.1 LVM Three-Layer Structure

PV (Physical Volume) : Physical disk or partition

VG (Volume Group) : Resource pool of multiple PVs

LV (Logical Volume) : Virtual partition allocated from VG

FS (File System) : Created on LV (ext4, xfs, etc.)

Analogy: Physical disk → PV; Multiple disks merged → VG; Partition from VG → LV; Format LV → FS.

3.2 PE and LE

PE (Physical Extent): Smallest allocation unit in VG, default 4MB

LE (Logical Extent): Smallest unit in LV, maps to PE in VG

LV size = LE count × PE size.

3.3 LVM Metadata

Each VG/LV/PV has metadata stored in /etc/lvm/backup/ (latest) and /etc/lvm/archive/ (historical).

3.4 LVM Toolchain

Commands grouped by object: pvcreate, pvs, pvdisplay, pvscan, pvremove, pvmove;

vgcreate, vgs, vgdisplay, vgscan, vgremove, vgextend, vgreduce, vgcfgbackup, vgcfgrestore

;

lvcreate, lvs, lvdisplay, lvscan, lvremove, lvextend, lvreduce, lvrename, lvresize, lvconvert

; lvchange, vgchange, pvchange.

3.5 LVM with RAID

RAID + LVM: Create RAID (mdadm/hardware) first, then LVM on top

LVM native RAID:

lvcreate --type raid1/raid5/raid6

3.6 LVM Snapshots

lvcreate -s -L 10G -n snap_lv /dev/vg0/lv0

. Snapshots use COW (Copy-On-Write), suitable for backups.

3.7 LVM Cache

Use SSD to cache HDD: lvcreate --type cache / lvconvert --type cache.

3.8 LVM Thin Provisioning

lvcreate --type thin

. Allocates space on demand, ideal for many small LVs (e.g., KVM).

3.9 LVM with Filesystems

ext4: Supports both expansion and shrinking

xfs: Only supports expansion, not shrinking

btrfs: Full support

zfs: Doesn't use LVM, uses zpool directly

4. Overall Implementation Flow

Complete expansion path:

1. Check status: df -h, lsblk, pvs, vgs, lvs
2. Add disk/partition: fdisk/parted prepare new PV
3. Create PV: pvcreate /dev/sdX
4. Extend VG: vgextend vg0 /dev/sdX
5. Extend LV: lvextend -L +100G /dev/vg0/lv0
6. Grow FS: xfs_growfs /mount/point or resize2fs /dev/vg0/lv0
7. Verify: df -h
8. Backup metadata: vgcfgbackup

Each step has a rollback plan; never run blindly.

5. Practical Scenarios

5.1 Scenario: /data Full, Need Expansion

5.1.1 Symptom

df -h
Filesystem           Size  Used Avail Use% Mounted on
/dev/mapper/centos-root 50G  48G  2.0G  96% /
/dev/mapper/vg0-data  100G  95G  5.0G  95% /data

Judgment: /data at 95%, needs expansion.

5.1.2 Check LVM Status

pvs
vgs
lvs

Expected output shows vg0 has 100G free.

5.1.3 Case 1: VG Has Free Space

Extend LV directly:

lvextend -L +50G /dev/vg0/data
# or use all free space
lvextend -l +100%FREE /dev/vg0/data

Then grow filesystem:

# XFS
xfs_growfs /data
# ext4
resize2fs /dev/vg0/data

5.1.4 Verify

df -h /data
# Expected: /dev/mapper/vg0-data 150G 95G 55G 63% /data

5.1.5 Backup Metadata

vgcfgbackup vg0

5.1.6 Case 2: VG Full, Add New Disk

Add disk /dev/sdc, rescan: echo 1 > /sys/class/scsi_host/host0/scan; lsblk Create partition (optional):

parted /dev/sdc mklabel gpt; parted /dev/sdc mkpart primary 0% 100%

or fdisk with type 8e (Linux LVM)

Create PV: pvcreate /dev/sdc1; pvs Extend VG: vgextend vg0 /dev/sdc1; vgs Extend LV + grow FS:

lvextend -L +200G /dev/vg0/data; xfs_growfs /data; df -h /data

Backup metadata:

vgcfgbackup vg0

5.1.7 Case 3: Move Free Space from Another VG

Only LVs within same VG can be adjusted directly. Across VGs use pvmove:

pvmove -n /dev/vg0/data /dev/sdc1
# or move entire PV
pvmove /dev/sdc1 /dev/sdd1

Note: pvmove is I/O intensive, run during low peak.

5.1.8 Case 4: Root Partition Full

Root expansion requires rescue mode (LiveCD/single user):

Rescue mode

Activate LVM: vgchange -ay Extend LV

Grow FS

Reboot

Or use lvextend --resizefs -L +10G /dev/centos/root (RHEL/CentOS 7+).

5.2 Scenario: Shrinking (Dangerous)

Only ext4 supports shrinking; xfs does not.

5.2.1 Backup Data

cp -a /data /backup/data.$(date +%F)
# or
rsync -a /data/ /backup/data/

5.2.2 Unmount FS

umount /data

5.2.3 Check Filesystem

e2fsck -f /dev/vg0/data

5.2.4 Shrink Filesystem

resize2fs /dev/vg0/data 80G

5.2.5 Shrink LV

lvreduce -L 80G /dev/vg0/data

Judgment: lvreduce size must be ≥ FS size.

5.2.6 Mount & Verify

mount /data
df -h /data

5.2.7 Shrinking XFS (Not Supported)

For xfs, must: backup → unmount → lvremove → lvcreate smaller → mkfs.xfs → mount → restore data. Xfs shrinking is effectively "recreate".

5.3 Scenario: Migrate Data to New Disk

Add new disk: pvcreate /dev/sdd1; vgextend vg0 /dev/sdd1 Migrate: pvmove /dev/sdc1 /dev/sdd1 (in-place, no downtime)

Remove old PV: vgreduce vg0 /dev/sdc1; pvremove /dev/sdc1 Verify:

pvs; df -h /data

5.4 Scenario: LVM Snapshot Backup

Create snapshot: lvcreate -s -L 10G -n data_snap /dev/vg0/data (size 10-20% of data change rate)

Mount snapshot: mkdir /mnt/snap; mount /dev/vg0/data_snap /mnt/snap Backup: rsync -a /mnt/snap/ /backup/data/$(date +%F)/ or tar czf /backup/data.$(date +%F).tgz -C /mnt/snap . Cleanup: umount /mnt/snap; lvremove /dev/vg0/data_snap Snapshot space grows with data changes; exceeds allocated size → snapshot invalidates.

5.4.5 Auto Snapshot Script

#!/bin/bash
set -euo pipefail
DATE=$(date +%F_%H%M%S)
LV_PATH=/dev/vg0/data
SNAP_SIZE=10G
SNAP_NAME=data_snap_$DATE
BACKUP_DIR=/backup/data/$DATE

lvcreate -s -L $SNAP_SIZE -n $SNAP_NAME $LV_PATH
mkdir -p /mnt/snap
mount /dev/vg0/$SNAP_NAME /mnt/snap
mkdir -p $BACKUP_DIR
rsync -a /mnt/snap/ $BACKUP_DIR/
umount /mnt/snap
lvremove -f /dev/vg0/$SNAP_NAME
echo "snapshot $SNAP_NAME backed up to $BACKUP_DIR"

5.5 Scenario: LVM Cache (SSD Accelerate HDD)

Create cache pool:

lvcreate -L 100G -n cache_meta vg0 /dev/sdc1; lvcreate -L 1T -n cache_data vg0 /dev/sdc1

Convert LV to cache: lvconvert --type cache --cachepool vg0/cache_data vg0/data Verify: lvs -o +cache_total_blocks Split cache:

lvconvert --splitcache vg0/data

5.6 Scenario: LVM Thin Provisioning

Create thin pool: lvcreate -L 100G --thinpool thin_pool vg0 Create thin LV: lvcreate -V 1T --thin -n thin_lv vg0/thin_pool (actual usage grows, max 1T)

5.7 Scenario: Replace Failed Disk

Check PV: pvdisplay /dev/sdc1 (shows missing)

Remove missing: vgreduce --removemissing vg0 --force Replace:

pvcreate /dev/sdd1; vgextend vg0 /dev/sdd1

5.8 Scenario: LVM Metadata Corruption Recovery

Backup: cp /etc/lvm/backup/vg0 /etc/lvm/backup/vg0.bak Restore: vgcfgrestore vg0 Force activate:

vgchange -ay vg0

5.9 Scenario: LVM with Multipath

In multipath, LVM may see duplicate PVs. Configure filter in /etc/lvm/lvm.conf: filter = [ "a|/dev/mapper/mpath.*|", "r/.*/" ] Whitelist multipath devices, reject others.

5.10 Scenario: LVM on Cloud Servers

Cloud servers (AWS, Azure, Aliyun) typically don't use LVM on root, but can use LVM on data disks. Operations identical to local.

6. Common Commands Reference

6.1 PV Commands

pvcreate /dev/sdc1          # Create PV
pvs                         # List PVs
pvdisplay                   # Detailed PV
pvscan                      # Scan PVs
pvremove /dev/sdc1          # Remove PV
pvmove /dev/sdc1            # Move PV data

6.2 VG Commands

vgcreate vg0 /dev/sdc1      # Create VG
vgs                         # List VGs
vgdisplay vg0               # Detailed VG
vgscan                      # Scan VGs
vgextend vg0 /dev/sdd1      # Extend VG
vgreduce vg0 /dev/sdc1      # Reduce VG
vgremove vg0                # Remove VG
vgcfgbackup vg0             # Backup metadata
vgcfgrestore vg0            # Restore metadata
vgchange -ay vg0            # Activate
vgchange -an vg0            # Deactivate

6.3 LV Commands

lvcreate -L 10G -n data vg0          # Create LV
lvs                                  # List LVs
lvdisplay /dev/vg0/data              # Detailed LV
lvscan                               # Scan LVs
lvextend -L +10G /dev/vg0/data       # Extend LV
lvextend -l +100%FREE /dev/vg0/data  # Use all VG free
lvreduce -L 10G /dev/vg0/data        # Reduce LV
lvremove /dev/vg0/data               # Remove LV
lvrename vg0 data data_new           # Rename LV
lvresize -L 20G /dev/vg0/data        # Resize LV

6.4 Filesystem Commands

# ext4
resize2fs /dev/vg0/data
e2fsck -f /dev/vg0/data
mkfs.ext4 /dev/vg0/data

# xfs
xfs_growfs /data
xfs_info /data
mkfs.xfs /dev/vg0/data

# Common
blkid /dev/vg0/data
fsck /dev/vg0/data

6.5 View LVM Details

pvs -o +pv_used,pv_free
vgs -o +vg_mda_count,vg_mda_size,vg_mda_free
lvs -o +devices  # Shows which PVs compose LV

6.6 View Filesystem Info

df -h /data
df -i /data
mount | grep /data
findmnt /data

6.7 View Disk Partitions

lsblk
fdisk -l /dev/sdc
parted /dev/sdc print

6.8 View LVM History

ls /etc/lvm/archive/
cat /etc/lvm/archive/vg0_00001.vg

6.9 LVM Auto Activation

vgchange -ay
lvchange -ay /dev/vg0/data

6.10 LVM Locking

LVM uses file locks in /run/lock/lvm/ to prevent concurrent operations.

7. Configuration Examples

7.1 /etc/lvm/lvm.conf Key Settings

locking_dir = "/run/lock/lvm"
backup_dir = "/etc/lvm/backup"
archive_dir = "/etc/lvm/archive"
use_lvmetad = 1
obtain_device_list_from_udev = 1

# Multipath filter
filter = [ "a|/dev/mapper/mpath.*|", "r/.*/" ]

# Cache settings
max_lv = 256
max_pv = 1024

# Lock type
locking_type = 1

7.2 Thin Pool Profile

activation {
    thin_pools {
        vg0/thin_pool {
            activation_mode = "deferred"
            monitor = "y"
        }
    }
}

7.3 Auto Activation Volume List

auto_activation_volume_list = [ "vg0/data", "centos/root", "centos/swap" ]

7.4 LVM Monitoring Script

#!/bin/bash
# lvm-monitor.sh
echo "=== PV ==="
pvs --noheadings | awk '{print $1, $2, $6}'

echo "=== VG ==="
vgs --noheadings | awk '{print $1, $6, $7}'

echo "=== LV ==="
lvs --noheadings | awk '{print $1, $2, $4, $5}'

echo "=== FS ==="
df -h | grep -E '/dev/mapper'

7.5 LVM with fstab

/dev/mapper/vg0-data /data xfs defaults,noatime 0 2

Better use UUID:

blkid /dev/vg0/data

7.6 LVM with systemd Mount

# /etc/systemd/system/data.mount
[Unit]
Description=Data mount
Before=local-fs.target

[Mount]
What=/dev/vg0/data
Where=/data
Type=xfs
Options=defaults,noatime

[Install]
WantedBy=multi-fs.target

7.7 LVM with Multipath

# /etc/multipath.conf
blacklist {
    devnode "^sda"
}

multipaths {
    multipath {
        wwid "3600508..."
        alias "mpath0"
    }
}

7.8 Detailed lvm.conf

# Metadata backup
backup {
    retain_min = 50
    retain_days = 30
}

# Global filter
devices {
    filter = [ "a|/dev/sd.*|", "r|/dev/cdrom|" ]
    cache_dir = "/etc/lvm/cache"
}

# Logging
log {
    level = 1
    file = "/var/log/lvm2.log"
}

7.9 LVM with udev

# /etc/udev/rules.d/99-lvm.rules
KERNEL=="dm-*", SUBSYSTEM=="block", RUN+="/usr/sbin/lvm vgchange -ay --monitor y vg0"

7.10 Auto Expansion Script

#!/bin/bash
# lvm-autoextend.sh
set -euo pipefail

VG=$1
LV=$2
FS_PATH=$3
INCREMENT=10G

if [ "$(vgs --noheadings -o vg_free $VG | tr -d ' ')" != "0" ]; then
    lvextend -L +$INCREMENT /dev/$VG/$LV
    if mount | grep $FS_PATH | grep -q xfs; then
        xfs_growfs $FS_PATH
    else
        resize2fs /dev/$VG/$LV
    fi
fi

8. Log & Metric Observation

8.1 LVM Logs

tail -f /var/log/lvm2.log

All LVM operations logged.

8.2 dmesg

dmesg | grep -i lvm
dmesg | grep -i dm-

8.3 System Logs

journalctl -u lvm2-monitor

8.4 Key Metrics

VG free space: avoid zero free

LV data%: thin pool usage

Snapshot usage: snapshot consumption

IO util: LV device I/O

8.5 Monitoring Script

#!/bin/bash
VG=vg0
LV=data

vgfree=$(vgs --noheadings -o vg_free --units g $VG | tr -d ' g')
lvsize=$(lvs --noheadings -o lv_size --units g $VG/$LV | tr -d ' g')

if (( $(echo "$vgfree < 10" | bc -l) )); then
    echo "WARN: VG $VG free space < 10G"
fi

if (( $(echo "$lvsize > 900" | bc -l) )); then
    echo "WARN: LV $LV > 900G"
fi

8.6 Capacity Planning

Monitor trends

Preemptive expansion

Capacity alerts

Regular audits

8.7 Snapshot Status

lvs -o +snap_percent

snap_percent 100% → snapshot invalid.

8.8 Thin Pool

lvs -o +data_percent

8.9 PV Status

pvs -o +pv_attr

attr 2nd char: z=normal, m=missing.

8.10 LV Device Mapping

dmsetup info /dev/vg0/data
dmsetup ls
dmsetup table /dev/vg0/data

9. Troubleshooting Paths

9.1 "LVM Not Found"

vgscan; pvscan; lvscan

9.2 "PV Status Missing"

pvdisplay; vgreduce --removemissing vg0 --force

9.3 "LV Cannot Mount"

lvdisplay /dev/vg0/data; fsck /dev/vg0/data

9.4 "Extend LV Failed"

vgs  # Check VFree
pvs  # Check PVs

Usually VG has no free space.

9.5 "xfs_growfs Failed"

df -h /data
mount | grep /data

xfs_growfs requires mounted FS; ext4 uses resize2fs.

9.6 "resize2fs Failed"

e2fsck -f /dev/vg0/data

Must e2fsck before resize2fs.

9.7 "VG Locked"

ps aux | grep lvm
fuser /run/lock/lvm

Find locking process.

9.8 "LVM Metadata Inconsistent"

vgcfgrestore vg0

9.9 "Multi-host VG Conflict"

vgchange -an vg0

Only cluster mode has multi-host issues.

9.10 "LVM Performance Poor"

iostat -xz 1 5

May need pvmove to rebalance data distribution.

10. Risk Warnings

lvremove /dev/vg0/data

: Deletes LV, all data lost vgremove vg0: Deletes entire VG, all LVs lost lvreduce: Shrink LV, data loss risk pvmove: Production I/O impact resize2fs: Shrink fails if space insufficient xfs_growfs: Must be mounted xfs shrink: Not supported by filesystem vgcfgbackup not done: Disaster recovery difficult vgextend wrong disk: Abnormal data distribution multipath filter missing: May detect wrong devices pvs shows missing: Physical disk may have issues

Cross-host LVM sharing: Requires cluster mode

LVM encryption: dm-crypt at LV layer

LVM + hardware RAID: Layer order matters pvmove interrupted: Recoverable but careful lvcreate without VG: May pick wrong default VG lvextend -l +100%FREE: May exceed expectation lvreduce > FS size: Filesystem corruption

LVM metadata corruption: Must restore from backup vgmerge / vgsplit: Complex operations

Cross-host pvmove: Requires cluster

11. Verification Methods

pvs

: Expected PVs listed vgs: Expected VG and free space lvs: Expected LVs listed df -h /data: Size matches expectation mount: Mount point correct

Application read/write test

LVM logs clean

dmesg no errors

Monitor capacity trends

Regular inspections

12. Rollback Plans

12.1 lvextend Rollback

umount /data
e2fsck -f /dev/vg0/data
resize2fs /dev/vg0/data 100G
lvreduce -L 100G /dev/vg0/data
mount /data

12.2 lvreduce Rollback

lvextend -L 200G /dev/vg0/data
resize2fs /dev/vg0/data

Must backup and fsck before lvreduce.

12.3 vgextend Rollback

vgreduce vg0 /dev/sdc1
pvremove /dev/sdc1

12.4 vgreduce Rollback

vgextend vg0 /dev/sdc1

12.5 pvremove Rollback

pvcreate /dev/sdc1
vgextend vg0 /dev/sdc1
pvmove

Restore metadata first, then data.

12.6 Snapshot Deletion Rollback

Cannot recover deleted snapshot; must recreate.

12.7 lvcreate Rollback

lvremove /dev/vg0/data

12.8 lvremove Rollback (if LV still exists)

lvcreate -L 100G -n data vg0
xfs_growfs /data  # No mkfs needed

After lvremove, only snapshot/backup can recover.

12.9 VG Deletion Rollback

vgcfgrestore vg0
vgchange -ay vg0

12.10 Metadata Corruption Rollback

cp /etc/lvm/backup/vg0.bak /etc/lvm/backup/vg0
vgcfgrestore vg0

13. Production Environment Notes

Operation window: Expand during low traffic

Canary: Verify on 1 server first

Backup: Metadata backup before every change

Documentation: Change record for each expansion

Monitoring: Capacity alerts in monitoring

Drills: Expansion drills

Tools: Ansible for uniform push

Images: Bake partition/LVM config in install images

Emergency: Keep spare disks

Notification: Inform team of expansion

Capacity: Regular capacity planning

Performance: Monitor I/O impact

Safety: Avoid manual reboot during pvmove

Data: Avoid massive writes during expansion

Verification: Test immediately after expansion

Documentation: Maintain LVM topology diagrams

Inspection: Weekly VG free check

Alerting: Alert when free < 20%

Automation: Automated inspections

Backup: Regular metadata export

Disaster recovery: LVM topology backup

14. Summary

LVM is not magic: three-layer structure + command chains handle it:

PV: Create with pvcreate

VG: Create with vgcreate, extend with vgextend

LV: Create with lvcreate, extend with lvextend

FS: xfs_growfs (XFS) or resize2fs (ext4)

90% of expansions: vgextend + lvextend + xfs_growfs.

Shrinking: Avoid unless necessary. XFS shrink = "recreate".

Data migration: pvmove is in-place, zero downtime.

Snapshots: COW-based, backup weapon.

Metadata: Regular vgcfgbackup, restore with vgcfgrestore.

Bookmark this; follow chapters during LVM incidents — faster than Googling.

15. Advanced LVM

15.1 LVM RAID

lvcreate --type raid1 -L 10G -n lv_raid1 vg0

LVM native software RAID, combinable with hardware RAID.

15.2 Thin Pool Monitoring

lvs -o +data_percent

Expand thin pool when data_percent nears 100%.

15.3 LVM Auto Activation

vgchange --monitor y vg0

Enables D-Bus monitoring for auto-activation.

15.4 LVM with ZFS

Not recommended; layering complexity.

15.5 LVM Encryption

cryptsetup luksFormat /dev/sdc1
cryptsetup open /dev/sdc1 sdc1_crypt
pvcreate /dev/mapper/sdc1_crypt

LUKS + LVM.

15.6 LVM I/O Throttling

dmsetup threshold /dev/vg0/data 0 100

Throttle I/O.

15.7 LVM with systemd

systemctl status lvm2-monitor

15.8 LVM Monitoring Integration (Prometheus)

- alert: VGAlmostFull
  expr: (node_lvm_vg_free_bytes / node_lvm_vg_size_bytes) < 0.1
  for: 30m
  annotations:
    summary: "VG free space < 10 percent"

- alert: ThinPoolHighUsage
  expr: node_lvm_lv_data_percent > 80
  for: 30m

15.9 LVM with Backup

LVM snapshots are standard for XFS/ext4 physical backups.

15.10 LVM with Cloud

Cloud disks via multipath can join LVM.

16. Case Study: Complete Expansion Postmortem

16.1 Background

Database server /data grew from 500G to 480G (96%), alert fired.

16.2 Current State

df -h /data
pvs; vgs; lvs

vg0 VFree=200G, can extend LV directly.

16.3 Backup Metadata

vgcfgbackup vg0
cp /etc/lvm/backup/vg0 /backup/lvm-backup-vg0-$(date +%F)

16.4 Backup Data

mysqldump --all-databases > /backup/db-$(date +%F).sql

16.5 Extend LV

lvextend -l +100%FREE /dev/vg0/data

16.6 Grow Filesystem

xfs_growfs /data

16.7 Verify

df -h /data

16.8 Application Verification

mysql -e "CREATE DATABASE test_lvm_expand;"
mysql -e "USE test_lvm_expand; CREATE TABLE t (id INT);"
mysql -e "DROP DATABASE test_lvm_expand;"

16.9 Monitoring

Confirm monitoring normal, alerts cleared.

16.10 Retrospective

Followed standard expansion process

Backup, extend, verify, monitor all done

Consider auto-expansion next

17. Operations Handbook: LVM Inspection Checklist

Run weekly:

#!/bin/bash
# lvm-audit.sh
echo "=== PV ==="
pvs --noheadings -o pv_name,vg_name,pv_size,pv_free,pv_attr

echo "=== VG ==="
vgs --noheadings -o vg_name,vg_size,vg_free,vg_attr

echo "=== LV ==="
lvs --noheadings -o lv_name,vg_name,lv_size,lv_attr

echo "=== FS ==="
df -h | grep -E '/dev/mapper'

echo "=== LVM metadata backup ==="
ls -lt /etc/lvm/backup/ | head

echo "=== Multipath ==="
multipath -ll

echo "=== Filter ==="
grep -E '^[[:space:]]*filter' /etc/lvm/lvm.conf

Integrate script into monitoring.

18. Final Words

LVM is ops fundamentals. Master three-layer structure, command chains, rollback plans, metadata backup. Practice commands here; follow chapters during incidents — more efficient than ad-hoc searches.

Remember: Expansion has risk; always backup metadata first. Never blindly run lvreduce, lvremove in production.

19. Deep Dive: LVM Internals

19.1 Physical Structure

Metadata written at start of each PV with UUID. VG metadata replicated across all PVs for redundancy.

19.2 LVM & device-mapper

LVM built on Linux device-mapper; each LV is a dm device:

ls -la /dev/mapper/
ls -la /dev/vg0/
ls -la /dev/dm-*

dm-0, dm-1 are device-mapper devices.

19.3 Allocation Policies

contiguous: Contiguous (good perf, hard to allocate)

normal: Default, spread across PVs

cling: Prefer same PV (snapshot scenarios)

anywhere: Free distribution

19.4 Locking Mechanism

locking_type = 1  # Local
locking_type = 2  # Cluster (dlm)

19.5 Activation Modes

active: Online

suspended: Suspended (snapshot merge)

partial: Some PVs missing

inactive: Not activated

lvchange -ay /dev/vg0/data
lvchange -an /dev/vg0/data
lvchange --refresh /dev/vg0/data

19.6 PE Size Adjustment

vgchange -s 8M vg0

PE size affects max LV size (65536 PE limit): PE=4M → 256G max; PE=8M → 512G; PE=16M → 1T; PE=64M → 4T.

19.7 Performance Tuning

Place hot data LVs on SSD PVs

Use pvmove to reorganize data

Adjust readahead

Use lvcreate --type raid1 + SSD for mirroring

19.8 LVM & I/O Schedulers

bfq: Desktop, mixed workloads

deadline: Databases

none: NVMe

echo none > /sys/block/dm-0/queue/scheduler

19.9 Integrity Check

vgck vg0
lvck /dev/vg0/data

19.10 Version Compatibility

LVM2 compatible with LVM1 format; advanced features need newer versions. RHEL/CentOS 7+ default LVM2.

20. LVM with Databases

20.1 MySQL Data Dir on LVM

lvcreate -L 500G -n mysql_data vg0
mkfs.xfs /dev/vg0/mysql_data
mount /dev/vg0/mysql_data /var/lib/mysql

Isolate DB on dedicated LV for manageability.

20.2 Snapshot Backup MySQL

lvcreate -s -L 50G -n mysql_snap /dev/vg0/mysql_data
mount /dev/vg0/mysql_snap /mnt/snap
rsync -a /mnt/snap/ /backup/mysql/
umount /mnt/snap
lvremove -f /dev/vg0/mysql_snap

Snapshot backup more complete than mysqldump.

20.3 Redis on LVM

lvcreate -L 100G -n redis_data vg0
mkfs.xfs /dev/vg0/redis_data
mount /dev/vg0/redis_data /var/lib/redis

20.4 PostgreSQL Tablespaces with LVM

lvcreate -L 200G -n pg_index vg0
mkdir /var/lib/pgsql/index
mount /dev/vg0/pg_index /var/lib/pgsql/index

CREATE TABLESPACE ts_index LOCATION '/var/lib/pgsql/index';
CREATE INDEX idx_user ON users(id) TABLESPACE ts_index;

20.5 MongoDB with LVM

lvcreate -L 1T -n mongo_data vg0
mkfs.xfs /dev/vg0/mongo_data
mount /dev/vg0/mongo_data /var/lib/mongo

20.6 Elasticsearch with LVM

lvcreate -L 5T -n es_data vg0
lvcreate -L 1T -n es_logs vg0

Separate data and logs LVs.

20.7 Kafka with LVM

lvcreate -L 10T -n kafka_data vg0
mkfs.xfs /dev/vg0/kafka_data

20.8 Database LVM Backup Strategy

Snapshot backup: Second-level, COW space

LVM mirror: Real-time replica

LVM RAID: Disk failure protection

LVM thin: Space efficiency

20.9 Database LVM Recovery

lvconvert --merge /dev/vg0/mysql_snap

Merge auto-restores.

20.10 Database LVM Monitoring

VG free space

LV usage

Snapshot consumption

Thin pool data_percent

21. LVM with Containers

21.1 Container Data Dir on LVM

lvcreate -L 1T -n docker_data vg0
mkfs.xfs /dev/vg0/docker_data
mount /dev/vg0/docker_data /var/lib/docker

21.2 Container Images on LVM

lvcreate -L 500G -n docker_imagedata vg0
mount /dev/vg0/docker_imagedata /var/lib/docker/image

21.3 Kubernetes Data on LVM

lvcreate -L 5T -n k8s_data vg0
mount /dev/vg0/k8s_data /var/lib/kubelet

21.4 Container LVM Snapshot Backup

lvcreate -s -L 100G -n docker_snap /dev/vg0/docker_data

21.5 Container Thin Pool

lvcreate -L 100G --thinpool docker_thin vg0
lvcreate -V 1T --thin -n container1 docker_thin

Thin LVs suit containers: each container gets independent LV.

21.6 Thin Pool Monitoring

lvs -o +data_percent docker_thin

21.7 Container LVM Expansion

lvextend -L +100G vg0/docker_thin

21.8 Container LVM with Overlay

mkfs.xfs /dev/vg0/container_overlay
mount /dev/vg0/container_overlay /var/lib/docker/overlay2

21.9 Container LVM Performance

If container I/O jitter, move LV to SSD.

21.10 Container LVM with NFS

mount -t nfs server:/data /var/lib/docker

NFS and LVM are alternatives.

22. LVM Across Distributions

22.1 RHEL/CentOS

Default install uses LVM

Install lvm2 via dnf

Configure /etc/lvm/lvm.conf

22.2 Ubuntu/Debian

Desktop default no LVM

Server can configure manually

Install lvm2 via apt

22.3 SUSE

Default LVM

Configure via yast

22.4 Aliyun Anolis OS

CentOS compatible

Same as RHEL

22.5 Domestic OS

Kylin, UOS default support LVM

22.6 Kernel Requirements

LVM supported since Linux 2.4; all modern distros work.

22.7 udev Integration

udevadm trigger

22.8 systemd Integration

systemctl status lvm2-monitor

22.9 Auto Activation Config

auto_activation_volume_list = [ "vg0/data", "centos/root" ]

Whitelist auto-activated LVs; avoid activating encrypted disks.

22.10 Cross-Distro Migration

LVM metadata compatible across distros. Use vgexport/vgimport:

vgexport vg0
vgimport vg0

23. LVM Monitoring & Alerting

23.1 Key Metrics

VG free: Avoid zero

LV data%: Thin pool usage

Snapshot percent: Snapshot consumption

IO util: LV device I/O

pvmove status: Migration progress

23.2 Prometheus Integration

- alert: VGLowSpace
  expr: (node_lvm_vg_free_bytes / node_lvm_vg_size_bytes) < 0.1
  for: 30m
  annotations:
    summary: "VG free space < 10 percent"

- alert: ThinPoolHighUsage
  expr: node_lvm_lv_data_percent > 80
  for: 30m

23.3 Custom Monitoring Script (Python)

import subprocess
import json

def parse_vgs():
    result = subprocess.run(['vgs', '--reportformat', 'json', '-o', 'vg_name,vg_free,vg_size'],
                            capture_output=True, text=True)
    return json.loads(result.stdout)

vgs = parse_vgs()
for vg in vgs['report'][0]['vg']:
    free = int(vg['vg_free'].replace('B', ''))
    size = int(vg['vg_size'].replace('B', ''))
    percent = (1 - free / size) * 100
    print(f"VG {vg['vg_name']}: {percent:.1f}% used")

23.4 Alert Channels

DingTalk bot

Feishu bot

WeCom

Email

SMS

23.5 Alert Tiers

Critical: VG free < 5%

Warning: VG free < 20%

Info: VG free < 30%

23.6 Inspection Automation

Integrate LVM inspection via: cron, systemd timer, Ansible playbook, SaltStack state.

23.7 Capacity Planning

Historical trend analysis

Business growth forecasting

Preemptive expansion

Capacity audits

23.8 Performance Monitoring

LV IO util

PV IO util

pvmove progress

Snapshot I/O impact

23.9 Alert Correlation

Correlate multiple LVM alerts to single root cause.

23.10 Alert Drills

Monthly alert drills.

24. LVM with Backup

24.1 LVM Snapshot Backup

lvcreate -s -L 10G -n data_snap /dev/vg0/data
mount /dev/vg0/data_snap /mnt/snap
rsync -a /mnt/snap/ /backup/
umount /mnt/snap
lvremove /dev/vg0/data_snap

24.2 LVM with BorgBackup

lvcreate -s -L 10G -n data_snap /dev/vg0/data
mount /dev/vg0/data_snap /mnt/snap
borg create /backup/repo::snapshot /mnt/snap
umount /mnt/snap
lvremove /dev/vg0/data_snap

24.3 LVM with tar

lvcreate -s -L 10G -n data_snap /dev/vg0/data
mount /dev/vg0/data_snap /mnt/snap
tar czf /backup/data.tgz -C /mnt/snap .
umount /mnt/snap
lvremove /dev/vg0/data_snap

24.4 LVM with rsync

lvcreate -s -L 10G -n data_snap /dev/vg0/data
mount /dev/vg0/data_snap /mnt/snap
rsync -a /mnt/snap/ /backup/data/
umount /mnt/snap
lvremove /dev/vg0/data_snap

24.5 LVM with ZFS

Not recommended; layering complexity.

24.6 LVM with mysqldump

mysqldump = logical backup; LVM snapshot = physical backup. Combine: mysqldump for schema/data, LVM snapshot for binlogs.

24.7 LVM with btrfs

btrfs has native snapshots; avoid stacking.

24.8 LVM with xfs

xfs has xfsdump:

lvcreate -s -L 10G -n data_snap /dev/vg0/data
mount /dev/vg0/data_snap /mnt/snap
xfsdump -f /backup/data.dump /mnt/snap
umount /mnt/snap
lvremove /dev/vg0/data_snap

24.9 LVM with Restic

lvcreate -s -L 10G -n data_snap /dev/vg0/data
mount /dev/vg0/data_snap /mnt/snap
restic backup /mnt/snap
umount /mnt/snap
lvremove /dev/vg0/data_snap

24.10 Backup Strategy

Daily snapshot backup

Weekly full backup

Monthly archive

Regular backup verification

Offsite backup storage

25. LVM with High Availability

25.1 LVM with Pacemaker

pcs resource create lv_data ocf:heartbeat:LVM volgrpname=vg0 exclusive=true
pcs resource create fs_data ocf:heartbeat:Filesystem device=/dev/vg0/data directory=/data fstype=xfs

25.2 LVM with DRBD

lvcreate -L 100G -n drbd_data vg0
drbdmanage add-volume drbd_data 100G

Classic DRBD + LVM combo.

25.3 LVM with iSCSI

lvcreate -L 500G -n iscsi_lun vg0
targetcli /backstores/block/ create name=lun1 dev=/dev/vg0/iscsi_lun

25.4 LVM with Cluster

Cluster LVM uses dlm:

lvmconf --enable-cluster

25.5 LVM Mirror

lvcreate --type mirror -L 100G -m 1 -n data vg0

LVM native mirroring, no hardware RAID needed.

25.6 LVM with GlusterFS

lvcreate -L 10T -n gluster_brick vg0
mkfs.xfs /dev/vg0/gluster_brick
mount /dev/vg0/gluster_brick /var/lib/glusterfs/brick

25.7 LVM with Ceph

lvcreate -L 10T -n ceph_osd vg0

25.8 LVM with HDFS

lvcreate -L 100T -n hdfs_data vg0

25.9 LVM with S3-Compatible Storage

lvcreate -L 50T -n minio_data vg0
mount /dev/vg0/minio_data /var/lib/minio

25.10 LVM with Cloud Native

CSI driver supports LVM provisioner:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: lvm-sc
provisioner: lvmplugin.example.com
parameters:
  vgName: vg0

26. LVM Limits & Boundaries

26.1 LV Count Limit

65536 PE per LV max:

PE=4M: Max LV = 256G

PE=8M: Max LV = 512G

PE=16M: Max LV = 1T

26.2 VG Count Limit

Theoretically limited by total LV count.

26.3 Snapshot Limits

COW table size fixed at creation

Exceeds → snapshot invalidates

Performance impacts origin LV

26.4 Thin Pool Limits

Writes fail when exceeding thin pool size

Metadata exhaustion also causes failure

26.5 Cache Limits

Cache pool size affects hit rate

Write-through impacts performance

26.6 RAID Limits

LVM RAID5/6 write performance poor

Rebuild time long

26.7 Filesystem Limits

ext4 single file max 16TB

xfs single file max 8EB

xfs no shrink support

26.8 Backup Limits

Snapshot ≠ backup

Must store on independent media

26.9 Performance Limits

Software I/O stack overhead

Multi-layer abstraction cost

26.10 Migration Limits

Cross-architecture migration hard

Cross-kernel version compatibility

27. LVM & Kernel I/O Stack

27.1 I/O Stack Layers

App fsync → VFS → Filesystem → LVM (device-mapper) → MD/hardware RAID → Driver → Hardware.

27.2 Per-Layer Overhead

VFS: 1-5%

FS: 5-10%

DM: 1-3%

MD: 5-15%

Driver: 1-5%

LVM single layer typically <5% overhead.

27.3 Bypass LVM Scenarios

Direct I/O: io_uring

DPDK userspace I/O

SPDK kernel bypass

High-perf scenarios can bypass LVM.

27.4 LVM & io_uring

LVM supports io_uring, better than traditional I/O.

27.5 LVM & NVMe-oF

nvme connect -t rdma -n subsystem -a ip -s 4420
pvcreate /dev/nvme0n1

27.6 LVM & Multi-queue

cat /sys/block/dm-0/queue/nr_requests

dm multi-queue enabled by default.

27.7 LVM & cgroup

cat /sys/fs/cgroup/blkio/system.slice/lvcheck.service/blkio.throttle.read_bps_device

cgroup can limit LV I/O.

27.8 LVM & NUMA

numactl --cpunodebind=0 --membind=0 dd if=/dev/zero of=/dev/vg0/test bs=1M count=1000

Cross-socket LV access slower.

27.9 LVM & Transparent Huge Pages

cat /sys/kernel/mm/transparent_hugepage/enabled

Databases: set to never.

27.10 LVM & hugeTLB

mount -t hugetlbfs hugetlbfs /mnt/huge

Huge pages suit large-memory apps.

28. LVM Quick Reference

28.1 Expansion (Standard)

df -h /data
vgs
lvextend -l +100%FREE /dev/vg0/data
xfs_growfs /data
df -h /data

28.2 Add Disk & Expand

pvcreate /dev/sdc1
vgextend vg0 /dev/sdc1
lvextend -l +100%FREE /dev/vg0/data
xfs_growfs /data

28.3 Shrink ext4

umount /data
e2fsck -f /dev/vg0/data
resize2fs /dev/vg0/data 80G
lvreduce -L 80G /dev/vg0/data
mount /data

28.4 Snapshot Backup

lvcreate -s -L 10G -n data_snap /dev/vg0/data
mount /dev/vg0/data_snap /mnt/snap
rsync -a /mnt/snap/ /backup/
umount /mnt/snap
lvremove /dev/vg0/data_snap

28.5 Data Migration

pvcreate /dev/sdd1
vgextend vg0 /dev/sdd1
pvmove /dev/sdc1 /dev/sdd1
vgreduce vg0 /dev/sdc1
pvremove /dev/sdc1

28.6 Delete LV

umount /data
lvremove /dev/vg0/data

28.7 Delete VG

vgremove vg0

28.8 Delete PV

vgreduce vg0 /dev/sdc1
pvremove /dev/sdc1

28.9 Restore VG Metadata

cp /etc/lvm/backup/vg0.bak /etc/lvm/backup/vg0
vgcfgrestore vg0

28.10 View LVM Topology

pvs
vgs
lvs -o +devices

29. LVM vs Other Storage Solutions

29.1 LVM vs ZFS

ZFS: Built-in RAID, snapshots, compression, checksum

LVM: Mature, stable, great ecosystem

29.2 LVM vs btrfs

btrfs: Native snapshots, subvolumes, COW

LVM: Better compatibility with existing FS

29.3 LVM vs Stratis

Stratis: RHEL 8+ next-gen

LVM: Mature solution

29.4 LVM vs Hardware RAID

Hardware RAID: Better perf, higher cost

LVM: Flexible, lower cost

29.5 LVM vs Software RAID

mdadm: Stable

LVM RAID: Integrated with LVM

29.6 LVM in Cloud

AWS: EBS + LVM

Azure: Managed Disks + LVM

Aliyun: Cloud disks + LVM

29.7 LVM with Docker Volumes

Docker volume driver supports LVM

Also supports overlay, btrfs, zfs

29.8 LVM with KVM

KVM virtual disks on LVM

LVM thin ideal for many VMs

29.9 LVM with OpenStack

Cinder supports LVM backend

Nova uses LVM for local storage

29.10 LVM Future

Future storage more cloud-native

LVM remains data center foundation

30. Article Summary

LVM is ops fundamentals. Three-layer structure + command chains + metadata backup. Practice commands here; follow chapters during incidents — more efficient than ad-hoc searches.

Key actions:

Check status: pvs / vgs / lvs / df -h

Add disk: pvcreate / vgextend

Expand: lvextend / xfs_growfs / resize2fs

Shrink: lvreduce / resize2fs (XFS not supported)

Backup metadata: vgcfgbackup

Restore metadata: vgcfgrestore

Snapshot: lvcreate -s

Migrate: pvmove

Remember: LVM not magic — three layers + command chain + backup habit.