Master Linux Virtualization: Tuning KVM Performance with Tuned and KSM

Preface

The article covers common Linux virtualization management operations and some performance tuning configurations.

Readers are invited to point out any misunderstandings.

“Do not be too obsessed with the present, nor worry too much about the future; after experiencing some things, the scenery before you is already different.” — Haruki Murakami

Using Tools for Tuning

You can directly use the tuned package parameters designed for virtualization. Tuned provides two profiles: one for the virtual guest and one for the host.

- virtual-guest               - Optimize for running inside a virtual guest<br/>- virtual-host                - Optimize for running KVM guests

virtual-guest : Optimizes the environment when running inside a VM.

virtual-host : Optimizes the host that runs KVM guests.

Below are the two strategy configurations and what they do.

The guest strategy inherits the throughput-performance profile, which provides maximum disk and network I/O throughput.

┌──[[email protected]]-[~]
└─$ cat /usr/lib/tuned/virtual-guest/tuned.conf
# tuned configuration

[main]
summary=Optimize for running inside a virtual guest
include=throughput-performance

[sysctl]
# When dirty pages reach a certain percentage, start writeback
vm.dirty_ratio = 30
# Reduce swapping tendency
vm.swappiness = 30

vm.dirty_ratio

specifies the percentage of dirty anonymous pages that triggers writeback; the default is 20%, here set to 30% to delay writeback and improve I/O performance. vm.swappiness controls how aggressively the system swaps memory; the default is 60, here reduced to 30 so the system prefers file I/O over swapping, which is safe on high‑performance storage.

The host strategy also inherits throughput-performance and adds additional settings:

┌──[[email protected]]-[~]
└─$ cat /usr/lib/tuned/virtual-host/tuned.conf
# tuned configuration

[main]
summary=Optimize for running KVM guests
include=throughput-performance

[sysctl]
# Start background writeback when dirty pages reach 5%
vm.dirty_background_ratio = 5

[cpu]
# Enable C3 state power savings
force_latency=cstate.id_no_zero:3|70

vm.dirty_background_ratio sets the threshold for background writeback (default 10%, set to 5%).

force_latency configures the CPU C3 sleep state for power saving.

Viewing Resource Usage

$ yum -y install numactl

numactl

helps manage memory and CPU allocation on NUMA systems.

$ numastat -c qemu-kvm
Per-node process memory usage (in MBs)
PID   Node  Total
1755  855   855
53693 759   759
53790 1218  1218
53890 875   875
54009 751   751
54128 811   811
54246 778   778
Total 6048  6048

Setting VM CPU Pinning

Use virsh to view and set VCPU affinity.

$ virsh vcpupin workstation
VCPU: CPU Affinity
----------------------------------
0: 0-7
1: 0-7

Pinning a specific VCPU to a single core:

$ virsh vcpupin serverc 0 0
$ virsh vcpupin serverc
VCPU: CPU Affinity
----------------------------------
0: 0
1: 0-7

Limiting VM CPU Resources

Adjust cpu_shares to control CPU priority. A VM with cpu_shares=2048 receives more CPU time than one with cpu_shares=1024.

$ virsh schedinfo workstation
Scheduler : posix
cpu_shares : 1024
...

$ virsh schedinfo workstation cpu_shares=2048
cpu_shares : 2048
...

Setting VM Memory Limits

Attempting to change the maximum memory of a running domain fails:

$ virsh setmaxmem workstation --size 2097152
error: Unable to change MaxMemorySize
error: Requested operation is not valid: cannot resize the maximum memory on an active domain

Set the running memory size (must be done while the VM is shut down): $ virsh setmem workstation --size 2097152 View current memory limits:

$ virsh memtune workstation
hard_limit : unlimited
soft_limit : unlimited
swap_hard_limit : unlimited

$ virsh memtune workstation --hard-limit 2G --soft-limit 1G
$ virsh memtune workstation
hard_limit : 2097152
soft_limit : 1048576
swap_hard_limit : unlimited

KSM Parameter Configuration

When multiple VMs run the same OS or workload, identical memory pages can be merged using Kernel Samepage Merging (KSM). Writes to shared pages trigger copy‑on‑write (COW).

$ systemctl status ksm.service
● ksm.service - Kernel Samepage Merging
   Loaded: loaded (/usr/lib/systemd/system/ksm.service; enabled)
   Active: active (exited) …

$ systemctl status ksmtuned.service
● ksmtuned.service - Kernel Samepage Merging (KSM) Tuning Daemon
   Loaded: loaded (/usr/lib/systemd/system/ksmtuned.service; enabled)
   Active: active (running) …

KSM parameters are exposed under /sys/kernel/mm/ksm/. Important knobs: run: 1 enables scanning, 0 disables it (default 0). pages_to_scan: number of pages scanned each cycle (default 100). sleep_millisecs: sleep time between cycles (default 20 ms). pages_shared, pages_sharing: total and current shared pages. full_scans: number of full memory scans performed. merge_across_nodes: whether to merge pages across NUMA nodes (default 1).

View and modify these values with virsh node-memory-tune:

$ virsh node-memory-tune
Shared memory:
        shm_pages_to_scan 100
        shm_sleep_millisecs 20
        shm_pages_shared 2013
        shm_pages_sharing 5130
        shm_pages_unshared 10144
        shm_pages_volatile 91431
        shm_full_scans 28
        shm_merge_across_nodes 1

$ virsh node-memory-tune --shm-sleep-millisecs 30
$ virsh node-memory-tune
... shm_sleep_millisecs 30 ...

Virtual Disk Configuration and Tuning

VM disks can be block devices or image files. Using a block device offers better performance; image files (especially qcow2) require more I/O resources but provide features such as snapshots, compression, and encryption.

raw – best performance.

qcow2 – supports sparse files, COW, snapshots, compression, encryption.

Creating a qcow2 Image

$ qemu-img create -f qcow2 -o preallocation=metadata /var/lib/libvirt/images/disk.qcow2 1G
Formatting '/var/lib/libvirt/images/disk.qcow2', fmt=qcow2 size=1073741824 cluster_size=65536 preallocation=metadata lazy_refcounts=off refcount_bits=16

Preallocation options: preallocation=off – no space allocated (default). preallocation=metadata – allocate metadata only. preallocation=falloc – allocate metadata and data blocks but mark them unallocated. preallocation=full – fully allocate all space.

Creating a Snapshot (ROW)

$ qemu-img create -f qcow2 -b /var/lib/libvirt/images/disk.qcow2 disk-snap.qcow2 30G
Formatting 'disk-snap.qcow2', fmt=qcow2 size=32212254720 backing_file=/var/lib/libvirt/images/disk.qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16

Virtual Disk Cache Modes

cache=none

– no caching, data written directly, supports live migration. cache=writethrough – writes go to cache and disk, ensures data integrity, slower. cache=writeback – writes cached first, flushed later, higher performance. cache=directsync – similar to writethrough, bypasses cache on demand. cache=unsafe – like writeback but ignores all flushes; highest performance but unsafe for production.

Virtual Disk I/O Tuning

Set I/O limits for a VM’s disk device:

$ virsh blkdeviotune workstation vda --total-iops-sec 1000 --total-bytes-sec 10MB

Verify the settings in the domain XML:

...<iotune>
  <total_bytes_sec>1000000</total_bytes_sec>
  <total_iops_sec>100</total_iops_sec>
</iotune>

Virtualization Performance Monitoring

Device I/O statistics:

$ virsh domblkstat workstation vda --human
Device: vda
 number of read operations:      15012
 number of bytes read:           437122048
 number of write operations:     3193
 number of bytes written:        71808512
 number of flush operations:     903
 total duration of reads (ns):   854291749703
 total duration of writes (ns):  93692774619
 total duration of flushes (ns): 7848596731

Network interface list and statistics:

$ virsh domiflist workstation
Interface  Type   Source   Model   MAC
---------------------------------------
vnet4      bridge privbr0  virtio  52:54:00:00:fa:09

$ virsh domifstat workstation vnet4
vnet4 rx_bytes 600038
vnet4 rx_packets 10176
vnet4 tx_bytes 72045
vnet4 tx_packets 610

Memory statistics (in KB):

$ virsh dommemstat workstation
actual 2097152
swap_in 0
swap_out 0
major_fault 304
minor_fault 147927
unused 1669348
available 1873292
usable 1631560
rss 1246776