Unlocking KVM & QEMU: How Linux Turns Hardware into Powerful Virtual Machines

KVM

KVM (Kernel-based Virtual Machine) is a TYPE1 hypervisor that integrates VMM and Host OS, running directly on host hardware. It relies on CPU hardware‑assisted virtualization (e.g., Intel VT‑x, AMD‑V), integrates VMM with the host OS, and offers high execution efficiency.

Depends on CPU hardware‑assisted virtualization (e.g., Intel VT‑x / AMD‑V);

VMM and Host OS are integrated;

High runtime efficiency.

Understanding KVM requires familiarity with hardware‑assisted virtualization; see the preceding reference list.

KVM is essentially a Linux kernel module (kvm.ko, kvm‑intel.ko, kvm‑amd.ko) that uses kernel services such as memory, process, and device management, adding CPU and memory virtualization to satisfy three conditions for a complete VMM:

Resource Control : VMM must manage all system resources.

Equivalence : Guest OS behavior must match host OS except for timing and hardware availability.

Efficiency : Most guest instructions should execute directly on host hardware without VMM intervention.

KVM was merged into Linux kernel 2.6.20 on 5 Feb 2007. Modern KVM supports SMP, NUMA, CPU affinity, overcommit, VirtIO, PCI pass‑through, SR‑IOV, hot‑plug, live migration, and KSM.

When the Linux kernel loads kvm.ko, it:

Initializes kvm.ko data structures;

Detects CPU architecture, enables VT‑x, executes VMXON, and runs VMM in root mode;

Creates /dev/kvm device node for user‑space applications (e.g., QEMU).

KVM runs in kernel space and cannot emulate devices itself; it relies on a user‑space application such as QEMU to provide virtual devices.

QEMU

QEMU (Quick Emulator) is an open‑source TYPE2 hypervisor released in 2001 by Fabrice Bellard. It uses dynamic binary translation to emulate many CPU architectures (x86, ARM, MIPS, SPARC, PowerPC) and most physical devices.

QEMU 4.0.0 (2019) added extensive architecture support but suffers performance penalties because it primarily uses software capture‑and‑emulation.

QEMU‑KVM

QEMU‑KVM is a QEMU branch tailored for KVM. The KVM software package includes four components: the KVM kernel module, QEMU, QEMU‑KVM, and virtio drivers. QEMU‑KVM is widely integrated into commercial products such as AWS and Alibaba Cloud.

Although later QEMU releases merged the branches, the term QEMU‑KVM remains common when discussing KVM‑specific usage.

Integrated Software Architecture

kvm.ko runs in kernel space and exposes /dev/kvm to user space, providing a libkvm library for QEMU.

QEMU opens /dev/kvm, uses ioctl() to invoke KVM interfaces for CPU, memory, and I/O virtualization, while handling VM configuration, lifecycle, and virtual devices itself.

open("/dev/kvm", O_RDWR|O_LARGEFILE) = 3
ioctl(3, KVM_GET_API_VERSION, 0) = 12
...
ioctl(3, KVM_CREATE_VM, 0) = 4
...

Creating a VM involves:

Open /dev/kvm to obtain a file descriptor (kvmfd).

kvmfd = open("/dev/kvm", O_RDWR);

Call KVM_CREATE_VM to create a VM and get vmfd.

vmfd = ioctl(kvmfd, KVM_CREATE_VM, 0);

Map host virtual/physical memory and PCI devices.

ioctl(kvmfd, KVM_SET_USER_MEMORY_REGION, &mem);

Map a QCOW2 image into the user process address space.

Create vCPU(s) and allocate vMemory according to NUMA topology.

ioctl(kvmfd, KVM_CREATE_VCPU, vcpuid);
vcpu->kvm_run_mmap_size = ioctl(kvm->dev_fd, KVM_GET_VCPU_MMAP_SIZE, 0);

Spawn user threads to run Guest OS code.

ioctl(kvm->vcpus->vcpu_fd, KVM_RUN, 0);

Main thread loops, handling VM exits (e.g., I/O, HLT).

for (;;) {
    ioctl(KVM_RUN);
    switch (exit_reason) {
        case KVM_EXIT_IO:   /* ... */
        case KVM_EXIT_HLT:  /* ... */
    }
}

CPU Virtualization Implementation

KVM provides three execution modes corresponding to Intel VT‑x privilege levels:

User Mode : QEMU user‑space code.

Kernel Mode (Root) : kvm.ko code.

Guest Mode (Non‑root) : Guest OS code.

Kernel Mode bridges User Mode and Guest Mode; it prepares vCPU context, executes VM entry, and transfers control to the guest.

Memory Virtualization Implementation

KVM exposes memory virtualization via /dev/kvm. After creating a VM, ioctl calls establish GPA‑HVA/HPA mappings using Intel EPT, ensuring isolation between guests.

I/O Virtualization Implementation

QEMU handles device emulation. When a guest issues an I/O request, the VMM captures a VM exit, loads the guest state, switches to host handler code, and forwards the request to QEMU for processing.

Nature of a QEMU‑KVM Virtual Machine

A QEMU‑KVM VM consists of vCPU, vMemory, virtual I/O devices, and a Guest OS. It appears to the host as a user process with multiple threads:

vCPU threads run guest code.

I/O threads run QEMU device emulation.

Other threads handle event loops and off‑loaded tasks.

Each vCPU is a thread inside the VM process; vMemory is a region of the process’s virtual address space. The VM can inherit host NUMA and huge‑page features and is represented on the host file system by an XML descriptor and a QCOW2 image.

Two‑Level vCPU Scheduling

Guest OS performs the second‑level scheduling of its user applications onto vCPUs, while the VMM (host OS) performs the first‑level scheduling of QEMU and vCPU threads onto physical CPUs.

vCPU Topology and Models

KVM supports SMP and NUMA topologies. Use

qemu‑kvm -smp <n>[,cores=<ncores>][,threads=<nthreads>][,sockets=<nsocks>][,maxcpus=<maxcpus>]

for SMP and

qemu‑kvm -numa <nodes>[,mem=<size>][,cpus=<cpu[-cpu]>][,nodeid=<node>]

for NUMA.

CPU models define which host CPU features are exposed to the guest. The -cpu host option makes the guest use the same model as the host.

$ kvm -cpu ?
x86       Opteron_G5  AMD Opteron 63xx class CPU
x86       Opteron_G4  AMD Opteron 62xx class CPU
...

Disk and Network Device Configuration

Disk devices are configured with qemu‑kvm -drive option[,option...] (file, if, index, media, format). Boot order is set with qemu‑kvm -boot [order=drives][,once=drives][,menu=on|off].

Network devices can be added with

qemu‑kvm -net nic[,vlan=n][,macaddr=mac][,model=type][,name=name]

or qemu‑kvm -net tap... for tap interfaces, and qemu‑kvm -net user... for user‑mode networking.