How KVM and QEMU Work Together to Power Linux Virtualization

KVM Architecture

KVM consists of two main components: the KVM driver, now a Linux kernel module responsible for creating virtual machines, allocating virtual memory, reading and writing virtual CPU registers, and running the virtual CPU; and QEMU, a generic open‑source machine emulator that provides user‑space I/O device models and access to peripherals.

QEMU is a pure‑software virtualization emulator that can simulate almost any hardware device, allowing a virtual machine to think it is interacting with real hardware while QEMU translates those instructions to the actual hardware.

Because QEMU is software‑only, its performance is lower; therefore, in production environments QEMU is combined with KVM, where KVM handles CPU and memory virtualization with hardware assistance, and QEMU handles I/O device virtualization.

Each virtual machine corresponds to a QEMU process on the host, with its execution threads mapped to QEMU threads.

Below is a step‑by‑step example of how KVM and QEMU interact when launching a VM:

kvmfd = open("/dev/kvm", O_RDWR);

vmfd = ioctl(kvmfd, KVM_CREATE_VM, 0);

ioctl(kvmfd, KVM_SET_USER_MEMORY_REGION, &mem);

ioctl(kvmfd, KVM_CREATE_VCPU, vcpuid);

vcpu->kvm_run_mmap_size = ioctl(kvm->dev_fd, KVM_GET_VCPU_MMAP_SIZE, 0);

ioctl(kvm->vcpus->vcpu_fd, KVM_RUN, 0);

for (;;) { ioctl(KVM_RUN); switch (exit_reason) { case KVM_EXIT_IO: /* ... */ case KVM_EXIT_HLT: /* ... */ } }

Exits from the VCPU loop occur for I/O operations, page faults, or other events, returning control to QEMU.

QEMU implements virtualization using binary translation: it extracts guest instructions, translates them to TCG intermediate code, and then to host architecture code (e.g., x86 or ARM).

Source Code Structure

The main source files include:

/vl.c – main emulation loop and CPU execution.

/target-arch/translate.c – translates guest code to TCG opcodes.

/tcg/tcg.c – core TCG implementation.

/tcg/arch/tcg-target.c – converts TCG code to host machine code.

/cpu-exec.c – finds or generates the next translated block.

KVM Functionality

KVM appears as a standard Linux character device /dev/kvm. QEMU uses the libkvm interface via ioctl to send commands to this device, which are handled by the kvm_dev_ioctl function in kvm_main.c.

KVM adds a new guest mode to Linux, alongside kernel and user modes, enabling each guest OS to run as a regular Linux process.

The three execution modes are:

Kernel mode – handles guest exits (VM_EXIT) caused by I/O or other instructions.

User mode – runs QEMU instructions for I/O handling.

Guest mode – executes non‑I/O guest code.

When QEMU issues an ioctl such as KVM_CREATE_VM, it receives a file descriptor representing the VM, which is then used for further commands like KVM_CREATE_VCPU.

KVM Working Principle

In operation, the user‑mode QEMU uses libkvm to enter kernel mode via ioctl; the KVM module creates virtual memory and a virtual CPU, then executes VMLAUNCH to enter guest mode. If the guest OS triggers an external interrupt or a page fault, execution pauses, exits guest mode for handling, and then resumes.

A virtual machine monitor (VMM) is the host program that enables multiple isolated execution environments on a single physical machine, allowing each user to feel they have a dedicated computer.

Modern CPUs provide hardware virtualization support, allowing VMMs to run directly on bare metal for improved performance.

Source: http://www.uml.org.cn/yunjisuan/202008171.asp?artid=23611