How KubeVirt Extends Kubernetes to Manage Virtual Machines and Bare Metal

Background

The environment runs two independent schedulers: OpenStack for bare‑metal and VM workloads, and Kubernetes for containers. Maintaining both stacks creates duplicated effort and resource waste, while workloads are moving toward containers. A unified scheduler that can handle VMs, bare metal, and containers is required.

OpenStack to Kubernetes transition

OpenStack’s community activity is declining and its architecture is complex, increasing operational overhead. Kubernetes offers a simpler, more extensible model, prompting the exploration of using Kubernetes to schedule VMs and bare metal alongside containers.

Technical selection

Several projects bridge OpenStack and Kubernetes (KubeVirt, Virtlet, Rancher/Harvester). KubeVirt is the most active and well‑designed, using Kubernetes CRDs and operators to manage VMs.

KubeVirt overview

KubeVirt is an open‑source Red Hat project that runs virtual machines as first‑class citizens in a Kubernetes cluster. It reuses the cluster’s CNI and CSI plugins for networking and storage, acting as a Kubernetes‑native VM management component rather than a full OpenStack replacement.

KubeVirt components

The core components are virt‑api , virt‑controller , virt‑handler and virt‑launcher . These replace OpenStack services such as Nova, Neutron and Cinder, while scheduling is delegated to Kubernetes.

Domain manager interface (Go)

type DomainManager interface {
    // SyncVMI creates a VM
    SyncVMI(*v1.VirtualMachineInstance, bool, *cmdv1.VirtualMachineOptions) (*api.DomainSpec, error)
    // PauseVMI pauses a VM
    PauseVMI(*v1.VirtualMachineInstance) error
    // UnpauseVMI resumes a paused VM
    UnpauseVMI(*v1.VirtualMachineInstance) error
    // KillVMI force‑kills a VM
    KillVMI(*v1.VirtualMachineInstance) error
    // DeleteVMI deletes a VM
    DeleteVMI(*v1.VirtualMachineInstance) error
    // SignalShutdownVMI initiates graceful shutdown
    SignalShutdownVMI(*v1.VirtualMachineInstance) error
    // MarkGracefulShutdownVMI marks a VM for graceful shutdown
    MarkGracefulShutdownVMI(*v1.VirtualMachineInstance) error
    // ListAllDomains lists all VM domains
    ListAllDomains() ([]*api.Domain, error)
    // MigrateVMI migrates a VM
    MigrateVMI(*v1.VirtualMachineInstance, *cmdclient.MigrationOptions) error
    // PrepareMigrationTarget prepares the migration target
    PrepareMigrationTarget(*v1.VirtualMachineInstance, bool) error
    // GetDomainStats retrieves VM statistics
    GetDomainStats() ([]*stats.DomainStats, error)
    // CancelVMIMigration cancels a migration
    CancelVMIMigration(*v1.VirtualMachineInstance) error
    // GetGuestInfo obtains guest‑agent information
    GetGuestInfo() (v1.VirtualMachineInstanceGuestAgentInfo, error)
    // GetUsers lists guest OS users
    GetUsers() ([]v1.VirtualMachineInstanceGuestOSUser, error)
    // GetFilesystems lists guest filesystems
    GetFilesystems() ([]v1.VirtualMachineInstanceFileSystem, error)
    // SetGuestTime sets the VM clock
    SetGuestTime(*v1.VirtualMachineInstance) error
}

VMI management commands

# kubectl get vmi -o wide
NAME                AGE   PHASE    IP           NODENAME        LIVE‑MIGRATABLE
test100.foo.demo.com 8d   Running  192.168.10.30 10.10.67.244   True
test200.foo.demo.com 8d   Running  192.168.10.31 10.10.67.245   True

# kubectl -n kubevirt get pod
NAME               READY   STATUS   RESTARTS   AGE
virt-api-68c958dd-6sx4n   1/1   Running   0   14d
virt-controller-647d666bd5-gsnzf   1/1   Running   1   14d
... (other virt‑* pods omitted)

Storage options

KubeVirt supports several disk sources, including cloudInitNoCloud (cloud‑init data via ConfigMap), DataVolume (automatic PVC import from HTTP or existing PVC), and direct PersistentVolumeClaim . The example below creates a block‑mode PVC backed by Ceph RBD, which provides RWX access required for live migration.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: testzhangsanlisi
spec:
  accessModes:
  - ReadWriteMany
  volumeMode: Block
  resources:
    requests:
      storage: 10Gi
  storageClassName: csi-rbd-sc

Network configuration (Kube‑OVN)

KubeVirt relies on the underlying Kubernetes network. In this deployment the Kube‑OVN CNI provides L2‑style networking with a VLAN underlay, allowing VMs to obtain fixed IPs and communicate through physical switches.

spec:
  cidrBlock: 192.168.10.0/23
  default: true
  excludeIps:
  - 192.168.10.1
  gateway: 192.168.10.1
  gatewayNode: ""
  gatewayType: distributed
  natOutgoing: false
  private: false
  protocol: IPv4
  provider: ovn
  underlayGateway: true
  vlan: ovn-vlan

Python SDK usage

The Python SDK wraps the KubeVirt CRD endpoints. A minimal client can be created as follows:

import kubevirt

def get_api_client(host):
    return kubevirt.ApiClient(host=host, header_name="Content-Type", header_value="application/json")

api_client = get_api_client(host="http://127.0.0.1:8001")
api_instance = kubevirt.DefaultApi(api_client)

Custom rename operation (Python)

The upstream SDK lacks the newName parameter for VM renaming. The following method adds the missing field:

def v1alpha3_rename_with_http_info(self, name, newName, namespace, **kwargs):
    body_params = {"newName": newName}
    api_route = "/apis/subresources.kubevirt.io/v1alpha3/namespaces/{namespace}/virtualmachines/{name}/rename".format(namespace=namespace, name=name)
    return self.api_client.call_api(api_route, 'PUT', {}, [], {}, body=body_params, response_type='str')

Live migration checks (Go)

Migration is allowed only when all attached volumes are shared (RWX) and the VM does not use local disks. The controller checks both volume and network suitability:

func (d *VirtualMachineController) calculateLiveMigrationCondition(vmi *v1.VirtualMachineInstance, hasHotplug bool) (*v1.VirtualMachineInstanceCondition, bool) {
    liveMigrationCondition := v1.VirtualMachineInstanceCondition{Type: v1.VirtualMachineInstanceIsMigratable, Status: k8sv1.ConditionTrue}
    isBlockMigration, err := d.checkVolumesForMigration(vmi)
    if err != nil {
        liveMigrationCondition.Status = k8sv1.ConditionFalse
        liveMigrationCondition.Message = err.Error()
        liveMigrationCondition.Reason = v1.VirtualMachineInstanceReasonDisksNotMigratable
        return &liveMigrationCondition, isBlockMigration
    }
    if err = d.checkNetworkInterfacesForMigration(vmi); err != nil {
        liveMigrationCondition = v1.VirtualMachineInstanceCondition{Type: v1.VirtualMachineInstanceIsMigratable, Status: k8sv1.ConditionFalse, Message: err.Error(), Reason: v1.VirtualMachineInstanceReasonInterfaceNotMigratable}
        return &liveMigrationCondition, isBlockMigration
    }
    if hasHotplug {
        liveMigrationCondition = v1.VirtualMachineInstanceCondition{Type: v1.VirtualMachineInstanceIsMigratable, Status: k8sv1.ConditionFalse, Message: "VMI has hotplugged disks", Reason: v1.VirtualMachineInstanceReasonHotplugNotMigratable}
        return &liveMigrationCondition, isBlockMigration
    }
    return &liveMigrationCondition, isBlockMigration
}

func (d *VirtualMachineController) checkVolumesForMigration(vmi *v1.VirtualMachineInstance) (bool, error) {
    for _, volume := range vmi.Spec.Volumes {
        volSrc := volume.VolumeSource
        if volSrc.PersistentVolumeClaim != nil || volSrc.DataVolume != nil {
            var volName string
            if volSrc.PersistentVolumeClaim != nil {
                volName = volSrc.PersistentVolumeClaim.ClaimName
            } else {
                volName = volSrc.DataVolume.Name
            }
            _, shared, err := pvcutils.IsSharedPVCFromClient(d.clientset, vmi.Namespace, volName)
            if err != nil {
                return false, err
            }
            if !shared {
                return true, fmt.Errorf("cannot migrate VMI with non-shared PVCs")
            }
        } else if volSrc.HostDisk != nil {
            if volSrc.HostDisk.Shared == nil || !*volSrc.HostDisk.Shared {
                return true, fmt.Errorf("cannot migrate VMI with non-shared HostDisk")
            }
        } else {
            // non‑shared local disk – block migration only
            return true, nil
        }
    }
    return false, nil
}

References

https://kubevirt.io/

https://github.com/kubevirt/client-python

https://github.com/kubevirt/client-go

Illustrations