Partitioning Is Mandatory: The Cost of Heterogeneous RISC‑V and Single‑Kernel Designs

Why the Kernel Rejects Half the Cores

Key Stone K3 ships with sixteen cores, eight of which have 1024‑bit vector registers for AI workloads. When the mainline Linux kernel boots, it checks that every core’s vector width matches the boot core; any mismatch causes the core to be rejected, effectively discarding half the silicon.

This behavior stems from a long‑standing OS guarantee that any process can run on any core . The kernel saves the vector register state on every context switch and restores it unchanged, assuming a uniform vector width across all cores.

On K3, the eight 1024‑bit vector units require 4 KB of active state, while the 256‑bit general‑purpose cores provide only 1 KB of register file space, making it impossible to store the larger state on the smaller cores.

Economic Reality of Heterogeneous Cores

Wide‑vector units are the most expensive part of modern cores. Equipping all sixteen cores with such units would multiply cost by sixteen, yet typical workloads would only use a quarter of that capacity. No server‑class vendor would pay that price, so the design places wide engines in a subset of cores and keeps the rest simple.

RISC‑V adds complexity because there is no central authority to enforce a common vector width across IP blocks from different vendors, unlike ARM’s big.LITTLE approach.

Concrete Kernel Check

if (riscv_v_vsize != this_vsize) {
    WARN(1, "RISCV_ISA_V only supports one vlenb on SMP systems");
    return -EOPNOTSUPP;
}

The kernel variable riscv_v_vsize is the system‑wide vector size; any core whose width differs is refused, as documented in arch/riscv/kernel/smpboot.c [17].

Software Patches Hide, Not Solve, the Partition

Downstream kernels can record the per‑process vector width and allocate buffers sized for the widest core cluster, allowing a single‑kernel boot on the hardware. However, the fundamental problem remains: the 4 KB state cannot fit into a 1 KB register file, and the vector width is baked into the hardware register file size.

Consequently, all vector‑using code is confined to the wide‑core cluster; migration between clusters is impossible because the vector register snapshot format differs between RVV 0.7.1 and RVV 1.0.

Costs Inherited from the Partition

Hot‑plug failure: CPU affinity limits break during hot‑plug, preventing the last capable core from being taken offline.

SCHED_DEADLINE rejection: Programs bound to the wide cluster fail admission tests unless the scheduler is globally disabled.

CPU affinity mask tampering: Systemd, Kubernetes, and numactl silently reduce the requested CPU set to the available subset.

cpuset isolation undefined: ARM documentation declares this behavior; isolating AI cores from the rest of the machine becomes necessary.

C‑library vector usage loss: Linux reports the intersection of capabilities across all cores via hwcap. When RVV 0.7.1 and RVV 1.0 coexist, user space sees a diluted capability set, causing glibc’s IFUNC dispatch to fall back to scalar implementations.

Kernel vector code limited to narrowest width: Encryption, RAID, and CVE patches compile once for the narrow core cluster.

Incompatible RVV versions cannot share a kernel: Different state layouts prevent a single binary from supporting both versions.

These costs are borne by the scalar cores, which lose hot‑plug, deadline scheduling, precise affinity, cpuset isolation, and full‑speed kernel crypto, merely to accommodate the AI cluster.

Multi‑Kernel as a Remedy

Instead of forcing a single kernel to manage heterogeneous clusters, a multi‑kernel architecture runs independent Linux kernels on each core cluster, each aware of its own vector width. This eliminates shared state, restores hot‑plug, SCHED_DEADLINE, and cpuset semantics, and allows the wide‑core kernel to fully exploit its vector units.

Each kernel sees a uniform machine, compiles its own code paths, and uses the correct ISA version. The approach requires platform support for per‑cluster boot, AIA/IMSIC interrupt routing, and appropriate device‑tree or ACPI descriptions, which are more challenging on RISC‑V than on x86/ARM but are essential for a viable solution.

Conclusion

Heterogeneous RISC‑V servers are not an outlier; they are a direct result of economic forces. Partitioning is inevitable, and the only way to avoid the hidden costs of a forced single‑kernel design is to make partitioning an explicit, first‑class boundary via multi‑kernel techniques.