Why the AI Research Community and Linux Kernel Engineers Remain Worlds Apart

Core kernel engineering challenges remain extensive despite the AI hype. In the scheduler, engineers must still address power‑performance balance, scheduling latency that causes UI jank, priority inversion, and priority inversion caused by user‑space locks.

Memory‑management problems include allocation and reclamation latency, fragmentation, LRU precision for reducing refaults, stable utilization of large folios, improving readahead hit rates for both file reads and swap‑in, and optimizing swap‑out/in performance.

Additional optimization space exists in file‑system performance, coordinated use of mixed storage media, and file‑system support for large folios, especially as memory and storage costs rise.

AI‑related kernel research

Recent years have seen a surge of papers proposing AI‑driven kernel optimizations, AI‑written schedulers, AI‑based I/O algorithms, and AI‑tuned system parameters. The Linux kernel community adopts only a tiny fraction of these ideas because:

The authors are often not long‑term, deep contributors to the kernel project.

The community follows a stable, conservative, engineering‑first evolution model and does not adopt a change solely because it shows gains on a specific workload or benchmark.

Consequently, roughly 99 % of such papers never become part of the mainline kernel. Inclusion typically requires:

Extended patch‑review cycles.

Acceptance by the relevant subsystem maintainers.

Broad community consensus covering performance, security, and maintainability.

Reviewers frequently question why a machine‑learning approach should replace an existing heuristic prefetch algorithm, probing the real benefit, overhead, generality, and maintainability.

Illustrative case: KML paper

The 2021 paper “KML: Using Machine Learning to Improve Storage Systems” (https://arxiv.org/abs/2111.11554) proposes a kernel‑space ML framework for I/O prefetch and NFS rsize tuning. Five years later it remains a well‑cited research work but has not been merged into the mainline kernel, exemplifying the long validation path.

Practical focus for kernel engineers

Rather than embedding AI everywhere, a more immediate and productive direction is to make the kernel friendly to AI workloads. Large models impose new resource‑allocation requirements across CPU, GPU, and NPU, demand reduced memory copies, zero‑copy data paths, and accelerated I/O.

Collaboration between kernel developers, AI‑framework teams, and application engineers can target end‑to‑end pipelines: coordinating large‑model execution with camera, display, GPU, NPU, and DMA engines; optimizing memory‑bandwidth usage; enabling zero‑copy; and improving storage I/O for speed and energy efficiency.

Key takeaways

Continue strengthening core kernel expertise independent of AI hype.

Explore opportunities to make the kernel more accommodating to AI workloads.

Approach AI‑driven kernel improvements cautiously, ensuring solid engineering validation before large‑scale adoption.