Why Microkernels Can Beat Monolithic Kernels: A Deep Dive with C Simulations

Background and Motivation

Recent discussion of microkernel concepts has been revived by the release of HarmonyOS. While microkernels are often criticized for inter‑process communication (IPC) overhead, the author argues that the real issue is how operating systems handle shared resources. Introducing a dedicated arbiter to serialize access can mitigate contention.

Linux Kernel Bias and Scalability

The Linux kernel’s evolution from 2.6 to 5.3 added several SMP‑related improvements (O(1) scheduling, load‑balancing algorithms, per‑CPU data structures, lock splitting). These are incremental refinements rather than fundamental architectural changes.

Arbitration vs. Contention

When a resource can be accessed by only one entity at a time, it should be managed by an arbiter that queues requests, similar to an Ethernet switch. This principle applies to CPU scheduling, file access, sockets, and other shared resources.

Code Simulation of Macrokernel and Microkernel

Two C programs illustrate the difference between a macrokernel style (global spin‑lock) and a microkernel style (task queue with a spin‑lock only around the queue).

Macrokernel simulation (spin‑lock protected shared counter)

#include <pthread.h>
#include <signal.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <sys/time.h>

static int count = 0;
static int curr = 0;
static pthread_spinlock_t spin;

long long gettime(){
    struct timeb t; ftime(&t);
    return 1000*t.time + t.millitm;
}

void print_result(){
    printf("%d
", curr);
    exit(0);
}

void do_task(){
    int i=0,j=2,k=0;
    for(i=0;i<0xff;i++) k+=i/j; // dummy work
}

int main(int argc, char**argv){
    count = atoi(argv[1]);
    int tcnt = atoi(argv[2]);
    pthread_spin_init(&spin, PTHREAD_PROCESS_PRIVATE);
    long long start = gettime();
    for(int i=0;i<tcnt;i++){
        pthread_t tid; int err = pthread_create(&tid, NULL, func, NULL);
        if(err) exit(1);
    }
    sleep(3600);
    return 0;
}

Microkernel simulation (task queue protected by a spin‑lock)

#include <pthread.h>
#include <signal.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <sys/time.h>

static int count = 0;
static int total = 0;
static int timer = 0, timer_start = 0;
static pthread_spinlock_t spin;

struct node{struct node*next; void*data;};
static struct node*head = NULL;

void insert(struct node*node){
    node->data = NULL;
    node->next = head;
    head = node;
}

struct node* delete(){
    struct node*temp = head;
    head = head->next;
    return temp;
}

int empty(){return head==NULL;}

void print_result(){
    printf("%d
", total);
    exit(0);
}

void* server_func(void*arg){
    while(timer || total!=count){
        pthread_spin_lock(&spin);
        if(empty()){pthread_spin_unlock(&spin); continue;}
        if(timer && timer_start==0){
            struct itimerval tick={0};
            timer_start=1; signal(SIGALRM,print_result);
            tick.it_value.tv_sec=10; tick.it_value.tv_usec=0;
            setitimer(ITIMER_REAL,&tick,NULL);
        }
        struct node*tsk = delete();
        pthread_spin_unlock(&spin);
        do_task();
        free(tsk);
        total++;
    }
    long long end = gettime();
    printf("%lld   %d
", end-start, total);
    exit(0);
}

int main(int argc, char**argv){
    count = atoi(argv[1]);
    int tcnt = atoi(argv[2]);
    if(argc==4) timer=1;
    pthread_spin_init(&spin, PTHREAD_PROCESS_PRIVATE);
    pthread_t stid; int err = pthread_create(&stid, NULL, server_func, NULL);
    if(err) exit(1);
    long long start = gettime();
    for(int i=0;i<tcnt;i++){
        pthread_t tid; err = pthread_create(&tid, NULL, func, NULL);
        if(err) exit(1);
    }
    sleep(3600);
    return 0;
}

Benchmark Results

Graphs show that the macrokernel version’s total execution time grows roughly linearly with the number of threads, reflecting spin‑lock contention. The microkernel version’s time remains almost constant, and its increase with CPU count is only slight.

Analysis of Hotspots

CPU profiling of the macrokernel run shows the spin‑lock as the dominant hotspot, indicating most cycles are spent spinning. The microkernel run still shows a spin‑lock hotspot, but its impact is far smaller because only the queue operations are locked.

Conclusions

Introducing a dedicated arbiter (as in microkernel designs) to serialize access to shared resources reduces lock contention and improves CPU utilization. True parallelism still requires application‑level design, but the kernel’s role should be to serialize shared‑resource access efficiently rather than allowing uncontrolled contention.

Broader Implications

The same principle applies to file systems, network stacks, and other system components. Many modern services (e.g., Nginx) already employ microkernel‑like arbitration, whereas others (e.g., Apache) do not. Rethinking kernel architecture beyond legacy monolithic assumptions can lead to more scalable and efficient systems.