Why Android Threads Stall: Inside Linux CFS, RT, Fair Scheduling and cgroup Limits

The author started the series out of curiosity after a team project review revealed that less than 15% of encountered problems were actually solved by the scheduler, prompting a deep dive into Linux task scheduling.

1. Linux Scheduler Classes

Linux defines five scheduling classes ordered by priority: stop (highest), deadline , realtime (RT) , fair , and idle (lowest). The core function __pick_next_task iterates over these classes in order, invoking each class's pick_next_task to find a runnable task.

for_each_class(class) {
    p = class->pick_next_task(rq);
    if (p)
        return p;
}

In Android devices the CPU count is limited (typically 4‑8 cores), yet an app can spawn hundreds of threads. The scheduler’s time‑sharing ensures the user perceives many threads running concurrently.

2. RT vs. Fair Scheduling

RT threads have priority values 0‑99, while fair‑class threads use 100‑139. In systrace these priorities appear as numbers, but Linux treats them differently: RT uses pick_next_task_rt, which scans a bitmap of ready priorities from high to low; fair scheduling (CFS) uses a red‑black tree ordered by virtual runtime , not by static priority.

Virtual runtime is calculated as:

curr->vruntime += calc_delta_fair(delta_exec, curr);

and ultimately simplified to:

virtual_runtime = physical_runtime / weight;

where weight is derived from the task’s static priority via sched_prio_to_weight. Thus, in CFS a higher static priority translates to a larger weight, giving the task a larger share of CPU time over a period, but it does not guarantee immediate pre‑emptive advantage.

3. What Tasks the Scheduler Manages

The scheduler only tracks tasks in the TASK_RUNNING state. Other states (e.g., Uninterruptible Sleep) are outside its queue, which explains why many threads appear idle even when the CPU is free.

4. Proportional Scheduling Issues

CFS implements proportional scheduling: each task receives CPU proportionally to its weight. When many low‑priority threads are created (e.g., make -j32), they can collectively dominate CPU time despite low individual weights.

5. Control Groups (cgroup)

cgroup introduces a second scheduling layer. Each group has a cpu.shares value (default 1024) that acts as a group‑level weight, limiting the total CPU share of all tasks inside the group regardless of their individual priorities.

Experiments:

Two threads (PIDs 24725 and 24722) with priority 120 were pinned to CPU 7.

Placing them in separate cgroups ( test1 and test2) with equal cpu.shares resulted in a ~1:1 runtime ratio.

Reducing cpu.shares of test1 to 512 while keeping test2 at 1024 produced an approximate 1:2 runtime ratio, confirming the group‑level control.

These results show that when tasks belong to different cgroups, the group’s cpu.shares can outweigh individual priority differences.

6. cgroup Limitations on Android

Android’s default groups (top‑app, foreground, background) all have cpu.shares =1024, which is sub‑optimal because foreground often contains many services and UI threads. Moreover, binder IPC calls from a top‑app to a system service execute in the foreground group, further skewing resource allocation.

7. Conclusions

CFS priority represents weight, not strict pre‑emptive ordering; high‑priority tasks usually win statistically but not deterministically.

cgroup adds a hierarchical resource‑share layer; mismatched cpu.shares between groups can cause a lower‑priority thread to outrun a higher‑priority one.

Properly tuning cpu.shares per Android group could mitigate runaway resource consumption during heavy workloads like massive parallel compilation.

Future work will explore CFS bandwidth control ( CFS_BANDWIDTH) and deeper integration of group scheduling with Android’s service architecture.