Understanding Linux Processes, Threads, IPC, System Calls and Scheduling

Basic Concepts

Linux treats a process as an independent program that runs its own code and has a unique PID. Each process can create additional threads, each with its own program counter, while sharing the same address space when needed.

Linux Process and Thread Implementation

Linux Processes

In the kernel a process is represented by a task_struct (also called a task). The structure stores scheduling parameters, memory image, open file descriptors, signal handling information, register state, system‑call state, accounting data, and other metadata required for management.

The PID is used to locate the corresponding task_struct directly, avoiding linear searches. The process descriptor lives in kernel memory and is loaded when the process is scheduled to run.

Linux Threads

Threads are lightweight processes that share most of the address space with their parent process. Linux implements user‑level threads, kernel‑level threads, and hybrid models that multiplex user threads onto kernel threads.

Inter‑Process Communication (IPC)

Linux provides several IPC mechanisms:

Signals : asynchronous notifications such as SIGINT , SIGKILL , SIGSTOP , etc. Some signals cannot be ignored ( SIGKILL , SIGSTOP ).

Pipes : unidirectional byte streams created with the pipe system call; the shell implements pipelines (e.g., sort < f | head ).

Named Pipes (FIFOs) : appear as special files in the filesystem and provide FIFO semantics.

Shared Memory : processes attach to a common memory segment using shmget() , shmat() , shmdt() , and shmctl() .

Message Queues : kernel‑managed queues identified by an IPC key; support strict (FIFO) and non‑strict ordering.

Sockets : endpoint‑based communication supporting TCP, UDP, and raw protocols; can be stream, datagram, sequential packet, or raw sockets.

Key System Calls for Process Management

The most important calls are:

fork() – creates a child process that is a copy of the parent (copy‑on‑write is used to share pages until they are modified).

exec() family (e.g., execl , execve ) – replaces the current image with a new program while keeping the same PID.

waitpid() – waits for a child to terminate.

exit() – terminates the calling process, returning a status code.

Other useful calls: pause , nice , ptrace , kill , mkfifo , sigaction , msgctl , semctl .

pid = fork(); // create a new process
if (pid < 0) {
    error(); // fork failed
} else if (pid > 0) {
    parent_handle(); // parent code
} else {
    child_handle(); // child code
}

Process Creation Details

When fork() is invoked, the kernel allocates a new kernel stack and a thread_info structure, copies most of the parent’s context, assigns a new PID, and sets up copy‑on‑write page tables. After exec() , the old address space is discarded, a page‑fault loads the new program’s code, and the stack is rebuilt with arguments and environment variables.

Linux Scheduling

Linux schedules kernel threads, not user processes. Threads are classified into three groups: real‑time FIFO, real‑time round‑robin, and normal (time‑sharing) tasks. Each thread has a nice value (‑20 to +19) that influences its static priority.

The historic O(1) scheduler used two arrays of priority queues (active and expired). Tasks moved between these arrays based on time‑slice expiration and I/O blocking. It employed heuristics such as rewarding interactive tasks and penalising CPU‑bound tasks using a sleep_avg metric.

Modern kernels use the Completely Fair Scheduler (CFS), which stores runnable tasks in a red‑black tree ordered by virtual runtime. The leftmost node (the task that has used the least CPU time) is selected to run next. Insertion is O(log N), where N is the number of runnable tasks.

Synchronization Primitives

Early Linux used a Big Kernel Lock (BKL). Modern kernels provide finer‑grained mechanisms:

Atomic operations ( atomic_set , atomic_read ) for lock‑free updates.

Spinlocks for short critical sections.

Mutexes and semaphores, with non‑blocking variants like mutex_trylock and mutex_trywait .

Memory barriers to prevent reordering.

Linux Boot Process

On power‑on, the BIOS performs POST, then loads the MBR (512 bytes) into a fixed memory location. The bootloader copies itself to high memory, loads the kernel image, and transfers control to the kernel’s assembly entry point, which sets up the stack, disables interrupts, and initializes the MMU.

The kernel then allocates a message buffer for early debugging, detects hardware, mounts the root filesystem, and creates the first user‑space process (PID 1, init ). init decides between single‑user or multi‑user mode, forks a shell or runs /etc/rc , which in turn starts getty on each terminal. getty prompts for a login name, hands control to /bin/login , which authenticates against /etc/passwd and finally starts the user’s shell.

Conclusion

The article walks through the essential concepts of Linux process and thread management, the variety of IPC mechanisms, the core system calls that drive process creation and termination, the evolution of the scheduler from O(1) to CFS, synchronization techniques, and the step‑by‑step boot sequence that brings the system from power‑on to a usable user environment.