Unlocking Linux: A Deep Dive into Processes, Memory Management, Filesystems, I/O and Security

Linux Overview

Linux is a multi‑user, multitasking operating system derived from UNIX. It runs on devices ranging from smartphones to supercomputers and follows the principle of small, well‑defined components.

Shell and basic commands

The command line (shell) reads a line, parses the first word as the command and executes it. Common utilities include:

ls A*          # list files starting with "A"
rm A*          # remove those files
grep ard f | sort | head -30 | tail -5 > f00

Pipelines (the | operator) connect programs without creating intermediate files.

Processes and threads

Linux creates a new process with fork(), which clones the parent’s address space using copy‑on‑write. The child receives a unique PID; the parent receives the child’s PID. exec() replaces the current program image without changing the PID. Threads are lightweight processes that share the same address space; each thread is represented by a task_struct containing scheduling parameters, open file table, registers, etc.

Key system calls: fork() – create a child process execve() (and the exec family) – load a new program image waitpid() – wait for a child to terminate exit() – terminate a process getpid() – obtain the calling process’s PID

Inter‑process communication (IPC)

Signals – asynchronous notifications (e.g., SIGKILL, SIGSTOP, SIGINT)

Pipes – one‑way byte streams created with pipe() and used via | in the shell

Shared memory – shmget(), shmat(), shmdt(), shmctl() FIFO (named pipes) – persistent pipe files in the filesystem

Message queues – ordered or unordered message passing

Sockets – TCP/UDP network communication

Scheduling

Linux classifies runnable threads into three classes: real‑time FIFO, real‑time round‑robin, and time‑sharing. Each thread has a nice value (‑20 … +19) that influences its static priority.

Historical O(1) scheduler used active/expired arrays; modern kernels use the Completely Fair Scheduler (CFS). CFS maintains a red‑black tree of runnable tasks ordered by virtual runtime, always selecting the left‑most (least‑run) task.

Memory management

Each process has a virtual address space divided into text, data (including BSS), heap, and stack segments. The kernel uses paging: virtual addresses are split into a virtual page number and offset, translated via multi‑level page tables. Demand paging loads pages on first access; page faults trigger allocation or loading.

When physical memory is scarce, Linux swaps out pages using an LRU‑based algorithm. The buddy allocator manages free physical pages in power‑of‑two blocks, splitting and coalescing to reduce fragmentation.

Cache hierarchy:

Buffer cache – caches block device I/O

Page cache – caches file data pages

Swap cache – holds dirty pages that have been written to the swap area

TLB – hardware translation‑lookaside buffer for recent page table entries

Kernel structure

The kernel consists of several subsystems: interrupt handling, scheduler, memory manager, I/O subsystem, and the system‑call interface. The top‑level start_kernel() initializes hardware, mounts the root filesystem, and starts init (PID 1), which spawns getty, login, and user shells.

Signals

SIGKILL

, SIGSTOP – cannot be caught or ignored SIGINT – generated by Ctrl‑C SIGTERM, SIGQUIT – termination requests SIGCHLD – child status change SIGSEGV – invalid memory reference

File system

Linux uses a Virtual File System (VFS) to abstract concrete file‑system implementations (ext2, ext4, NFS, procfs, etc.). VFS objects:

Superblock – global file‑system metadata

Inode – per‑file metadata (mode, UID/GID, size, timestamps, block pointers)

Dentry – directory entry cache for path lookup

File – open file description associated with a process

ext2 layout (boot block, superblock, group descriptor, inode table, data blocks) stores directories as linear lists of entries ( inode, rec_len, name_len, file_type, name). ext4 adds a journaling layer (JBD) and extents for larger files.

The /proc pseudo‑filesystem provides per‑process information (e.g., /proc/1234/status) without storing data on disk.

NFS exports directories on a server; clients mount them into their namespace. The protocol uses file handles (unique identifiers) and supports both read/write and metadata‑only operations.

File operations

open(path, flags, mode)

/ creat(name, mode) – obtain a file descriptor read(fd, buf, count) /

write(fd, buf, count)

lseek(fd, offset, SEEK_SET|SEEK_CUR|SEEK_END)

– reposition file offset stat(path, &buf) / fstat(fd, &buf) – retrieve inode information pipe(fd[2]) – create a unidirectional pipe fcntl(fd, F_SETLK, ...) – apply advisory locks (shared or exclusive)

mkdir, rmdir, link, unlink, chdir, opendir, readdir, closedir

– directory management

Standard file descriptors: 0 = STDIN, 1 = STDOUT, 2 = STDERR.

I/O subsystem

All devices appear as special files under /dev. Two major classes:

Block devices – accessed in fixed‑size sectors (e.g., disks, USB sticks). The kernel uses the generic block layer and an I/O scheduler (elevator/deadline) to merge, reorder, and dispatch requests, reducing seek time.

Character devices – stream of bytes (e.g., terminals, serial ports). Access is unbuffered and non‑seekable; line disciplines (tty) provide canonical processing.

Network sockets are created with socket() and support TCP (reliable byte stream) and UDP (unreliable datagrams). After bind() and listen(), a server accepts connections with accept(); a client connects with connect(). Data is transferred via read()/write() or send()/recv().

Loadable kernel modules

Modules can be inserted at runtime with insmod and removed with rmmod. The kernel resolves symbols, registers device numbers, and hooks the driver into the appropriate subsystem without rebooting.

Security model

Every process has a real and an effective UID/GID (0 = root). Files have an owner UID, a group GID, and a 9‑bit permission mask (rwx for owner, group, others). The setuid and setgid bits cause an executable to run with the file’s owner/group IDs, enabling privileged helpers such as /bin/ping. Common security‑related system calls: chmod(path, mode) – change permission bits chown(path, uid, gid) – change owner/group (root only) access(path, R_OK|W_OK|X_OK) – test real UID/GID permissions setuid(uid), setgid(gid) – change effective IDs (root only) getuid(), getgid(), geteuid(), getegid() The login program authenticates users against /etc/passwd (hashed passwords), then uses setuid / setgid to assume the user’s identity before launching the user’s shell.

Boot process

On power‑on the BIOS performs POST, loads the MBR, and transfers control to a bootloader (e.g., GRUB). The bootloader switches to protected mode, loads the kernel image, and jumps to start_kernel(). The kernel initializes hardware, mounts the root filesystem, starts the init process (PID 1), which spawns getty, login, and other system services.