Understanding Modern Operating Systems: From Hardware Basics to System Calls and Architecture

Operating System Overview

Modern computers consist of a CPU, main memory, I/O devices and storage. The operating system (OS) sits between hardware and user programs, providing a uniform, abstract view of the machine.

CPU and Execution Model

The CPU repeatedly fetches, decodes and executes instructions. Key registers include the program counter (PC), stack pointer (SP) and program status word (PSW). Modern CPUs use pipelines and superscalar execution to process multiple instructions per clock cycle.

Memory Hierarchy

Memory is organised in levels: registers, L1/L2 caches (64‑byte cache lines), main RAM and secondary storage (disk/SSD). A cache hit satisfies a request in a few cycles; a miss incurs a longer latency. Virtual memory, managed by the Memory Management Unit (MMU), extends the address space beyond physical RAM.

Process Management

A process is an executing program with its own address space, registers, open files and other resources. Processes are created with fork(), terminated with exit(), and can replace their image with execve(). The OS assigns each process a PID and runs it in either kernel or user mode.

while (TRUE) {
    type_prompt();
    read_command(command, parameters);
    if (fork() != 0) {
        waitpid(-1, &status, 0);
    } else {
        execve(command, parameters, 0);
    }
}

Address Space Layout

Each process’s address space is divided into a text segment (code), a data segment (globals), a heap (dynamic allocation) and a stack (grows downward). The kernel enforces protection so that one process cannot access another’s memory.

File System Basics

Files are accessed via system calls such as open(), read(), write() and close(). Directories organise files in a hierarchical tree, with absolute paths starting at the root /. Special files in /dev represent devices and are accessed with the same read/write interface.

System Calls

System calls are the interface between user programs and the kernel. A typical call (e.g., read(fd, buffer, nbytes)) places arguments in registers or on the stack, executes a trap instruction to switch to kernel mode, the kernel dispatches to the appropriate handler, and finally returns the result to the user program.

Process‑related System Calls

fork()

– create a child process. waitpid(pid, &status, options) – wait for a child to terminate. execve(path, argv, envp) – replace the current image with a new program. exit(status) – terminate the process.

File‑related System Calls

open(path, flags, ...)

– open a file for reading, writing or both. close(fd) – close a file descriptor. read(fd, buffer, nbytes) – read data from a file. write(fd, buffer, nbytes) – write data to a file. lseek(fd, offset, whence) – reposition the file offset. stat(path, &buf) – obtain file status information.

Directory‑related System Calls

mkdir(path, mode)

– create a new directory. rmdir(path) – remove an empty directory. link(oldpath, newpath) – create a hard link. unlink(path) – delete a name (file or link). mount(source, target, flags) – mount a filesystem. umount(target) – unmount a filesystem.

Other Useful System Calls

chdir(path)

– change the working directory. chmod(path, mode) – change file permission bits. kill(pid, signal) – send a signal to a process. time(&seconds) – get the current time in seconds since the Unix epoch.

Operating‑System Structures

Different design philosophies balance performance, reliability and modularity.

Monolithic Kernel

All services run in kernel mode as a single large binary. Internal calls are fast, but a bug in any component can crash the entire system.

Layered System

Functionality is divided into layers (hardware, memory management, I/O, user interface). Each layer only interacts with its immediate neighbours, simplifying design and debugging.

Microkernel

Only minimal services (scheduler, inter‑process communication, low‑level I/O) run in kernel mode. Device drivers, file systems and other services run as user‑mode servers, improving reliability because a server crash does not bring down the kernel. MINIX 3 is a classic example.

Client‑Server Model

Clients request services from servers, which may reside on the same machine or across a network. The model abstracts away whether a service is local or remote, enabling scalable distributed systems such as web services.

Windows API vs. POSIX

Windows provides the Win32 API (e.g., CreateProcess, ReadFile, WriteFile), an extensive set of functions for GUI, file I/O, process creation and synchronization. POSIX defines a smaller, portable set of system calls (e.g., fork, execve, read, write) used by Unix‑like systems. Both ultimately perform the same low‑level operations, but their programming models differ: Unix is process‑centric and command‑line oriented, while Windows is event‑driven and GUI‑centric.

Process Concept

A process encapsulates everything needed to run a program: its address space, registers (PC, SP, PSW), open file descriptors, and scheduling information. The OS maintains a process table that records the state of each process. Parent processes can create child processes with fork(); the child receives PID 0 from fork(), the parent receives the child’s PID. After forking, the child typically calls execve() to run a new program, while the parent may call waitpid() to reap the child’s exit status.

Address Space

Virtual memory allows a process to have an address space larger than physical RAM. The MMU maps virtual pages to physical frames; pages not resident in RAM are swapped to disk on demand.

File Concept

Files are accessed through system calls. Directories form a hierarchical namespace rooted at /. Absolute paths start at the root; relative paths are resolved against the process’s current working directory. Permissions are expressed with the familiar rwx bits for owner, group and others.

Shell

The shell (e.g., sh, bash) reads user commands, forks a child, execs the requested program, and waits for it to finish. It supports I/O redirection ( >, <) and pipelines ( |) to connect the output of one program to the input of another.

date
# redirects the output of the date command to a file
date > file
# uses a pipeline to sort the concatenation of three files and send the result to a printer
cat file1 file2 file3 | sort > /dev/lp

System Call Mechanics

When a user program needs kernel services, it places arguments in registers or on the stack and executes a trap (e.g., int 0x80 on x86). The trap switches the CPU to kernel mode, the kernel looks up the system‑call number in a dispatch table, invokes the appropriate handler, and finally returns to user mode, restoring the saved registers and stack pointer. If the requested operation cannot be completed immediately (e.g., reading from a terminal with no input), the kernel may block the process and schedule another one.

Operating‑System Design Summary

Monolithic kernel – all services run in a single address space.

Layered architecture – clear separation of concerns across layers.

Microkernel – minimal kernel with user‑mode servers for drivers and filesystems.

Client‑server model – services accessed via request/response, local or remote.