How CPU Reads and Writes Memory: From Cache to Virtual Memory

The article begins by identifying what triggers CPU memory access: machine instructions generated by the compiler from high-level code. In RISC architectures, dedicated Load/Store instructions move data between registers and memory, while CISC (x86) allows memory operands directly in instructions.

Two distinct memory access types exist: (1) data reads/writes specified by instructions, and (2) instruction fetches driven by the program counter (PC), both rooted in the von Neumann architecture where program and data share memory.

The CPU–Memory Speed Gap and Cache

CPU speed far exceeds DRAM latency (analogy: a diner eating 100× faster than the chef cooks). To bridge this, a small, fast SRAM cache exploits locality of reference : temporal locality (recently accessed data likely reused) and spatial locality (adjacent data likely accessed). Only a small fraction of a program's memory footprint is hot at any time.

Cache Write Policies

Write-through : update cache and memory synchronously. Simple but forces every write to wait for slow memory.

Write-back : update cache only; mark the line dirty. Memory updates occur asynchronously when the line is evicted. Higher performance, more complex.

If a write misses in cache, the line must first be fetched from memory (read-for-ownership) before modification.

Multi-Level Cache Hierarchy

Modern CPUs use three cache levels: L1 (smallest, fastest, per-core), L2 (larger, per-core or shared), L3 (largest, shared across cores). Access proceeds L1 → L2 → L3 → main memory. Each level trades capacity for speed and cost.

Multi-Core Cache Coherence

With multiple cores, each has private L1/L2 caches. A variable X in memory can have cached copies in several cores. If Core 1 increments X (2→4) and Core 2 increments X (2→6) using stale cached values, memory ends at 6 instead of the correct 8. The problem: multiple cached copies of the same address .

Solution: hardware coherence protocols (e.g., MESI) ensure that when one core modifies a line, other cores' copies are invalidated or updated. Programmers should write cache-coherence-friendly code to avoid performance penalties from excessive coherence traffic.

Virtual Memory and MMU Translation

The OS presents each process with a large, contiguous virtual address space. The CPU issues virtual addresses; the on-chip Memory Management Unit (MMU) translates them to physical addresses before cache lookup. This translation enables memory isolation, paging, and the illusion of abundant memory.

Memory as a Cache for Disk

Physical memory itself acts as a cache for secondary storage. A page fault occurs when the needed page resides on disk; the OS loads it into a free frame, updates page tables, and resumes the instruction. The full access chain: virtual address → MMU → L1 → L2 → L3 → physical memory → (page fault) → disk.

The article concludes that a seemingly simple memory operation involves nearly every core component of a computer system—caches, coherence protocols, MMU, OS paging—and demands cache-aware programming for peak performance.

Code example

来源丨经授
权转自 码农的荒岛求生（ID：escape-it）
作者丨
码农的荒岛求生