Understanding Main Memory Architecture: Channels, Ranks, Banks, and Cache Line Filling

Memory System Overview

In computer architecture, memory usually refers to main memory, but the memory system consists of two parts: the large‑capacity, slower, cheaper main memory and the small‑capacity, faster, more expensive cache (see Figure 1).

Main memory (main memory) – large capacity, slower access, low cost.

Cache – small capacity, fast access, high cost.

Main Memory Organization

Modern main memory is implemented with DRAM (Dynamic Random Access Memory). DDR (Double Data Rate) is a common DRAM implementation.

Memory Slot

Memory slots host independent memory modules (e.g., DIMM – Dual In‑line Memory Module) that contain DRAM chips.

Memory Channel

A memory channel is the physical data path between the CPU and memory modules. A Xeon processor typically provides 8 memory channels, each with 2 slots, for a total of 16 slots per CPU.

Memory Controller

The memory controller manages data flow from main memory to each channel. Xeon CPUs usually have four memory controllers, each requiring at least one DIMM attached.

DIMM

A DIMM has two sides (single‑sided versions also exist) and can host many memory chips. The front side contains an EEPROM that stores module information such as capacity, DDR type, timing, voltage, and device I/O width. During boot, the CPU reads this data via SPD (Serial Presence Detect) to configure the memory controller.

Rank

A rank is a group of DRAM chips that share a single chip‑select signal ( CS#) and operate in lockstep. Typical DIMMs have two ranks (rank0 and rank1). Each rank contains a fixed number of chips; the number depends on chip width (e.g., 16 chips for x4, 8 chips for x8, 4 chips for x16).

Bank

A memory bank is a hardware‑implemented logical storage unit that contains rows and columns. A typical DDR3 chip contains eight banks, enabling bank‑level parallelism where banks can be in different states (activate, precharge, read/write) simultaneously.

Bank‑level parallelism does not conflict with the lockstep operation of chips within a rank because each bank has its own state machine and row buffer, allowing the memory controller to interleave accesses across banks.

From Main Memory to Cache

A burst is a continuous data transfer performed by DRAM without issuing separate commands for each datum. The burst length ( BL ) is set by the memory controller based on DRAM capabilities (e.g., BL=8 ).

When a memory request is issued, the memory controller follows three steps for the target rank: ACTIVATE – loads an entire row (cells) into the row buffer. READ/WRITE – accesses the desired column within the row buffer. For a read, data moves from the row buffer to the processor; for a write, the processor’s data updates the row buffer. PRECHARGE – closes the row buffer, preparing the bank for the next activation.

Each DRAM command incurs a timing delay, so commands must be spaced appropriately. In a typical configuration (e.g., 8‑bit I/O width, BL=8), a single READ command results in 8 beats of 8‑bit data per chip, yielding 64 bytes – exactly one cache line.

References

https://infohub.delltechnologies.com/zh-cn/l/memory-population-rules-for-3rd-generation-intel-xeon-scalable-processors-on-poweredge-servers-1/memory-topography-and-terminology/

https://computerlounge.co.nz/blogs/product-insights/memory-channels-ranks-configurations

https://www.dejazzer.com/coen4730/doc/lecture05_dram.pdf

https://en.wikipedia.org/wiki/Memory_rank

https://en.wikipedia.org/wiki/Memory_controller

https://www.kingston.com.cn/en/memory/kingston-glossary

http://bitsavers.informatik.uni-stuttgart.de/components/intel/spd/SPD_Spec_1.2B_199911.pdf