Understanding SMP, NUMA, and MPP: Which Server Architecture Fits Your Needs?

1. SMP (Symmetric Multi‑Processor)

SMP systems contain tightly coupled processors that share all resources (bus, memory, I/O) and run a single operating system instance. All CPUs have equal access to memory and peripherals, and resource contention is resolved by hardware/software locking. Because all CPUs share the same memory bus, memory access becomes a bottleneck as CPU count grows, limiting effective scalability to about 2‑4 CPUs.

SMP servers are characterized by shared resources, which restricts expansion; the most efficient CPU utilization is typically observed with 2‑4 CPUs. An 8‑socket server marks the practical limit of SMP; beyond that, NUMA is required.

2. NUMA (Non‑Uniform Memory Access)

NUMA groups CPUs into modules, each with its own local memory and I/O. Modules are connected via an interconnect (e.g., Crossbar Switch). Access to local memory is fast, while remote memory access incurs higher latency, making it essential to minimize cross‑module communication in applications.

NUMA can support hundreds of CPUs in a single server (e.g., HP Superdome, SUN15K, IBM p690). However, performance does not scale linearly because remote memory accesses are slower; a 64‑CPU NUMA system may deliver only three times the performance of an 8‑CPU SMP system.

Modern CPUs (e.g., Intel Nehalem) integrate memory controllers, making NUMA awareness mandatory. Operating systems may enable Automatic NUMA balancing (Ubuntu 12.02, SUSE 12). Virtualization considerations include limiting KVM VMs to the CPUs within a single NUMA node and using vNUMA in VMware ESX to expose host NUMA topology to guests.

3. MPP (Massive Parallel Processing)

MPP builds a large system by interconnecting multiple independent SMP nodes via a high‑speed network. Each node accesses only its local memory and storage (share‑nothing architecture), allowing near‑linear scalability up to hundreds of nodes and thousands of CPUs.

MPP nodes run their own OS and databases; the complexity of load balancing and parallel execution is often hidden by database systems (e.g., NCR Teradata).

4. Comparing the Three Architectures

4.1 SMP vs. MPP

SMP suffers from a shared bus bottleneck, making it unsuitable for large‑scale decision‑support or data‑warehouse workloads, whereas MPP’s lack of shared resources allows better performance when inter‑node communication is low.

4.2 NUMA vs. MPP

Both use multiple nodes, but NUMA’s interconnect is internal to a single chassis, leading to latency when accessing remote memory, while MPP’s external network enables independent local memory access and near‑linear scaling.

4.3 Performance Differences

NUMA can scale to hundreds of CPUs but suffers from remote memory latency; MPP can scale to hundreds of nodes with linear performance growth; SMP offers limited scalability, typically optimal with 2‑4 CPUs.

4.4 Expansion Limits

NUMA and MPP theoretically support unlimited expansion (hundreds of CPUs or thousands of CPUs respectively), while SMP’s expansion is poor, though IBM’s BOOK technology can extend SMP to 8 CPUs.

4.5 Application Suitability

MPP excels in decision‑support and data‑mining workloads where communication overhead is low.

SMP provides higher efficiency for workloads with heavy inter‑process communication.

NUMA offers strong OLTP transaction processing but degrades for data‑warehouse workloads due to extensive cross‑module data exchange.