What Powers Supercomputers? A Deep Dive into High‑Performance Computing

How HPC Works

High‑performance computing (HPC) uses supercomputers and parallel processing techniques to complete long‑running or multiple tasks quickly. In HPC, information is processed mainly in two ways: serial processing performed by a central processing unit (CPU), where each core handles one task at a time, and parallel processing that leverages multiple CPUs or graphics processing units (GPUs). GPUs, originally designed for graphics, can execute many arithmetic operations simultaneously on data matrices, making them ideal for parallel workloads such as machine‑learning inference.

Performance Metrics

The computing power of a system is measured in FLOPS (floating‑point operations per second). By early 2019, top‑tier supercomputers could achieve 143.5 petaFLOPS (1.43 × 10¹⁵ FLOPS). In contrast, a high‑end desktop GPU reaches about 200 gigaFLOPS (2 × 10⁸ FLOPS), roughly one‑millionth of a modern supercomputer. The next milestone, exaFLOPS (10¹⁸ FLOPS), would be about a thousand times faster than the current peta‑scale machines.

System Design Considerations

Achieving high FLOPS requires not only fast processors but also sufficient data throughput. System memory and the interconnects between processing nodes heavily influence how quickly data can reach the CPUs/GPUs. Most HPC systems interconnect multiple processors and memory modules with ultra‑high‑bandwidth links, and many adopt heterogeneous architectures that combine CPUs and GPUs.

Why Use HPC?

From a system perspective, HPC integrates resources to meet growing performance and functionality demands. From an application perspective, it enables the decomposition of large or fine‑grained problems, allowing scientists and engineers to run computationally, data‑intensive, and network‑intensive simulations that would be infeasible on ordinary hardware.

Key Terminology

High‑Performance Computing (HPC): A broad class of powerful computing systems ranging from a single CPU with multiple GPUs to world‑leading supercomputers.

Supercomputer: The most advanced HPC machines, defined by continuously rising performance benchmarks.

Heterogeneous Computing: An HPC architecture that optimizes both serial (CPU) and parallel (GPU) processing capabilities.

Memory: Storage within an HPC system that provides fast data access for computation.

Interconnect: The system layer that enables communication between processing nodes, existing at multiple levels in supercomputers.

PetaFLOPS (10¹⁵ FLOPS): Systems designed to perform quadrillion floating‑point operations per second.

ExaFLOPS (10¹⁸ FLOPS): Systems designed to perform quintillion floating‑point operations per second.