Understanding High‑Performance Computing: Principles, FLOPS, and Future Limits

High‑performance computing (HPC) uses supercomputers and parallel processing techniques to quickly complete long‑running tasks or execute many tasks simultaneously.

The HPC market is both traditional and rapidly emerging, targeting high‑end users and benchmark projects, with growing influence and a trend toward broader accessibility.

High‑Performance Computing Principles

In HPC, information is processed mainly in two ways:

Serial processing is performed by the central processing unit (CPU). Each CPU core typically handles one task at a time, and the CPU is essential for running operating systems and basic applications.

Parallel processing utilizes multiple CPUs or graphics processing units (GPUs). GPUs, originally designed for graphics, can perform many arithmetic operations simultaneously on data matrices, making them ideal for parallel tasks in machine learning such as object detection in videos.

Breaking the limits of supercomputing requires diverse system architectures. Most HPC systems interconnect multiple processors and memory modules with ultra‑high bandwidth to enable parallel processing. Some systems combine CPUs and GPUs, known as heterogeneous computing.

The performance metric for computers is FLOPS (floating‑point operations per second). As of early 2019, top‑tier supercomputers achieve 143.5 × 10¹⁵ FLOPS (peta‑FLOPS). High‑end gaming desktops are over a million times slower, delivering about 1 × 10⁹ FLOPS. The next milestone, exa‑FLOPS (10¹⁸ FLOPS), would be roughly 1,000 times faster than current peta‑FLOPS machines, requiring around 5 million desktop‑class computers each capable of 200 × 10⁹ FLOPS.

Achieving theoretical FLOPS speeds demands continuous data transfer to processors, so system design must consider data throughput. Memory and interconnects between processing nodes significantly affect data delivery speed.

Terminology

High‑Performance Computing (HPC): a broad class of powerful computing systems ranging from a single CPU + 8 GPUs to world‑class supercomputers.

Supercomputer: the most advanced high‑performance computer, defined by ever‑increasing performance standards.

Heterogeneous Computing: an HPC architecture that optimizes serial (CPU) and parallel (GPU) processing capabilities.

Memory: storage used in HPC systems for fast data access.

Interconnect: system layers that enable communication between processing nodes; supercomputers feature multiple interconnect levels.

Peta‑FLOPS class: supercomputers designed to achieve 10¹⁵ FLOPS.

Exa‑FLOPS class: supercomputers designed to achieve 10¹⁸ FLOPS.

Why Do High‑Performance Computing?

From a system perspective, HPC integrates resources to meet growing performance and functionality demands.

From an application perspective, it decomposes workloads to enable larger‑scale or finer‑grained computation.

It solves scientific and engineering problems through numerical simulation and modeling, which are computation‑intensive, data‑intensive, network‑intensive, or a mix of these.