How NVIDIA Grace Hopper Superchip Redefines HPC and AI Performance

Grace Hopper Superchip Overview

NVIDIA Grace Hopper combines the pioneering performance of the Hopper GPU with the versatile Grace CPU on a single superchip, linked by high‑bandwidth, low‑latency NVLink‑C2C (Chip‑to‑Chip) interconnect and a NVLink Switch system, enabling a unified programming model for both HPC and AI workloads.

Key Architectural Innovations

Grace CPU : Up to 72 Arm Neoverse V2 cores, Armv9.0‑A ISA, 4×128‑bit SIMD units per core, 512 GB LPDDR5X memory, 3.2 TB/s memory bandwidth, 117 MB L3 cache, and SCF (Scalable Consistency Fabric) with distributed cache.

Hopper GPU : NVLink 4, PCIe 5, 60 MB L2 cache, up to 96 GB HBM3 (3000 GB/s), 144 SMs with fourth‑generation Tensor cores, transformer engine, DPX, and 3× FP32/FP64 performance over A100.

NVLink‑C2C : 900 GB/s total bidirectional bandwidth (450 GB/s per direction), hardware‑coherent memory, address‑translation services (ATS) for unified virtual memory, enabling GPUs to directly access CPU memory without page migration.

NVLink Switch System : Connects up to 256 Grace Hopper chips, providing up to 115.2 TB/s aggregate bandwidth and addressing up to 150 TB of system memory across the fabric.

Extended GPU Memory (EGM) : Allows GPUs to access all system memory (CPU + HBM3) at up to 450 GB/s, supporting massive datasets for AI training.

Unified Programming Model

The platform supports ISO C++, ISO Fortran, Python, and standard parallel models such as OpenACC, OpenMP, CUDA C++, and CUDA Fortran. NVIDIA’s CUDA LLVM Compiler API lets developers use their preferred language while benefiting from the same code generation quality and optimizations as native CUDA tools.

Hardware coherence via NVLink‑C2C and ATS provides a single process page table shared by CPU and GPU threads, enabling transparent access to CPU heap, GPU global memory, and mapped files without explicit data movement. Scoped atomic operations and fine‑grained synchronization are fully supported.

System‑Level Scaling with HGX Grace Hopper

Each HGX Grace Hopper node integrates a Grace Hopper chip with BlueField‑3 NICs or optional NVLink switches, supporting both air and liquid cooling with up to 1000 W TDP. Combined with InfiniBand, the system scales AI and HPC workloads without network bottlenecks, offering up to 100 GB/s per node and up to 115.2 TB/s across 256‑chip configurations.

Performance Impact

Grace Hopper delivers up to 7× the bandwidth of x16 PCIe Gen5, 2.5× higher CPU speed and 4× lower power consumption compared to AMD Milan, and up to 9× AI training acceleration and 30× inference acceleration over A100 for large language models.

Reference

All technical details are drawn from the NVIDIA Grace Hopper Architecture whitepaper.