Nvidia’s latest GTC briefing details the Blackwell and Rubin GPU roadmaps, highlighting massive GPU shipments, new NVLink 6.0 interconnects, 448 Gbps SerDes, and architectural innovations aimed at boosting AI compute performance, efficiency, and scalability across data‑center workloads.
The article uses a whimsical journey to illustrate how token data is dispatched across GPU clusters—detailing functions like get_dispatch_layout, notify_dispatch, and combine_token, showing RDMA and NVLink pathways, performance experiments, and the final verification of token integrity.
The article explains the concept of SuperNode (SuperPod) as a new AI‑focused compute infrastructure, outlines its high‑density integration, ultra‑fast interconnects, and unified resource management, and compares three leading implementations from NVIDIA, Huawei, and the ETH‑X project.
This article examines the architectural differences between Scale‑Out and Scale‑Up networking, compares PCIe, NVLink, UALink, Infiniband and RoCE, and explains why high‑bandwidth, low‑latency GPU interconnects like NVLink are essential for modern AI and HPC workloads.
This article, part of the 2025 AI Network Technology Whitepaper, classifies AI high‑performance networking into Scale‑Up, Scale‑Out, and frontier breakthroughs, then dives deep into NVLink’s evolution, technical features, NVSwitch’s full‑mesh architecture, and the newly opened NVLink Fusion ecosystem.
This article examines the differences between Scale‑Out and Scale‑Up networking in AI compute clusters, comparing PCIe, Ethernet, InfiniBand, NVLink, UALink, and emerging standards like UB‑Mesh, and explains how each technology impacts bandwidth, latency, scalability, and cost for large‑scale model training.
The article analyzes Nvidia's 2023 Blackwell GPU series and GB200 NVL72 architecture, detailing their advanced 3‑4nm manufacturing, redesigned CUDA cores, next‑gen ray‑tracing and DLSS upgrades, massive compute and memory bandwidth gains, NVLink Gen5 improvements, and the diverse GB200 product configurations for high‑performance AI workloads.
This article traces the evolution of NVIDIA's GPU architectures—from Volta's pioneering Tensor Cores through Turing, Ampere, Hopper, and the latest Blackwell—highlighting key innovations such as mixed‑precision support, NVLink, and specialized Tensor Core designs that have dramatically boosted AI training and inference performance.
This article compares the main networking technologies used in large‑scale AI GPU clusters—NVLink, InfiniBand, RoCE Ethernet, and the emerging DDC full‑schedule fabric—examining latency, lossless transmission, congestion control, cost, power and scalability to help engineers choose the optimal solution for training massive language models.
A recent migration of a multimodal image inference system from an internal network to a cloud environment revealed that NVLink bridges dramatically improve multi‑GPU inference speed by reducing inter‑GPU communication overhead, while tensor‑parallel and data‑parallel strategies each have distinct trade‑offs for model deployment.
NVLink Fusion, unveiled at Computex 2025, extends NVIDIA’s NVLink technology to enable high‑bandwidth, low‑latency connections between CPUs and GPUs or third‑party accelerators, offering up to 900 GB/s bandwidth, flexible heterogeneous configurations, ecosystem expansion, performance gains for AI training and inference, and potential cost reductions.
This article analyzes NVIDIA's evolving AI supercomputer architectures—detailing the H100‑based 256‑GPU SuperPod, the GH200‑based 256‑GPU SuperPod with integrated Grace CPU, and the GB200‑based 576‑GPU SuperPod—examining their NVLink and InfiniBand topologies, bandwidth limits, and scalability challenges.
NVLink, NVIDIA's high‑bandwidth interconnect introduced with the P100 GPU, replaces PCIe by offering significantly higher data rates and lower latency for GPU‑GPU and GPU‑CPU communication, and has evolved through multiple generations to support modern AI and high‑performance computing workloads.
This article examines NVIDIA's NVSwitch technology, explaining why it was needed, how it builds on NVLink to overcome PCIe bottlenecks, tracing its evolution from Pascal to the third‑generation design, and detailing its architectural features, scalability, full‑duplex bandwidth, non‑blocking communication, and optimized network topologies for high‑performance AI and HPC systems.
As AI models grow to trillion‑parameter scales, training them demands massive GPU clusters whose performance is increasingly limited by network bandwidth; this article examines why traditional PCIe interconnects become bottlenecks and how NVIDIA's NVLink and NVSwitch technologies dramatically improve multi‑GPU communication and overall system efficiency.
Training today's massive AI foundation models demands extensive GPU resources and sophisticated multi‑GPU communication, making technologies like NVLink and NVSwitch crucial for efficient distributed training, while data‑parallel and model‑parallel strategies together optimize performance across large‑scale hardware clusters.
This article explains the design and implementation of GPU cloud servers, covering data processing pipelines, hardware selection, topology, interconnect technologies, virtualization, multi‑GPU communication methods, and Baidu's practical solutions for both virtualized and bare‑metal instances to boost AI inference and training performance.
With modern large‑language‑model workloads demanding massive parallelism, 8‑GPU servers have become the norm; this article explains the roles of CPUs, compares GPU‑to‑GPU interconnect options—including PCIe direct, PCIe Switch, NVLink, and NVSwitch—detailing their architectures, bandwidths, topologies, and trade‑offs for AI training.
An in‑depth analysis shows that NVLink 3.0 reduces all‑reduce communication latency for Llama 3 70B inference from over 1.8 seconds to under 100 ms, delivering a dramatic speedup compared with PCIe 4.0 and highlighting the critical role of high‑bandwidth interconnects in large‑model deployments.
This article systematically reviews GPU monitoring for large‑scale AI training, covering MFU/HFU definitions, key DCGM metrics, NVLink bandwidth, common failure codes such as Xid and SXid, experimental insights on T4 and H100 GPUs, and practical case studies for diagnosing and mitigating performance drops.
This article breaks down the core components of high‑performance GPU servers—including PCIe switch chips, the evolution of NVLink from version 1.0 to 4.0, NVSwitch architecture, HBM memory tiers, and the nuances of bandwidth units—providing a comprehensive technical foundation for large‑scale model training.
The Venado supercomputer, built for Los Alamos National Laboratory, combines Nvidia Grace CPUs with Hopper GPUs, leverages high‑bandwidth memory and Slingshot interconnects, and targets a balanced 80/20 CPU‑GPU workload split to support demanding AI and HPC applications.
The article provides an in‑depth technical overview of NVIDIA's Grace Hopper superchip, detailing its heterogeneous CPU‑GPU architecture, high‑bandwidth NVLink‑C2C interconnect, unified memory model, programming support, and system‑level scaling features that together deliver unprecedented performance for high‑performance computing and large‑scale AI workloads.
The article analyzes NVIDIA's DGX SuperPOD architectures across three GPU generations—H100, GH200, and GB200—detailing their NVLink/NVSwitch topologies, bandwidth calculations, scalability limits, and the practical challenges of constructing 256‑GPU and 576‑GPU supercomputing clusters.
This article reviews the main GPU/TPU cluster networking solutions—including NVLink, InfiniBand, RoCE Ethernet, and DDC full‑schedule fabrics—examining their latency, loss‑free transmission, congestion control, cost, scalability, and suitability for large‑scale LLM training workloads.
The article analyzes Nvidia's latest AI‑focused hardware and software breakthroughs showcased at ComputeX 2024, detailing how GPU‑CPU hybrid architectures, new libraries, and high‑speed interconnects like NVLink dramatically boost performance while keeping power and cost growth modest.
This article compares Nvidia's NVLink (SXM) and PCIe GPU versions for AI servers, detailing their architectures, bandwidth, power consumption, and ideal use cases, helping readers choose the optimal configuration based on performance needs and budget constraints.
Nvidia’s latest Blackwell GPU, presented by Jensen Huang, promises unprecedented performance and energy efficiency for large‑scale AI models, while the company also showcases accelerated computing, NVLink interconnects, AI‑optimized DGX servers, the NIM platform for rapid LLM deployment, and ambitious projects such as Earth‑2 digital twins and next‑generation embodied AI robots.
This article provides a comprehensive technical overview of large‑scale GPU server architectures, detailing the component topology of 8‑GPU A100/A800 and H100/H800 nodes, explaining storage network cards, NVSwitch interconnects, bandwidth calculations, and the trade‑offs between RoCEv2 and InfiniBand for AI workloads.
This article provides a comprehensive overview of the core components and terminology of large‑scale GPU computing, covering GPU server architecture, PCIe interconnects, NVLink generations, NVSwitch, high‑bandwidth memory (HBM), and bandwidth unit considerations for AI and HPC workloads.
The rapid growth of AI large‑model training and inference is driving unprecedented demand for compute and high‑speed networking, prompting a shift from traditional GPU clusters to super‑pooled intelligent computing centers that must balance multiple intra‑ and inter‑node interconnect solutions such as NVLink, OAM/UBB, InfiniBand and RoCEv2.
The article examines NVIDIA’s Blackwell platform network architecture, detailing the fifth‑generation NVLink, sixth‑generation PCIe, 800 Gb/s InfiniBand and Ethernet adapters, the DGX B200 and GB200 configurations, new IB and Ethernet switches, and the implications of increased optical module demands for large‑scale AI clusters.
This article analyzes the main GPU/TPU cluster networking options—NVLink, InfiniBand, RoCE Ethernet, and DDC full‑schedule fabrics—examining latency, lossless transmission, congestion control, cost, power, and scalability to determine their suitability for large‑scale LLM training.
Based on TrendForce data, AI server shipments are projected to grow at a 12.2% CAGR through 2027, while advances in PCIe switching, retiming chips, and high‑speed GPU interconnects such as NVLink and NVSwitch are reshaping the architecture and performance of next‑generation AI compute platforms.
This article provides an in‑depth technical analysis of NVIDIA NVLink, covering its evolution, the NVSwitch chip, NVLink‑enabled servers and switches, and a performance comparison with InfiniBand networks, highlighting its impact on high‑performance computing and artificial intelligence workloads.
This article provides a detailed technical overview of modern multi‑GPU server architectures—including PCIe switches, NVLink, NVSwitch, and HBM—explaining their hardware topologies, bandwidth characteristics, monitoring methods, and network choices to help engineers design efficient AI training clusters.
This article provides a comprehensive breakdown of high‑performance GPU server infrastructure, covering PCIe generations, NVLink evolution, NVSwitch and NVLink switches, HBM memory technologies, and bandwidth measurement units, helping readers understand the hardware connections and performance considerations essential for large‑scale model training.
The article analyses NVIDIA’s new Blackwell platform, focusing on the GB200 NVL72 GPU and its 10 TB/s NVLink‑C2C interconnect, detailing massive training and inference speedups, rack‑level DGX SuperPOD architecture, copper‑cable trends, and the broader impact on AI‑driven data‑center workloads.
The article analyses the historical evolution of Nvidia's NVLink and NVLink C2C interconnect technologies, compares them with PCIe, Ethernet and InfiniBand, and uses these trends to predict future AI‑chip architectures such as the B100 and X100 GPUs, highlighting design trade‑offs and packaging challenges.
This article analyzes the evolving data‑center network architectures for large AI clusters, detailing leaf‑spine and Fat‑Tree designs, NVLink interconnects, and calculating the precise optical‑module requirements for NVIDIA A100, H100, and GH200 deployments, while also comparing industry examples from Meta, AWS, and Google.
This article compares NVIDIA's NVLink (SXM) and PCIe GPU versions for AI servers, detailing their architectures, bandwidth, power consumption, and ideal use cases, and provides guidance on selecting the right GPU based on workload size, flexibility, and cost considerations.
The article provides an in‑depth technical analysis of Nvidia’s NVLink C2C interconnect, comparing its latency, bandwidth, power efficiency, density and cost against traditional SerDes solutions and examining its role in building SuperChip architectures with Grace CPUs and Hopper GPUs.
The article analyzes Nvidia’s 2023 investor‑meeting roadmap, revealing an annual GPU release cadence with H200, B100 and X100 chips, a unified "One Architecture" strategy spanning x86 and ARM, accelerated interconnects like NVLink‑C2C, and competitive pressures shaping its AI ecosystem.
This article examines the key GPU/TPU cluster networking options—NVLink, InfiniBand, RoCE Ethernet, and emerging DDC full‑scheduling fabrics—detailing their latency, loss‑less transmission, congestion control, cost, power, and scalability considerations for large‑scale AI training deployments.
The article provides an in‑depth industry analysis of AI server hardware, covering shipment forecasts, NVIDIA DGX H100 specifications, the role of PCIe switches and retimers, the evolution of NVLink/NVSwitch, and the market dynamics of DDR4/DDR5 memory interface chips.
This article examines the main GPU/TPU cluster networking options—including NVLink, InfiniBand, RoCE Ethernet Fabric, and DDC full‑schedule networks—explaining their latency, loss‑less transmission, congestion control, cost, scalability, and suitability for large‑scale LLM training workloads.
The article surveys the AI compute ecosystem, explaining why CPUs are unsuitable for AI workloads, how heterogeneous CPU‑plus‑accelerator designs dominate, and detailing the evolution of NVIDIA GPUs, Tensor Cores, memory technologies, and inter‑GPU networking that enable large‑scale model training.
The article provides a detailed overview of NVIDIA's Grace CPU Superchip, describing its Arm‑based architecture, NVLink‑C2C interconnect, scalable coherency fabric, high‑bandwidth LPDDR5X memory, extensive I/O options, and software ecosystem, highlighting its suitability for HPC and AI workloads.
The article explains GPU evolution, its integration with CPUs, interconnect technologies like PCIe and NVLink, market shares of NVIDIA, AMD and Intel, AI accelerator types (GPU, FPGA, ASIC), and the roles of training and inference in cloud AI, while also promoting a paid 182‑page PPT resource.
The article introduces NVIDIA’s third‑generation NVLink technology, detailing its high‑bandwidth GPU‑GPU and GPU‑CPU interconnect, key architectural breakthroughs such as the Ampere‑based A100 GPU, multi‑instance GPU, and NVSwitch, and discusses its impact on AI, HPC, and graphics workloads.
This article explains NVIDIA's NVLink interconnect technology, covering its history, protocol layers, bandwidth advantages over PCIe, topologies such as the HGX-1/DGX-1 mesh, the NVSwitch extension, and performance gains for deep‑learning and high‑performance computing workloads.
This article explains the background of GPU communication, details NVIDIA's GPUDirect and its Peer‑to‑Peer features, discusses virtualization challenges, and presents performance measurements on an Alibaba Cloud GN5 instance showing latency reduction and near‑linear scaling for deep‑learning workloads.
