GPFS Technical Practice Sharing and Building‑Block Design Overview

IBM GPFS (General Parallel File System) is a distributed, shared, parallel cluster file system that allows simultaneous access to a single or multiple file systems from many nodes, running on AIX, Linux, Windows and on Power or x86 architectures, and is widely used in HPC environments.

GPFS clusters consist of three main components: the management command set, the GPFS kernel extensions, and multithreaded daemon processes.

GPFS Management Command Set provides scripts to control GPFS operations and configuration; commands can be executed on any node, with the cluster automatically routing requests to the appropriate node.

GPFS Kernel Extensions register GPFS as a native file system and provide an interface between the OS vnode/VFS layer and GPFS.

GPFS Daemons handle all I/O and buffer management, including read‑ahead and write‑behind, and protect I/O with token management to ensure data consistency across nodes.

GPFS NSD (Network Shared Disk) components expose storage to applications, either via direct physical connections or through NSD servers; up to eight NSD servers can be assigned per NSD, with fail‑over handling.

When a GPFS file system is mounted, the daemons discover available NSDs, preferring local block devices (SAN, SCSI, IDE) before NSD servers.

The relationship among GPFS components is summarized as follows: a Node is an independent OS instance, an NSD is a storage device visible to the GPFS cluster, an NSD server provides I/O access to a specific NSD, and an Application Node runs workloads that mount the GPFS file system.

GPFS supports configurations where some nodes connect directly to disks while others access them through dedicated nodes, a cost‑effective topology common in large HPC clusters. Nodes that serve disks are called NSD servers, and the nodes that consume those disks are GPFS clients.

Each GPFS cluster elects a quorum node as the cluster management server, responsible for monitoring disk health, detecting node failures, and coordinating recovery.

Every file server has a file‑system management server selected by the cluster manager; these servers handle configuration, disk space allocation, token management, quota enforcement, and high‑availability coordination.

To maintain metadata consistency during writes, GPFS introduces a Meta node that aggregates and merges metadata updates from all nodes; the Meta node is dynamically chosen per file based on the longest open duration.

GPFS offers several networking models, each suited to different workloads: SAN Model, Network Shared Disk (NSD) Server Model, Shared‑Nothing Cluster (SNC) Model, Remote Cluster Mount Model, and Hybrid configurations that combine multiple models.

Building‑Block design for GPFS focuses on two primary types: Capacity Building Blocks for maximal storage and Performance Building Blocks for maximal throughput; both can be combined to achieve near‑linear scaling of performance and capacity.

Configurable options include compute network topology (default InfiniBand, optional 10 GE), storage disk types and enclosures, number of quorum nodes, and data replication factors.

When deploying new Building Blocks, it is recommended to keep configurations uniform across blocks to avoid bottlenecks caused by GPFS’s wide‑striping strategy, and to plan for future expansion by using existing blocks as templates.

For expanding an existing GPFS deployment, new Building Blocks should be added online as additional NSDs, allowing seamless capacity growth without interrupting running workloads.

For the full GPFS analysis, see the original link titled “High‑Performance Computing (HPC) Technology, Solutions, and Industry Comprehensive Analysis”.

Second Part – Book Giveaway

Teacher An Xiaohui’s new book “Programmer’s Growth Course” is now published, with three copies available for free, including postage. The book compiles over a decade of development and management experience and offers practical guidance on developer value growth and career choices.

The Architecture Alliance apologizes for limited online activities due to work commitments and thanks readers for their support; the giveaway will be awarded to the three commenters with the most likes by the deadline.

Please search for “ICT_Architect” or scan the QR code below to follow the public account for more content.

Focused on building a heartfelt technology sharing platform