Unlocking NFS & pNFS: How Parallel File Systems Boost Performance

Network File System (NFS) enables a computer to share its physical file system with other machines on the same network, making the shared file system appear as local storage to applications running on NFS clients.

The typical deployment involves Linux as the NFS server exporting one or more file systems, while macOS and Windows act as NFS clients that mount these shared file systems.

Simple NFS Configuration

NFS hides the underlying server file system implementation and type; read and write operations from clients traverse to the server, which fulfills data requests and updates metadata such as permissions and timestamps.

NFS is powerful, running over TCP or UDP, easy to manage, and the latest version, NFS v4, improves security and interoperability between Windows and UNIX‑like systems while providing better locking semantics.

However, traditional NFS struggles with high‑performance computing (HPC) workloads that involve massive files and thousands of client nodes, where server bandwidth, storage capacity, and CPU become bottlenecks.

The next evolution, NFS v4.1 with pNFS (parallel NFS), separates data transfer from the metadata server. Clients obtain layout information and then read/write data directly from the storage system, bypassing the NFS server and eliminating it as a performance choke point.

pNFS Conceptual Architecture

Like NFS, a pNFS server exports a file system and maintains standard metadata. Clients mount the exported file system, but during read/write operations they communicate directly with the underlying storage system, while the pNFS server only handles metadata updates.

This design retains all NFS advantages while improving scalability and performance; expanding storage capacity has minimal impact on client configuration, and more clients can be added to increase compute capability.

pNFS Detailed Mechanics

pNFS operates through three protocols: a layout protocol that describes how file data is distributed across storage devices, a storage access protocol that defines how clients access the data, and a control protocol that synchronizes metadata between the metadata server and storage servers.

Client requests a layout for a file.

Client opens the file on the metadata server to obtain access.

With the layout, the client accesses the data directly from the storage system using the appropriate storage access protocol.

If the client modifies the file, it updates the layout instance and commits changes back to the metadata server.

When the client no longer needs the file, it returns the layout to the server and closes the file.

Read operations consist of a series of steps: the client sends a LOOKUP+OPEN request, receives a file handle, issues a LAYOUTGET to obtain the layout, performs one or more READ requests directly to the storage, and finally sends LAYOUTRETURN. If a layout becomes stale, the server issues CB_LAYOUTRECALL to invalidate it.

Write operations follow the same flow but require a LAYOUTCOMMIT before the client can publish changes to the metadata server.

Layouts can be cached on each client to further boost performance; the server can limit write layout byte ranges to enforce quotas or reduce allocation overhead.

Three primary storage layout types are supported:

File storage – traditional NFS servers that distribute file fragments across multiple servers.

Block storage – typically implemented via a Storage Area Network (SAN) using SCSI block commands.

Object storage – uses object IDs instead of file handles, offering more complex file segmentation.

Regardless of layout type, pNFS references servers by a unique ID rather than hostnames or volume names.

Choosing the best storage technology depends on budget, speed, scalability, and simplicity.

Current State of pNFS

As of November 2008, the NFSv4.1 RFC draft was in its final stage, with a two‑month comment period before formal publication. The draft provides a solid foundation for product development, and open‑source implementations were expected within months, with early adopters able to build simple pNFS networks.

pNFS’s predecessor technology has already been deployed in top‑ranking supercomputers. For example, the Roadrunner system at Los Alamos National Laboratory, built on a Panasas parallel file system, achieved petaflop performance and demonstrated transfer rates of 1.6 GB/s in 2006, scaling to several hundred GB/s by 2008—far surpassing traditional NFS peaks.

NFSv4.1 and pNFS represent the most significant evolution of a technology that originated in the 1980s, now ready to deliver super‑storage speeds for modern high‑performance computing.