Understanding Linux Virtual File Systems and NVDIMM Persistent Memory

1. Linux Virtual File System Overview

In Linux, everything is treated as a file, including directories, devices, sockets and pipes. The term "file system" can refer to the on‑disk data structures, a formatted storage device, or the kernel module that manages files.

The Linux VFS layer abstracts different file system types, providing a uniform API for user programs.

Cache vs. buffer distinction: "cache" refers to page cache for files, while "buffer" is block cache for block devices.

2. Next‑Generation Storage NVDIMM

2.1 Types of NVDIMM

NVDIMM‑N combines DRAM and flash, offering byte‑addressable memory with a backup power source; NVDIMM‑F is flash‑only block storage; NVDIMM‑P (under development) aims to provide both block and byte access.

Key differences between NAND and NOR flash include capacity, erase cycles, read/write speed, and bad‑block handling.

2.2 Why Flash‑Specific File Systems?

Bad‑block detection and wear‑leveling are required for NAND.

Flash has limited erase cycles, so wear‑leveling spreads wear.

2.3 NVDIMM‑P Hardware Support

Intel Optane™ DC Persistent Memory (based on 3D XPoint) is a commercial NVDIMM‑P implementation.

2.4 File Systems for Persistent Memory

Examples include NOVA (PMEM‑native), ZUFS (zero‑copy user file system), and traditional file systems with DAX support (XFS, EXT4).

3. NVDIMM Implementation in Linux

3.1 Persistent Memory (PMEM)

PMEM provides byte‑addressable persistent storage. It can be used in two modes: DAX (direct access, bypassing page cache) or BTT (block translation table for sector‑based access).

3.2 Management Tools

The ndctl utility (and its library libndctl) manages NVDIMM devices, regions and namespaces. Common sub‑commands include list, create-namespace, destroy-namespace, enable-namespace, disable-namespace, etc.

ndctl list --dimms

ndctl list --regions

3.3 Creating a DAX PMEM Namespace

Example:

ndctl create-namespace --type=pmem --mode=fsdax --map=memory

This creates /dev/pmem3, which can be formatted and mounted with DAX:

mkfs.xfs /dev/pmem3
mount -o dax /dev/pmem3 /mnt/pmem3

After mounting, mmap() on files directly accesses persistent memory without page cache.

3.4 Creating a BTT PMEM Namespace

Example:

ndctl create-namespace --type=pmem --mode=sector

This creates /dev/pmem3s (the trailing s indicates sector mode). It can be formatted like any block device.

3.5 Simulating Persistent Memory with memmap

On systems without real NVDIMM, a region of RAM can be reserved using the kernel memmap parameter (e.g., memmap=4G!4G) and then accessed as /dev/pmem0. The steps include adding the parameter to GRUB, updating GRUB, and rebooting.

After reboot, the device can be formatted and mounted with DAX:

mkfs.ext4 /dev/pmem0
mount -o dax /dev/pmem0 /mnt/pmemdir

4. References

Linux Virtual File System Introduction

Persistent Memory Documentation

Next‑Generation Storage: NVDIMM (Part 1 & 2)

How to Emulate Persistent Memory on Intel Architecture Servers

Source: FreeOA, author: 阿炯 ( https://reurl.cc/8y3nEM )