Recent Trends and Hot Topics in Storage Technologies: Open‑Channel SSD, NVM, Learned Indexes, LSM‑Tree Optimizations, and Crash Consistency

The author, after reading over 30 storage‑related papers in six months, presents a concise overview of emerging and evolving topics in storage systems, ranging from novel hardware interfaces to software optimizations.

Open‑Channel SSD provides a bare‑metal SSD without an internal Flash Translation Layer (FTL), allowing custom FTL implementations in the OS or applications. Using RocksDB as an example, the article explains how eliminating the file‑system and FTL layers can collapse three separate write‑ahead logs into one, dramatically reducing write amplification. It also discusses alignment of SSTables with SSD blocks, IO isolation via data placement across NAND chips, and the Linux LightNVM/pblk subsystem that exposes Open‑Channel SSDs to existing block‑layer software.

Non‑Volatile Memory (NVM) offers byte‑addressable, persistent storage with ~10 ns latency, bridging the gap between DRAM and SSD. The article outlines two usage models—persistent transactional memory and DAX‑enabled block/file interfaces (e.g., Ext4‑DAX, PMFS, NOVA)—and highlights programming challenges illustrated by a linked‑list insertion function:

void list_add_tail(struct cds_list_head *newp, struct cds_list_head *head) {
    head->prev->next = newp;
    newp->next = head;
    newp->prev = head->prev;
    head->prev = newp;
}

Because NVM is non‑volatile, power loss during such updates can leave data structures inconsistent, requiring explicit transaction mechanisms and, in distributed settings, tight integration with RDMA (e.g., the Octopus project).

Machine Learning for Systems is introduced through the seminal "The Case for Learned Index Structures" paper, which treats an index as a predictive model (often a neural network) to replace traditional B‑tree structures, reducing memory footprint and lookup latency. Limitations include sensitivity to data‑distribution changes, necessitating hybrid schemes that fall back to conventional indexes for new data.

LSM‑Tree Optimizations are surveyed: WiscKey separates keys from values, storing values in a log‑structured value store to cut read/write amplification at the cost of range‑query performance; PebblesDB introduces Guard‑based fragmentation (FLSM) to avoid reading lower levels during compaction, achieving up to 6.7× reduction in write amplification while similarly impacting range queries.

Crash Consistency focuses on formally verified file systems such as MIT's FSCQ (Coq‑based) and Washington University's Yggdrasil (Z3‑based), noting the high engineering effort required and current limitations like lack of multi‑threading support.

Industry Progress includes the shift from HBA‑based storage to hypervisor‑mediated I/O (vhost‑blk/net), container‑native storage orchestration via CSI and rook.io, Linux kernel enhancements (AIO, block‑layer error handling, Kyber scheduler), and the broader move toward software‑defined storage solutions by vendors such as Nutanix and Kaminario.

The article concludes with a brief invitation to explore these topics further and a short promotional note about SmartX, a leading distributed storage vendor.