NVIDIA Merlin: Product Overview, Models, Distributed Embeddings, Hierarchical KV and Parameter Server

NVIDIA Merlin is a comprehensive framework for building and deploying recommendation systems. It provides a high‑level library (Merlin Models & Systems) that bundles popular recommendation models such as DLRM, DCN, and YouTube DNN, and integrates feature‑engineering tools like NVTABULAR to simplify ETL, training, and deployment pipelines.

The training stack includes native HugeCTR, Merlin Data Loader for efficient data ingestion, and the TensorFlow Distributed Embedding (TFDE) plugin, which accelerates embedding lookups by distributing them across GPUs and reducing communication overhead. Benchmarks show speed‑ups of up to 600× for embedding‑heavy workloads.

At the lowest level, Merlin Hierarchical‑KV (HKV) is a C++ key‑value store optimized for recommendation workloads. It supports unified CPU/GPU memory, high performance, eviction policies (LRU, LFU, custom), and an API similar to std::unordered_map , making it easy to integrate into existing training frameworks.

For inference, Merlin Hierarchical Parameter Server (HPS) provides a GPU‑resident cache for hot features, falling back to CPU memory or external back‑ends (e.g., RocksDB, HDFS) when needed. HPS integrates with Triton and offers plugins for TensorFlow, PyTorch, and Triton Ensemble, delivering low‑latency inference across a range of batch sizes.

The overall design emphasizes ease of use: users can switch models with a single function call, combine Merlin Models with NVTABULAR without code changes, and employ continuous training pipelines that export incremental models via Kafka for near‑real‑time serving. Together, these components enable scalable, high‑performance recommendation systems on modern GPU infrastructure.