How Mantle Breaks the Hierarchical Namespace Bottleneck in Cloud Object Storage

Congratulations to Baidu's Canghai Storage team and their collaborators from University of Science and Technology of China and Tsinghua University for having their paper "Mantle: Efficient Hierarchical Metadata Management for Cloud Object Storage Services" accepted at the top‑tier operating systems conference SOSP'25.

SOSP, together with OSDI, is one of the most prestigious conferences in operating systems and system software, with an acceptance rate of only 17.7% (65 out of 368 submissions), highlighting the cutting‑edge nature of this work.

Traditional Object Storage Challenges: Hierarchical Namespace Hits Single‑Node Bottlenecks

For years HDFS has been the de‑facto storage for big data, but its three‑copy model, single‑NameNode design, and complex operations limit scalability and increase cost, especially for AI‑era workloads that require billions of files per bucket.

Object storage, with its low cost, unlimited scalability, and cloud‑native, zero‑ops advantages, is becoming the foundation of next‑generation data lakes, yet its flat namespace cannot efficiently support file‑system‑style operations such as directory traversal, recursive delete, or path rename, leading to performance far below HDFS in many scenarios.

The industry agrees that adding hierarchical namespace capabilities to object storage is inevitable, but existing solutions still suffer from single‑node performance bottlenecks and do not achieve true scalability.

Two Core Challenges: Why Is High‑Performance Distributed Hierarchical Namespace So Hard?

Long‑path parsing overhead: Resolving deep paths (e.g., /A/B/C/D/file.txt) requires multiple network round‑trips, causing high latency; client‑side caching is difficult in stateless RESTful APIs, and parallel parsing can cause resource contention under high concurrency.

Frequent distributed transaction conflicts: Thousands of concurrent tasks operating on the same directory lead to massive read/write conflicts and retries, sharply dropping throughput; even relaxed isolation levels cannot fully mitigate cross‑directory rename conflicts.

Consequently, the industry lacks a solution that combines the low‑cost, elastic nature of object storage with HDFS‑level performance for hierarchical namespaces, hindering the vision of object storage as the backbone of data lakes.

Mantle: The First Global Breakthrough – Merging File‑System Semantics with Object‑Storage Advantages

Mantle is the world’s first publicly disclosed, fully functional distributed hierarchical namespace system that resolves the two challenges above and has been proven in large‑scale production for over two years on Baidu Intelligent Cloud Object Storage (BOS).

By fusing the rich semantics of traditional file systems with the cost‑effective, scalable, cloud‑native characteristics of object storage, Mantle achieves unprecedented performance: metadata access latency is reduced by 6.6% to 99.1% compared with state‑of‑the‑art systems, and throughput improves by up to 115×.

In real workloads, Mantle shortens Spark interactive analysis job completion times by 63.3%–93.3% and AI‑driven audio preprocessing tasks by 38.5%–47.7%.

Today, Mantle powers critical big‑data analytics, AI, and autonomous driving services for numerous cloud customers, providing a robust and efficient storage foundation for the AI era.