multi-protocol | BestHub

DataFunSummit

Sep 12, 2024 · Cloud Native

Design and Implementation of a Next‑Generation Multi‑Protocol Unstructured Storage System for Machine Learning

This article presents the challenges of storing massive machine‑learning datasets, evaluates existing storage solutions, and details the design of OrangeFS—a cloud‑native, multi‑protocol, multi‑tenant unstructured storage system that integrates object and file interfaces, optimizes metadata services, supports hot upgrades, and provides robust scalability and reliability for AI workloads.

cloud nativedistributed systemshigh performance

0 likes · 24 min read

Design and Implementation of a Next‑Generation Multi‑Protocol Unstructured Storage System for Machine Learning

Didi Tech

Sep 19, 2023 · Cloud Native

OrangeFS: A Cloud‑Native Multi‑Protocol Distributed Data Lake Storage System

OrangeFS is Didi’s cloud‑native, multi‑protocol distributed data‑lake storage system that unifies POSIX, S3 and HDFS access on a single logical hierarchy, integrates with Kubernetes via a CSI plugin, supports on‑premise and public‑cloud backends, provides multi‑tenant isolation, and dramatically improves elasticity, utilization and latency for petabyte‑scale workloads such as ride‑hailing logs, machine‑learning training, finance and analytics.

CSIFUSEKubernetes

0 likes · 17 min read

OrangeFS: A Cloud‑Native Multi‑Protocol Distributed Data Lake Storage System