metadata warehouse

Bilibili Tech

Oct 29, 2024 · Big Data

Bilibili One‑Stop Big Data Cluster Management Platform (BMR): Architecture, Modules, and Future Outlook

Bilibili's One‑Stop Big Data Cluster Management Platform (BMR) unifies cluster, metadata, intelligent operations, and custom managers to oversee 50+ services, 10,000 machines, exabyte storage, and millions of cores, using cloud‑native containers, fault prediction, and resource‑sharing techniques to boost efficiency, stability, and cost savings.

BMRbig datacluster management

0 likes · 17 min read

Bilibili One‑Stop Big Data Cluster Management Platform (BMR): Architecture, Modules, and Future Outlook

DataFunSummit

Dec 11, 2023 · Big Data

Design and Implementation of a Big Data Metadata Warehouse at Bilibili

This article presents Bilibili's big‑data metadata warehouse, covering its background, technology selection between data‑lake and data‑warehouse solutions, the architecture built on Prometheus, StarRocks, Flink and Routine Load, performance comparisons, diagnostic system design, and future development plans.

DiagnosticsFlinkStarRocks

0 likes · 20 min read

Design and Implementation of a Big Data Metadata Warehouse at Bilibili