Resource Overcommit

DataFunSummit

Feb 16, 2025 · Big Data

Bilibili Big Data Task Migration to Cloud‑Native Kubernetes Using Volcano Scheduler

This article shares Bilibili’s experience migrating its offline big‑data workloads to a cloud‑native Kubernetes environment using the Volcano scheduler, covering migration background, scheduler adaptation, hierarchical queue implementation, over‑commit framework (Amiyad), and future work to improve performance and resource utilization.

Big DataKubernetesResource Overcommit

0 likes · 15 min read

Bilibili Big Data Task Migration to Cloud‑Native Kubernetes Using Volcano Scheduler

Ops Development Stories

Jan 14, 2025 · Cloud Native

Dynamic Local Disk Allocation & Resource Overcommit in KubeVirt using OpenEBS‑LVM

This guide explains how to replace KubeVirt's local‑storage with OpenEBS‑LVM for dynamic PV allocation, configure CPU/memory overcommit ratios, perform hot‑plug upgrades, expand disks online, and set node affinity and fixed IPs, providing full YAML examples and reference links.

Dynamic StorageKubeVirtKubernetes

0 likes · 6 min read

Dynamic Local Disk Allocation & Resource Overcommit in KubeVirt using OpenEBS‑LVM

vivo Internet Technology

Dec 20, 2023 · Cloud Native

Resource Overcommit Strategies in Vivo Container Platform: Static and Dynamic Approaches

Vivo’s container platform combats oversized resource requests by first applying static coefficient‑based overcommit at deployment and then using a dynamic recommender that continuously gathers usage metrics, builds exponential histograms with a half‑life sliding‑window model, and adjusts CPU (and optionally memory) requests, improving packing efficiency, reducing billing, and boosting CPU utilization by up to eight percent while maintaining HPA accuracy.

HPAKubernetesResource Overcommit

0 likes · 15 min read

Resource Overcommit Strategies in Vivo Container Platform: Static and Dynamic Approaches

DataFunSummit

Sep 2, 2023 · Big Data

Practical Experience of Bilibili's Big Data Cluster Mixed Deployment Architecture

This article details Bilibili's offline big‑data cluster challenges, the mixed‑deployment architecture that combines offline and online resources, the Amiya service's over‑commit and eviction mechanisms, performance optimizations, monitoring strategies, and future plans to further improve resource utilization and scheduling.

AmiyaBig DataBilibili

0 likes · 14 min read

Practical Experience of Bilibili's Big Data Cluster Mixed Deployment Architecture

Bilibili Tech

May 23, 2023 · Big Data

Amiya: Dynamic Overcommit Component for Bilibili Offline Big Data Cluster

Amiya, a self‑developed dynamic over‑commit component for Bilibili’s offline big‑data cluster, inflates reported resources on under‑utilized nodes and adjusts them when load rises, adding roughly 683 TB of memory and 137 k vCores, boosting per‑node memory by 15 % and CPU usage by over 20 % while keeping eviction rates below 3 %.

AmiyaBig DataBilibili

0 likes · 22 min read

Amiya: Dynamic Overcommit Component for Bilibili Offline Big Data Cluster