BestHub
Sign in
Tag

GPU sharing

0 views collected around this technical thread.

ByteDance Cloud Native
ByteDance Cloud Native
Aug 12, 2024 · Cloud Native

How mGPU Enables Efficient GPU Sharing for AI Workloads in Cloud‑Native Environments

The article explains the mGPU solution from Volcano Engine, detailing its kernel‑level GPU virtualization, container runtime hooks, and scheduling mechanisms that allow multiple containers to share a single NVIDIA GPU with isolated compute and memory resources, achieving near‑lossless performance and up to 50% higher utilization for AI tasks.

AI workloadsGPU sharingResource Isolation
0 likes · 9 min read
How mGPU Enables Efficient GPU Sharing for AI Workloads in Cloud‑Native Environments
ByteDance SYS Tech
ByteDance SYS Tech
Aug 12, 2024 · Cloud Native

How mGPU Enables Efficient GPU Sharing for AI Workloads

This article explains the mGPU solution that virtualizes NVIDIA GPUs for containers, detailing its driver architecture, compute and memory isolation mechanisms, performance benchmarks on ResNet‑50 inference, and how it boosts GPU utilization by over 50% for AI and high‑performance computing tasks.

AI accelerationContainer OrchestrationGPU sharing
0 likes · 10 min read
How mGPU Enables Efficient GPU Sharing for AI Workloads