Tagged articles
GPU scheduling
30 articles · Page 1 of 1
DeepSeek has open-sourced its production inference engine DeepInfer and token codec component deepseek-recipe, revealing a single-cluster throughput of 155 trillion tokens per day and $38 per GPU per hour revenue, signaling further reductions in per-token inference pricing for developers.
AI infrastructureDeepInferDeepSeek
0 likes · 3 min read

Running AI workloads on Kubernetes introduces five production‑grade challenges—complex GPU and accelerator management, workload‑aware scheduling, inference autoscaling beyond CPU metrics, multi‑layer observability, and Day 2 governance—requiring platform teams to extend their capabilities beyond traditional container operations.
AI workloadsDay 2 operationsGPU scheduling
0 likes · 10 min read

This guide explains how to schedule large‑model inference pods onto GPU nodes that meet exact hardware requirements—such as A100 80 GB cards, specific node pools, and zones—by converting those needs into Kubernetes node‑affinity, taint, and topology constraints, verifying the deployment, monitoring its health, and safely rolling out or rolling back changes.
GPU schedulingKubernetesNode Affinity
0 likes · 23 min read

The article presents a detailed, step‑by‑step analysis of the L0 resource layer that underpins AI infrastructure, covering GPU scheduling, multi‑tier storage, low‑latency networking, core architectural components, key technologies such as MIG, Volcano, Kueue and RDMA, practical implementation patterns, quantitative acceptance criteria, and common pitfalls with best‑practice mitigations.
AI infrastructureGPU schedulingJuiceFS
0 likes · 26 min read

This article presents a detailed, step‑by‑step analysis of deploying the high‑performance vLLM inference engine on Kubernetes, covering GPU memory management, tensor parallelism, quantization choices, continuous batching, and automated scaling with HPA/KEDA to achieve low latency and high throughput for large language models.
DockerGPU schedulingKubernetes
0 likes · 49 min read

The article argues that AI agents are rapidly automating traditional DevOps tasks such as IaC, CI/CD pipeline maintenance, and incident triage, and outlines three emerging career paths—AI infrastructure/LLMOps engineering, platform engineering, and AI‑focused SRE—to stay relevant in 2026.
AI OpsGPU schedulingInfrastructure as Code
0 likes · 7 min read

From its 2015 debut as a stateless microservice orchestrator, Kubernetes now powers large‑scale data pipelines, distributed training, high‑throughput inference, and autonomous agents, unifying these workloads on a single platform while addressing resource coordination, multi‑cluster scheduling, and GPU economics.
AIGPU schedulingKubernetes
0 likes · 10 min read

This article examines the limitations of Vibe Coding for large AI infrastructure, proposes a text‑driven, document‑centric workflow, and presents a time‑multiplexed GPU scheduling solution that dramatically improves rollout throughput and reduces timeouts in large‑scale Agentic RL training.
Design DocumentsGPU schedulingTime Multiplexing
0 likes · 21 min read

The HBox scheduling platform tackles large‑scale AI cluster challenges by introducing a three‑pool resource model, priority‑based preemptive scheduling, network‑topology and NUMA‑aware dispatch, and GPU virtualization techniques like MIG and vGPU, dramatically improving GPU utilization, SLA guarantees, and overall cluster efficiency.
AI clustersGPU schedulingGPU virtualization
0 likes · 24 min read

In the era of large AI models, ACK One’s multi‑cluster fleet provides inventory‑aware elastic scheduling, cluster‑level priority dispatch, and hybrid‑cloud strategies to maximize GPU utilization, ensure business continuity, and reduce costs across regions and on‑premise data centers.
ACK OneAI workloadGPU scheduling
0 likes · 11 min read

ACK One Fleet, Alibaba Cloud's enterprise multi‑cluster solution, provides inventory‑aware elastic GPU scheduling, cross‑region resource sharing, multi‑cluster HPA and model distribution, allowing AI inference and training workloads to scale efficiently, reduce costs, and maximize GPU utilization.
AIGPU schedulingHPA
0 likes · 12 min read

Alibaba Cloud’s container experts unveiled a comprehensive, AI‑focused upgrade across its cloud‑native stack—introducing AMD compute, dynamic scaling, AI‑native scheduling, secure execution environments, and advanced GPU profiling—to make containers the native foundation for AI workloads and accelerate enterprise AI adoption.
AI infrastructureGPU schedulingcontainer computing
0 likes · 9 min read

360’s AI platform migrated its GPU clusters to a cloud‑native architecture and adopted the Volcano scheduler, achieving over 45% GPU utilization, less than 7% fragmentation, and more than 1000000 scheduled Pods, while leveraging flexible plugins, hierarchical queues, and resource pooling to optimize AI and big‑data workloads.
AI PlatformGPU schedulingKubernetes
0 likes · 13 min read

This guide explains how Alibaba Cloud's ACK One registered clusters can provide multi‑region, serverless GPU compute for AI workloads by using Kubernetes‑compatible labels, the ack‑co‑scheduler, and ResourcePolicy objects to dynamically allocate resources across regions, with step‑by‑step configuration examples.
ACK OneGPU schedulingserverless
0 likes · 11 min read

The Koordinator v1.6 release introduces a suite of innovations—including GPU topology‑aware scheduling, end‑to‑end GPU & RDMA joint allocation, strong GPU isolation, differentiated GPU scoring, fine‑grained resource reservation, mixed‑workload QoS, and extensive scheduler and rescheduler optimizations—to efficiently manage heterogeneous resources in Kubernetes clusters for AI and high‑performance computing workloads.
GPU schedulingHeterogeneous ResourcesKoordinator
0 likes · 24 min read

This guide walks through installing the NVIDIA container toolkit, configuring Docker to use the NVIDIA runtime, verifying GPU access, deploying the NVIDIA device plugin in Kubernetes, labeling GPU nodes, and running a GPU‑accelerated FFmpeg pod to confirm successful GPU integration.
Container ToolkitDockerGPU
0 likes · 12 min read

The article introduces NanoFlow, a novel service framework that leverages intra‑device parallelism, operation‑based pipelining, and async scheduling to significantly improve large language model serving throughput, achieving up to 1.91× higher performance while integrating with Alibaba Cloud PAI.
Alibaba Cloud PAIGPU schedulingLLM serving
0 likes · 7 min read

This article explains Volcano Engine's next‑generation GPU sharing scheduling technology, detailing the two‑layer scheduler, card‑level bin‑pack/spread strategies, system architecture, API definitions, and optimization algorithms that together increase GPU deployment density over 500% and improve utilization by more than 50% for AI workloads.
GPU schedulingKubernetesmGPU
0 likes · 13 min read

REEF introduces a reset‑based preemption mechanism and dynamic kernel padding to achieve microsecond‑scale GPU kernel preemption, enabling concurrent real‑time and best‑effort DNN inference with only 2 % added latency for real‑time tasks while boosting overall throughput by up to 7.7×, as demonstrated on the DISB benchmark.
DNN inferenceGPU schedulingREEF
0 likes · 9 min read

The Punica system unifies AI inference development, testing, deployment, and maintenance on a FaaS‑based one‑stop platform that automates resource scheduling, self‑healing, and monitoring, supporting multiple frameworks and GPUs, thereby doubling onboarding speed, quintuple scaling efficiency, and reclaiming hundreds of GPU cards.
AI inferenceAutonomous OperationsFaaS architecture
0 likes · 13 min read

Huolala’s Big Data Intelligent Platform team describes how they built a cloud‑native, one‑stop AI solution on Kubernetes, integrating Flink‑based feature engineering, a multi‑tenant Zeppelin notebook, GPU‑aware training, and a unified model‑serving platform, while addressing resource isolation, storage persistence, and cross‑cloud deployment.
AI PlatformGPU schedulingKubernetes
0 likes · 17 min read

This talk describes Huya’s elastic distributed training system, covering the motivation behind elasticity, its design using Kubernetes and ETCD for dynamic node registration and scaling, implementation details of the EFDL framework, performance evaluations on ResNet‑50, and the resulting benefits and future directions.
AI PlatformGPU schedulingHuya
0 likes · 11 min read

This article describes Huya's elastic distributed training system, explaining why elasticity is needed, the architectural design using Kubernetes and ETCD, the dynamic scaling process, performance evaluations on ResNet‑50, and future improvements for more efficient and reliable AI model training.
AI PlatformGPU schedulingKubernetes
0 likes · 10 min read


Code DAODec 11, 2021 · Artificial Intelligence
The article analyzes how Nimble reduces GPU scheduling overhead and enables parallel execution through ahead‑of‑time scheduling and automatic multi‑stream assignment, achieving up to 22.3× inference speedup over PyTorch and significantly improving GPU utilization for deep learning workloads.
GPU schedulingPerformance Accelerationahead-of-time
0 likes · 9 min read

KubeSphere 3.2.0, the latest cloud‑native distribution built on Kubernetes, introduces GPU resource scheduling and monitoring, enhanced observability with Grafana panels, multi‑cluster and multi‑tenant management, advanced storage features, a global gateway, OpenID Connect authentication, a dynamic application store, and a more independent DevOps suite, all aimed at improving user experience and operational efficiency.
GPU schedulingKubernetesStorage Management
0 likes · 12 min read


58 TechNov 20, 2020 · Artificial Intelligence
The article details the development, architecture, and optimization of 58.com’s AI algorithm platform (WPAI), covering its background, overall design, large‑scale distributed machine learning, deep‑learning platform features, inference performance enhancements, GPU resource scheduling improvements, and future directions.
AI PlatformGPU schedulingInference Optimization
0 likes · 15 min read

TCOS 2.0, the container operating system from Transwarp, expands compatibility to Windows, ARM, MIPS, and domestic platforms, adds GPU heterogeneous scheduling, HPA autoscaling, enhanced local storage management, and improved monitoring, providing a robust foundation for big data, AI, and cloud-native applications.
GPU schedulingKubernetesTCOS
0 likes · 11 min read

This article details the practical steps to install nvidia-docker2, configure Docker’s runtime, enable GPU support via Kubernetes device plugins, and verify GPU scheduling on a large Kubernetes cluster, providing code snippets and best‑practice recommendations for production environments.
Device PluginDockerGPU
0 likes · 8 min read


21CTOSep 17, 2017 · Artificial Intelligence
Since September 2016, JD’s AI foundation platform has leveraged Docker and Kubernetes to build a scalable machine‑learning infrastructure that now runs over 5,000 container instances, supports more than 20 AI services, and provides unified GPU, storage, networking, and multi‑tenant capabilities for both inference and training workloads.
AI PlatformGPU schedulingKubernetes
0 likes · 14 min read

This article explains how to deploy TensorFlow on Kubernetes, addressing resource isolation, GPU scheduling, and distributed training challenges by introducing a custom TensorFlow‑on‑K8s system with client, task, and autospec modules, plus container design for reliable job execution.
AI DeploymentGPU schedulingKubernetes
0 likes · 9 min read
