Nanjing University's analysis of supernode scale-up interconnect systems reveals how NVLink evolution and domestic solutions like Huawei CloudMatrix384 reduce AI training communication overhead from 37% to 16%, delivering 15x bandwidth gains and 70% latency reduction over scale-out clusters.
AI TrainingCommunication OverheadHuawei CloudMatrix
This article analyzes how recent research redefines source code's role in training coding agents, moving beyond mere training corpus to serve as executable task environments, verification oracles, and skill repositories, citing works like CodeMidas, R2E-Gym, and SWE-smith that leverage implemented functionality for task generation and validation.
RustFS 1.0 GA releases a production-ready, Apache 2.0-licensed S3-compatible object store written in Rust, achieving 2.3× MinIO performance on 4KB objects via SIMD-accelerated erasure coding, io_uring zero-copy I/O, and a GC-free runtime, while adding built-in Iceberg REST Catalog (S3 Tables) for data-lake workloads.
The article analyzes the definition, technical traits, and three classifications of supernodes (SuperNode, SuperPoD, SuperCluster), explains how they boost large‑model training and inference efficiency, and examines the evolution of domestic supernode architectures, interconnects, and ecosystem design.
AI TrainingGPU interconnectHigh-Performance Computing
Lego‑RL is an open‑source reinforcement‑learning framework that trains coding agents directly on unmodified OpenHands SDK, Claude Code, and OpenCode harnesses, boosting SWE‑bench Verified scores from 64/62/57 to 70.4/68.2/66.6 while addressing faithful optimization, reliable execution, and observable training through GSPO and a sandboxed architecture.
The article analyzes how Alibaba Cloud’s CPFS, KVCacheStore, AgenticFS, OSS Agent and CDE Agent together address exploding AI data scales, accelerate model training and inference, and support massive AI Agent workloads with concrete performance numbers and architectural details.
Recent interview trends show a sharp shift from traditional Java topics to AI‑focused questions about Retrieval‑Augmented Generation and Agent design, with data revealing AI roles topping demand and salary charts while companies struggle to find talent capable of deploying large models in real business contexts.
The article outlines a step‑by‑step framework for enterprises to adopt AI responsibly, emphasizing business‑first pilot projects, human‑AI collaboration, process redesign, organization‑wide training, and risk controls rather than costly, blanket technology purchases.
This article breaks down NVIDIA GPU fundamentals—contrasting GPU with CPU design, tracing CUDA’s evolution, detailing the hardware hierarchy from chips to SM units, explaining memory tiers, and presenting a step‑by‑step performance‑optimization methodology for AI training and inference workloads.
The article analyzes how combining formal verification with expert‑derived Rubrics provides fine‑grained process supervision for large language models, presents the CRAFT data‑production pipeline, and shows experimental gains on math and medical benchmarks using Rubric‑driven RL, SFT, and alternating RL‑SFT training.
In a ten‑thousand‑word analysis, former OpenAI safety VP Lilian Weng retraces the history of model scaling laws from Kaplan’s 2020 formulation, demonstrates how DeepMind’s Chinchilla overturns the original parameter‑to‑data ratio, uncovers two critical bugs in the Chinchilla paper, and warns that the impending 2026‑2028 data wall makes naïve scaling of parameters and compute unsustainable.
PhoneWorld reconstructs mock Android apps from real‑world usage traces, creating scalable, resettable, and verifiable environments that let Mobile Agents train on realistic page structures, navigation paths, and state changes, and the paper shows substantial gains across four mobile benchmarks.
Meta’s third 2026 layoff round cut roughly 8,000 employees—delivered at 4 am in Singapore—citing a secret program that tracks keyboard, mouse and screenshots to train AI, with Evercore estimating $3 billion in annual savings, while the company reallocates survivors to an AI‑focused division.
LimX Dynamics partners with Alibaba Cloud PAI to migrate training workloads, achieving a 10% boost in training efficiency and a 17% drop in operational complexity, while open‑sourcing the FluxVLA Engine to lower the barrier for deploying embodied‑intelligence models at scale.
The article argues that successful speech model training starts with understanding user scenarios, then selecting appropriate data, and finally choosing metrics, detailing six key questions, data sourcing strategies, evaluation criteria, and compliance considerations to avoid the misconception that sheer data volume guarantees performance.
The article explains why pretrained LLMs cannot call tools, then breaks down the three‑stage training pipeline—Supervised Fine‑Tuning, Reinforcement Learning from Human Feedback, and knowledge distillation—showing how each step teaches models to read tool schemas, decide when to invoke a tool, generate JSON calls, and finally transfer the capability to smaller models with LoRA.
Meta is installing software on U.S. employees' computers to capture mouse movements, clicks, and keystrokes for AI model training, a move detailed in an internal memo that has provoked strong backlash, raised privacy concerns, and highlighted the company's broader push toward autonomous AI agents amid industry‑wide automation trends.
The article explains how YouTube’s AI‑video rating and Google’s reCAPTCHA system covertly collect billions of user interactions each day, converting them into labeled data that fuels Google’s computer‑vision models such as Veo, Maps and Waymo, effectively turning routine security checks into a massive, unpaid AI training workforce.
This guide walks you through interpreting training logs and loss curves, diagnosing common issues such as oscillation, under‑fitting, and over‑fitting, and applying concrete adjustments to learning rate, LoRA settings, batch size, and epochs, with scenario‑specific strategies to turn a novice into a tuning expert.
The paper reveals that over‑optimizing supervised fine‑tuning (SFT) for large language models can diminish their reinforcement‑learning (RL) potential, proposes an Adaptive Early‑Stop Loss (AESL) that balances accuracy and output diversity during cold‑start, and demonstrates across multiple LLMs that AESL consistently yields superior RL results.
Starting April 24, GitHub will collect user interaction data—including code inputs, outputs, snippets, context, comments, repository structure, and feedback—to train its AI models, affecting Copilot Free, Pro, and Pro+ users while offering an opt‑out option via settings, and mirroring similar policies at Anthropic, JetBrains, and Microsoft.
A detailed investigation of two public programming‑training datasets shows that AI‑only synthetic data suffers from severe quality issues, and even AI‑plus‑expert review yields only about ten percent usable examples, proving that high‑quality training data still requires domain experts and rigorous quality‑control processes.
This article explains how Kubernetes VolumePopulator can automatically transfer AI training data from low‑cost OSS storage to high‑performance CPFS volumes, enabling on‑demand model loading, cost‑effective hot‑cold data management, and fully automated lifecycle handling in cloud‑native AI workloads.
The article explains how loss masking in supervised fine‑tuning of large language models prevents the model from learning irrelevant tokens such as user inputs, system prompts, tool outputs, and padding, thereby focusing training on the assistant’s responses and improving performance and generalization.
This article breaks down the three core stages of large language model training—pretraining, supervised fine‑tuning, and alignment—detailing their objectives, typical data formats, scale requirements, and the latest techniques such as RLHF and DPO.
The article analyzes TeleChat3-105B-A4.7-Thinking, the first domestically built 100‑billion‑parameter Mixture‑of‑Experts model, detailing its multi‑dimensional evaluation, three‑stage training pipeline, hardware‑level optimizations, fine‑grained architecture, and its significance for the evolving AI competition landscape.
The article examines how the rise of large‑model AI training reintroduces the need for gang scheduling in Kubernetes, contrasting the rigid resource requirements of HPC‑style workloads with cloud‑native elasticity, and outlines the historical evolution, current implementations, and future directions for achieving more flexible, high‑throughput compute orchestration.
AI TrainingGang SchedulingHigh-Performance Computing
This tutorial walks you through renting a GPUFree RTX 4090 cloud instance, uploading your Lerobot dataset, launching training via a lightweight Flask web UI, automatically shutting down the server, and downloading the trained model, all with detailed code snippets and practical tips.
The newly available Amazon EC2 P6‑B300 instance, powered by NVIDIA Blackwell Ultra GPUs, offers up to 2× network bandwidth and 1.5× GPU memory compared with its predecessor, delivering 6.4 Tbps EFA throughput, 2.1 TB GPU memory, and optimized storage options for large‑scale AI training and deployment, especially for MoE and multimodal models.
This tutorial walks through a complete workflow for building a navigation model using Isaac Sim and MobilityGen to generate synthetic data, applying Cosmos‑Transfer1‑7B for visual data augmentation, training the X‑Mobility model via imitation learning, converting it for ROS2 deployment, and performing software‑in‑the‑loop validation.
Koordinator v1.7.0, the open‑source Kubernetes scheduler, adds network‑topology‑aware scheduling, job‑level preemption, and support for Ascend NPU and Cambricon MLU, delivering unified heterogeneous device management, enhanced GPU sharing, comprehensive API documentation, and best‑practice guides to improve large‑scale AI training efficiency and cluster operations.
Game‑TARS, a general‑purpose game AI from ByteDance's Seed team, replaces custom function calls with low‑level keyboard‑mouse actions, leverages massive multimodal data, sparse‑thinking and decaying‑loss algorithms, and achieves zero‑shot mastery across diverse games, surpassing top large models like GPT‑5 and Gemini‑2.5‑Pro.
This article examines the challenges of building scalable, low‑latency, and cost‑effective network architectures—such as Clos/Fat‑Tree, Spine‑Leaf, Dragonfly, and Torus—for massive GPU clusters used in training trillion‑parameter AI models, comparing multi‑rail and single‑rail designs and highlighting real‑world implementations from Tencent and Alibaba.
The article reveals that Microsoft's Gaming Copilot feature captures on‑screen text via OCR and uploads it to the cloud for AI model training, discusses privacy concerns, performance impacts on games like Battlefield 6, and provides steps to disable or uninstall the feature.
NanoChat, an open‑source full‑stack AI model solution created by Andrej Karpathy, enables users to train a functional chat model on a modest $100 cloud GPU rental, offering a low‑cost, hands‑on alternative to proprietary large‑language‑model services.
The 2025 Vivo Developer Conference showcased cutting‑edge advances in traffic‑driven growth, microservice and Dubbo performance tuning, full‑link multi‑version environment automation, and GPU‑container AI training platforms, highlighting how these innovations boost efficiency, reliability, and cost‑effectiveness across Vivo’s internet services.
This article explains how InfiniBand’s architecture, native RDMA, GPUDirect, and evolving bandwidth enable ultra‑low‑latency, high‑throughput communication for AI model training, compares it with Ethernet, and details the role of RoCEv2 and other high‑performance interconnect technologies.
This talk explores large‑scale AI training storage challenges and presents PCache, a high‑performance, cloud‑native caching system that optimizes metadata, read/write paths, deployment, and high‑availability, delivering significant throughput gains and cost savings for massive model training workloads.
This article examines the differences between Scale‑Out and Scale‑Up networking in AI compute clusters, comparing PCIe, Ethernet, InfiniBand, NVLink, UALink, and emerging standards like UB‑Mesh, and explains how each technology impacts bandwidth, latency, scalability, and cost for large‑scale model training.
Alluxio’s distributed caching dramatically accelerates AI training and checkpointing workloads, achieving up to 99.57% GPU utilization and linear scaling across clusters in the MLPerf Storage v2.0 benchmark, while using cost‑effective commodity hardware to eliminate I/O bottlenecks.
Meta has been accused of illegally downloading 2,396 paid adult videos since 2018 to train its AI models, including Meta Movie Gen and LLaMA, prompting lawsuits that could cost up to $359 million, highlighting broader industry concerns over copyright infringement in AI training data.
This article explains why pre‑training large AI models consumes 90‑99% of total compute, describes the full training and inference pipelines, introduces resource‑saving strategies such as PD‑separation, and reviews market trends and infrastructure challenges shaping the next generation of AI systems.
This article compares the main networking technologies used in large‑scale AI GPU clusters—NVLink, InfiniBand, RoCE Ethernet, and the emerging DDC full‑schedule fabric—examining latency, lossless transmission, congestion control, cost, power and scalability to help engineers choose the optimal solution for training massive language models.
Tencent’s Star‑Network team partnered with DeepSeek to open‑source the DeepEP communication library, then used its self‑developed TRMT stack to overcome RoCE limitations, achieving up to 100% speedup on RoCEv2 and 30% on InfiniBand, cutting training costs and inference latency for large MoE models.
This article explains the memory limits of a single GPU and systematically introduces data parallelism, tensor parallelism, pipeline parallelism, sequence parallelism, and expert parallelism, describing their communication costs, advantages, drawbacks, and practical implementation details for training large AI models.
This article breaks down GPU memory consumption for large Transformer models, explains how to estimate each component—parameters, optimizer state, activations, gradients—and shows how parallelism, mixed precision, and recomputation strategies can dramatically reduce the footprint.
The article explains how large‑scale AI model training and inference require massive data handling, describes the RapidFS storage acceleration cluster deployed on a 30,000‑card Kunlun chip system with hundreds of domestic CPU servers, and presents performance tests showing linear throughput scaling up to over 1 TiB/s, demonstrating the impact of high‑performance storage on compute efficiency.
The article explains how RapidFS, a near‑compute storage acceleration solution built on BOS object storage, delivers up to 10 TiB/s throughput for massive AI model training, detailing its architecture, deployment on a 30,000‑card Kunlun cluster, and performance test results that show linear scaling from 20 to 70 nodes.
This article analyzes NVIDIA's A100, H100, A800, H800, and H20 GPUs, compares their architectures, performance, and pricing, and provides a step‑by‑step guide for building a private AI compute center tailored to training, inference, and high‑performance computing workloads.
As AI models grow to trillion‑parameter scales, training them demands massive GPU clusters whose performance is increasingly limited by network bandwidth; this article examines why traditional PCIe interconnects become bottlenecks and how NVIDIA's NVLink and NVSwitch technologies dramatically improve multi‑GPU communication and overall system efficiency.
This article examines the latest Nvidia GPU lineup—including A100, H100, A800, H800, and the upcoming H20—detailing their architectures, performance metrics for AI training and inference, cost considerations, and provides a step‑by‑step guide for building a high‑performance compute center.
The article breaks down the three core stages of large‑language‑model training—pre‑training, supervised fine‑tuning, and RLHF—explaining their purpose, methods, and concrete examples while noting DeepSeek‑R1’s recent breakthrough and its implications for AI development.
The interview explores DeepSeek R1's open‑source weights, its multi‑stage training pipeline—including pre‑training, supervised fine‑tuning, and RLHF—alongside innovations such as self‑consistency, chain‑of‑thought prompting, distillation, MoE architectures, and cost considerations, highlighting its impact on the future of large language models.
This article traces the decade‑long evolution of AI training stability—from early small‑model manual operations to large‑scale, multi‑thousand‑GPU clusters—detailing metrics like invalid training time, fault‑tolerance architectures, eBPF‑based hidden‑fault detection, BCCL enhancements, multi‑level restart strategies, and trigger‑based checkpointing that together shrink downtime from minutes to seconds.
This article details how a 7B parameter language model, fine‑tuned with DeepSeek's GRPO reinforcement‑learning algorithm and a carefully crafted multi‑component reward system, learned to solve Sudoku puzzles without any cold‑start data, outperforming a comparable 3B model and revealing key insights for structured reasoning tasks.
This article compares 3FS, vePFS, and CloudFS across metadata and data planes, presents detailed benchmark results for AI training scenarios, analyzes architectural trade‑offs, and draws insights for future cloud‑native file storage development.
This article analyzes DeepSeek’s latest large‑model breakthroughs, covering the MLA attention compression, GRPO alignment algorithm, MoE load‑balancing redesign, multi‑stage training pipelines, reinforcement‑learning tricks, and performance comparisons with GPT‑4o‑Mini and Llama 3.1, highlighting both strengths and remaining challenges.
The article examines how Baidu Baige evolved AI training stability from manual operations to precise engineering, detailing metrics, fault‑perception techniques, eBPF‑based diagnostics, multi‑level restart strategies, and trigger‑based checkpointing that together achieve sub‑minute recovery and 99.5% effective training time on massive GPU clusters.
This article dives deep into DeepSeek's 3FS parallel file system, detailing its four-component architecture, RDMA‑based high‑speed networking, client options, metadata and storage services, replication protocols, dynamic stripe sizing, and recovery mechanisms that enable efficient AI model training and inference.
This article provides a detailed technical analysis of NVIDIA's A100, H100, A800, H800 and H20 GPUs, compares their architectures, performance and cost, and offers step‑by‑step guidance on building a private AI compute center, selecting hardware, software stacks and budgeting for different workloads.
This article provides a detailed technical analysis of DeepSeek's R1 large language model, covering its background, architecture, training methods, hardware optimizations, performance claims, user impressions, deployment options, and the challenges of reproducing its results.
With modern large‑language‑model workloads demanding massive parallelism, 8‑GPU servers have become the norm; this article explains the roles of CPUs, compares GPU‑to‑GPU interconnect options—including PCIe direct, PCIe Switch, NVLink, and NVSwitch—detailing their architectures, bandwidths, topologies, and trade‑offs for AI training.
The article examines the evolution of data lake storage acceleration, compares various solutions, and explains how metadata, read/write, and end‑to‑end optimizations enable scalable, cost‑effective AI and big‑data workloads in cloud‑native environments.
This article provides a comprehensive, step‑by‑step guide to training customized large language models, covering industry‑specific needs, data privacy, meticulous data cleaning, optimal data‑ratio balancing, token budgeting, GPU memory accounting, LoRA fine‑tuning techniques, and practical evaluation metrics for robust AI deployment.
The article explains how RDMA technologies—including InfiniBand, RoCE, and iWARP—provide high‑throughput, low‑latency, CPU‑free data transfer for massive generative AI model training, compares their architectures, and discusses modern network designs and load‑balancing strategies to optimize AI‑focused data‑center networks.
This article provides a detailed technical analysis of Meta's SIGCOMM paper on RDMA over Ethernet for large‑scale AI training, examining the physical network deployment, congestion‑control mechanisms, topology choices, routing strategies, hardware design, and the practical challenges that remain.
This article presents Xiaohongshu's multi‑cloud unified data acceleration layer built with Alluxio, detailing the challenges of multi‑cloud architectures, the design goals, Alluxio's architecture and features, real‑world case studies in AI training and recommendation indexing, performance improvements, and future plans.
This article explains the motivations behind domain‑specific continue pretraining for large language models, outlines a three‑step workflow—including vocabulary expansion, data replay, ratio control, and scaling‑law calculations—provides concrete hyper‑parameter recommendations, and discusses challenges across different domain types and future research directions.
This article reviews the main GPU/TPU cluster networking solutions—including NVLink, InfiniBand, RoCE Ethernet, and DDC full‑schedule fabrics—examining their latency, loss‑free transmission, congestion control, cost, scalability, and suitability for large‑scale LLM training workloads.
This article presents a comprehensive overview of modern Data+AI data lake challenges and solutions, covering the evolution of data lakes, an introduction to Apache Iceberg, practical use of PyIceberg for AI training and inference pipelines, and advanced vector table and indexing techniques for efficient similarity search.
Midjourney enhances its AI models by integrating a wide range of data sources—including public datasets like ImageNet and COCO, academic research from top conferences, partner collaborations, and its own proprietary data—while continuously updating and managing these datasets for quality, privacy, and security.
Using a martial‑arts analogy, the article explains why training massive AI models now requires thousands of GPUs or mixed‑chip clusters, outlines three key steps—inter‑connect, distributed parallel strategies, and accelerator acceleration—and shows how Baidu’s Baige platform achieves near‑full efficiency across GPU, Kunlun and Ascend chips.
AI TrainingGPU interconnectaccelerator optimization
This article analyzes the challenges and solutions for constructing a super‑large GPU training cluster, outlining five fundamental design principles, a four‑layer plus one‑domain architecture, and practical considerations for hardware, networking, and operational reliability in AI workloads.
CAPTCHAs, originally designed as human‑based computation tools to block bots, have become costly, discriminatory, and largely ineffective security measures that waste billions of dollars annually while providing profit to service providers, prompting a 2024 debate on their continued use.
This article explains how to construct high‑performance RoCE v2 and InfiniBand networks within a cloud‑native Kubernetes environment, detailing the underlying technologies, required components, configuration steps, and performance test results that demonstrate significant communication speed improvements for large‑scale AI model training.
This article provides a detailed technical overview of modern multi‑GPU server architectures—including PCIe switches, NVLink, NVSwitch, and HBM—explaining their hardware topologies, bandwidth characteristics, monitoring methods, and network choices to help engineers design efficient AI training clusters.
This article provides a detailed technical breakdown of modern multi‑GPU server nodes, covering component composition, storage network cards, NVSwitch interconnects, bandwidth calculations, and the architectural differences between NVIDIA A100/A800 and H100/H800 configurations for AI training workloads.
The rapid growth of AI model sizes has created unprecedented demands on network bandwidth, latency, stability, and automation, making efficient RDMA‑based interconnects, advanced congestion control, and intelligent deployment essential for scaling distributed training clusters to thousands of GPUs.
The article draws parallels between expert viewers dissecting the drama "The Legend of Zhen Huan," efficient paper‑reading techniques, and the active‑prediction plus contrast‑learning approach that underpins modern AI model training, highlighting how proactive thinking boosts both personal and machine learning outcomes.
Vector retrieval, based on converting data into high‑dimensional vectors and measuring similarity, enables fast, accurate search across massive datasets, supporting AI tasks such as search engines, recommendation, NLP, and computer vision, and plays a crucial role in large‑model training for data selection, anomaly detection, and model optimization.
AI TrainingApproximate Nearest Neighborinformation retrieval
This article examines the key GPU/TPU cluster networking options—NVLink, InfiniBand, RoCE Ethernet, and emerging DDC full‑scheduling fabrics—detailing their latency, loss‑less transmission, congestion control, cost, power, and scalability considerations for large‑scale AI training deployments.
This article examines the challenges of data‑intensive AI training on heterogeneous cloud‑native infrastructure and explains how the Fluid framework combined with JindoCache and KubeDL provides distributed caching, metadata acceleration, and seamless POSIX access to dramatically improve I/O performance, GPU utilization, and cost efficiency.
The article analyzes AI‑training traffic patterns, critiques existing flow‑based, flowlet‑based, and packet‑based ECMP load‑balancing, introduces the NSLB solution tailored for AI clusters, and presents experimental results showing up to 113% speed improvement and sub‑millisecond failover with DPFF, while also discussing direct‑topology and intelligent lossless networking techniques.
This article explains how the open‑source Koordinator scheduler, combined with KubeDL, tackles the resource‑intensive demands of large‑scale AI and LLM training on Kubernetes by introducing heterogeneous resource management, elastic quota, coscheduling, and fine‑grained GPU & RDMA allocation.
This article examines the challenges of scaling AI model training beyond single-chip limits, introduces Alibaba Cloud’s Predictable Network architecture—including high‑performance Ethernet, dual‑uplink, and adaptive routing—and compares its performance, scalability, and reliability against InfiniBand, showing how Ethernet can meet AI workloads with minimal loss.
AI TrainingEthernet vs InfiniBandPredictable Network
The article examines how Alibaba Cloud’s Predictable Network, InfiniBand versus Ethernet trade‑offs, and the HPN high‑performance network design together address the extreme bandwidth, latency, scalability and reliability requirements of modern large‑model AI training workloads in cloud data centers.
This article examines the storage bottlenecks in large‑scale AI training, evaluates local‑disk and Alluxio‑based distributed caching strategies, proposes uniform cache eviction and replica‑aware global policies, and introduces the SiloD framework for coordinated compute‑storage scheduling to dramatically improve GPU utilization and overall cluster throughput.
Elon Musk announced plans to sue Microsoft for allegedly using Twitter data without permission to train AI models, sparking a heated debate over data rights, API pricing changes, and the broader competition between major tech platforms in the AI landscape.
Tencent’s Star Network delivers a 1.6 Tbps Ethernet‑RDMA fabric, fat‑tree topology supporting up to 4 K GPUs, multi‑track traffic aggregation and adaptive heterogeneous links plus a custom TCCL library, cutting AllReduce overhead from 35 % to 3.7 %, speeding AI training iterations by 32 % while automating deployment and providing sub‑second self‑healing.
This article systematically examines the major performance bottlenecks in AI model training, explains the underlying hardware and software causes, and presents a comprehensive set of acceleration strategies—including data‑loading optimizations, compute‑side enhancements, communication tricks, and the AIAK‑Training suite—backed by real‑world case studies and quantitative results.
This article systematically analyzes the main performance bottlenecks in AI model training, explains why acceleration is essential, and presents current hardware‑ and software‑based solutions—including data‑loading optimizations, operator fusion, mixed‑precision and Tensor Core usage, as well as distributed communication strategies—followed by real‑world case studies of Baidu's AIAK‑Training suite that demonstrate significant speed‑ups.
Veteran programmers have filed a class-action lawsuit alleging that GitHub’s Copilot AI violates copyright by training on their contributed code without proper attribution, licensing notices, or permission, claiming repeated breaches of Section 1202 and infringing creators’ rights.
This article examines the comprehensive challenges AI applications face from storage to compute, traces the evolution of AI training infrastructure, analyzes key bottlenecks such as compute acceleration, resource scheduling, massive data handling and data flow, and presents Baidu Cloud's storage acceleration solutions—including parallel file systems, caching, and the Fluid scheduler—to dramatically improve AI training performance.
Sky Computing, introduced by Ant Technology Research Institute, proposes a cloud‑agnostic platform that abstracts heterogeneous public and private clouds into a unified service layer, enabling applications to seamlessly migrate workloads across clouds, reduce costs, avoid vendor lock‑in, and support AI training via the SkyML prototype.
This article analyzes the storage challenges of high‑performance computing—including traditional HPC, AI‑driven HPC, and high‑performance data analysis—examines Baidu’s internal practices, and presents the Canghai storage platform with its object storage, parallel file system (PFS) and RapidFS solutions that address throughput, latency, and scalability requirements.
AI TrainingHigh-Performance Computingcloud storage
This article explains the storage challenges of high‑performance computing—including traditional HPC, AI‑driven HPC, and HPDA—then details Baidu’s unified storage platform, object storage BOS, and runtime solutions PFS and RapidFS, illustrating their architecture, features, and a real‑world autonomous‑driving customer case.
This article examines the emerging Storage Class Memory (SCM) market, outlines its various technologies, evaluates performance and cost trade‑offs, explores three concrete use cases—AI training acceleration, instant data recovery, and greener data‑center operation—and discusses the latency and workload‑model challenges that must be solved for widespread adoption.
This article investigates why AI training tasks on Tencent's Xingchen compute platform experience severe I/O slowdown when using CephFS, analyzes the underlying Ceph‑FUSE and MDS mechanisms, and proposes metadata‑caching and file‑caching optimizations that can accelerate training speed by three to four times.
This article examines the challenges of large‑scale deep‑learning model training on Kubernetes, analyzes performance bottlenecks caused by Alluxio‑FUSE integration, and presents a series of configuration and system‑level optimizations that dramatically improve data‑access speed and overall training throughput.