Skip to content
Tagged articles

recomputation

3 articles · Page 1 of 1
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Oct 28, 2026 · Artificial Intelligence

BaGuaLu-2: Runtime-Adaptive Memory Management for MoE Training on 10K+ GPU Clusters

Tsinghua University's BaGuaLu-2 system dynamically selects per-operator recomputation or offloading after MoE gating to keep activation memory within device limits, achieving 1.14× throughput over baselines on 12,288 GPUs with 32.9% average MFU, zero config changes, and open-source code.

BaGuaLuGPU clustersMoE training
0 likes · 11 min read
BaGuaLu-2: Runtime-Adaptive Memory Management for MoE Training on 10K+ GPU Clusters
Xiaohongshu Tech REDtech
Xiaohongshu Tech REDtech
Dec 11, 2025 · Artificial Intelligence

Fine‑Grained Activation Offloading: Cutting Memory Use While Preserving LLM Throughput

The article introduces a fine‑grained activation offloading technique implemented in Megatron‑Core that offloads module‑level activations to CPU, overlaps transfer with computation, and remains compatible with pipeline and virtual pipeline parallelism, dramatically reducing peak GPU memory for large language models while incurring minimal throughput loss.

LLMMegatronPipeline Parallelism
0 likes · 18 min read
Fine‑Grained Activation Offloading: Cutting Memory Use While Preserving LLM Throughput
DataFunSummit
DataFunSummit
Apr 11, 2023 · Artificial Intelligence

OneFlow Coop: Joint Optimization of Dynamic‑Graph Recomputation and Memory Allocation

This article introduces OneFlow Coop, a memory‑optimization technique that jointly optimizes dynamic‑graph recomputation strategies and GPU memory allocation by analyzing existing DTR limitations, proposing recomputable in‑place, op‑guided tensor allocation, and layout‑aware eviction modules, and demonstrating superior experimental results.

Dynamic GraphGPU memoryOneFlow
0 likes · 18 min read
OneFlow Coop: Joint Optimization of Dynamic‑Graph Recomputation and Memory Allocation