How DeepSeek’s Open‑Source Tools Are Supercharging AI Model Performance

On February 21, 2025 DeepSeek announced an "Open‑Source Week" during which it released five code repositories, marking a significant upgrade to its open‑source strategy.

1. FlashMLA – Accelerating AI Scene Generation

FlashMLA is a high‑efficiency MLA decoding kernel optimized for Hopper‑class GPUs such as the H800. It applies low‑rank joint compression to the key and value matrices of multi‑head attention, projecting them into a lower‑dimensional space to cut data volume and boost compute efficiency. The KV cache is split into 64‑token blocks, reducing memory fragmentation.

Performance on the H800 platform reaches 3000 GB/s data throughput and 580 TFLOPS, approaching the 740 TFLOPS of H100‑based FlashAttention‑3. Compared with FlashAttention‑2, FlashMLA delivers roughly a 2× speedup while lowering memory usage and compute cost.

2. DeepEP – Efficient MoE Communication Library

DeepEP provides an all‑to‑all GPU kernel for Mixture‑of‑Experts (MoE) model training and inference, addressing token distribution and aggregation bottlenecks. It supports low‑precision formats such as FP8 and FP16, improving compute efficiency and reducing memory demand.

Network‑level optimizations target asymmetric bandwidth forwarding from NVLink to RDMA. Measured NVLink bandwidths are 153 GB/s (distribution) and 158 GB/s (merge); across nodes the bandwidth stabilises at 43–46 GB/s. Latency for an 8‑expert scenario is 163 µs for distribution and 318 µs for merging, with latency increasing as the number of experts grows.

3. DeepGEMM – FP8‑Optimized Matrix Multiplication

DeepGEMM is a lightweight FP8‑focused GEMM library for the DeepSeek‑V3/R1 architecture. Its core kernel consists of only about 300 lines of CUDA code and is JIT‑compiled at runtime, simplifying integration.

Benchmarking on an H800 with NVCC 12.8 shows peak compute performance of 1358 TFLOPS and memory bandwidth up to 2668 GB/s. Compared with CUTLASS 3.6, DeepGEMM achieves up to 2.7× speedup. For MoE workloads, grouped GEMM under a continuous layout can improve throughput by 1.2×.

4. DualPipe & EPLB – Parallelism Strategies for Large Models

DualPipe is a bidirectional pipeline‑parallel algorithm designed for the V3/R1 architecture. It overlaps forward computation, communication, and backward computation, reducing pipeline bubbles at the cost of roughly doubling parameter usage.

EPLB (Expert Load Balancer) dynamically estimates each expert’s load, adjusts replica counts, and balances tasks across GPUs to prevent overload. It offers two balancing modes: hierarchical (balancing first across nodes, then within GPUs) and global (replicating experts across all GPUs), the latter targeting large‑scale inference decoding.

5. 3FS – High‑Performance Distributed File System

3FS is a distributed file system built for AI training and inference workloads. It leverages modern SSDs and RDMA networking to provide a shared storage layer that simplifies distributed application development.

In a 180‑node cluster, 3FS achieves an aggregate read throughput of about 6.6 TiB/s, enabling parallel processing. In the GraySort benchmark, 3FS reaches 3.66 TiB per minute. Its KVCache lookup can sustain peak throughput exceeding 40 GiB/s, supporting fast data access during inference.

These five open‑source projects collectively address key bottlenecks in AI model training and inference—memory fragmentation, communication overhead, matrix multiplication efficiency, parallelism coordination, and storage throughput—offering the community powerful tools to accelerate large‑scale AI workloads.