What’s Inside DeepSeek’s Open‑Source Week? DualPipe, EPLB, 3FS and More Explained

DualPipe: Bidirectional Pipeline Parallelism

DualPipe is a bidirectional pipeline parallel algorithm that overlaps computation and communication, eliminating pipeline bubbles. It allows forward and backward phases to run concurrently, improving efficiency for large‑scale distributed training. The project is hosted at https://github.com/deepseek-ai/DualPipe. Performance comparisons show it reduces bubbles compared with 1F1B and ZB1P methods.

EPLB: Expert Parallel Load Balancer

EPLB balances expert workloads in expert‑parallel training by heuristically packing replicated experts onto GPUs, ensuring load balance and reducing communication overhead. It is used in DeepSeek‑V3 and is available at https://github.com/deepseek-ai/eplb. The design leverages group‑restricted expert routing to keep experts of the same group on the same node.

Training and Inference Framework Analysis Data

DeepSeek released analysis data for its training and inference frameworks, including configuration files that illustrate overlapping strategies for forward/backward blocks in DualPipe and pre‑fill/decoding settings for inference. The dataset is accessible at https://github.com/deepseek-ai/profile-data and helps the community understand compute‑communication overlap.

3FS: High‑Performance Parallel File System

3FS (Fire‑Flyer File System) combines modern SSDs with RDMA networking to provide a shared storage layer with strong consistency via chain replication (CRAQ). Benchmarks show 6.6 TiB/s aggregate read throughput on a 180‑node cluster and 3.66 TiB/min on GraySort. Repository: https://github.com/deepseek-ai/3FS. It supports training data preprocessing, dataset loading, checkpoint handling, and KVCache lookup for inference.

Smallpond: Data‑Processing Framework on 3FS

Smallpond builds on 3FS to offer a data‑processing framework that leverages the file system’s high‑throughput storage for tasks such as data analysis and machine learning. Source code is available at https://github.com/deepseek-ai/smallpond.

Conclusion

Beyond the open‑source releases, DeepSeek announced API platform recharge restoration and off‑peak pricing discounts, reducing costs for DeepSeek‑V3 and DeepSeek‑R1. These developments illustrate DeepSeek’s commitment to advancing AI infrastructure and providing community‑ready tools.