PaddlePaddle Framework 3.0 Released: Five Core Innovations for Large Models and Scientific Computing

PaddlePaddle (飞桨) framework 3.0 was officially released on April 1, 2025, representing a major upgrade for deep learning and large‑model development.

The release introduces five core innovations: (1) dynamic‑static unified automatic parallelism, which allows users to add only a few tensor‑splitting marks to turn single‑card programs into distributed training, reducing distributed‑related code by up to 80%; (2) training‑inference integration, built on a highly extensible intermediate representation (PIR) that optimizes model compression, inference, deployment and multi‑hardware inference, enabling single‑machine deployment of DeepSeek‑R1 with twice the throughput; (3) scientific‑computing high‑order automatic differentiation, based on combined‑operator technology and the CINN compiler, achieving differential‑equation solving speeds 115% faster than PyTorch with compiler optimizations; (4) neural network compiler (CINN), a one‑stage compilation flow that directly generates CUDA C code, delivering up to 4× operator speed‑up and 27.4% end‑to‑end training acceleration; (5) heterogeneous multi‑chip adaptation, via abstracted hardware interfaces that cut required adaptation interfaces by 56% and code by 80% compared with PyTorch, supporting over 60 chip series and cooperation with more than 40 hardware partners.

These innovations lower the barrier for large‑model parallel training, improve performance across training and inference, and extend the framework to scientific computing domains such as aerospace, automotive, meteorology and life sciences. PaddlePaddle 3.0 remains fully compatible with the 2.0 API set and is now open for developers.

Code example of RMSNorm implementation:

class RMSNorm(paddle.nn.Layer): def __init__(self): super().__init__() self.variance_epsilon = 1e-6 self.weight = paddle.create_parameter(shape=[768], ...) def forward(self, x): variance = x.pow(2).mean(-1, keepdim=True) x = paddle.rsqrt(variance + self.variance_epsilon) * x return x * self.weight