How to Deploy Yuan 2.0 LLM with PaddleNLP: A Step‑by‑Step Guide

Overview

The open‑source Yuan 2.0 large language model is now fully compatible with Baidu PaddleNLP. Users can invoke Yuan 2.0’s pretrained capabilities for semantics, mathematics, reasoning, code generation, and knowledge retrieval, and fine‑tune the model on domain‑specific datasets with modest hardware.

Model Variants

Yuan 2.0‑102.6 B

Yuan 2.0‑51.8 B

Yuan 2.0‑21 B

These three scales constitute the first fully open‑source trillion‑parameter family, excelling in dialogue, programming, and logical reasoning.

Key Features in PaddleNLP

Unified pre‑train‑fine‑tune‑inference workflow that requires no code changes for different hardware.

Supports data parallelism, group‑wise sharding, and model parallelism for high‑performance distributed training and inference.

Zero Padding optimization: a greedy group‑wise data‑filling strategy that removes ineffective padding, accelerating supervised fine‑tuning compared with LLaMA‑Factory.

Repository

https://github.com/PaddlePaddle/PaddleNLP/tree/develop/paddlenlp/transformers/yuan

Running Yuan 2.0 with PaddleNLP

Prepare the hardware environment and install PaddlePaddle (CPU/GPU/Ascend as appropriate).

Download the desired Yuan 2.0 checkpoint (e.g., 51B) from the official release page.

Execute a quick inference script to verify the installation.

For detailed configuration, refer to the README at https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm/config/yuan/README.md.

Interaction Examples

Code Generation

Prompt: “Implement bubble sort in Java.”

Response: Explanation of the algorithm followed by the implementation.

public static void bubbleSort(int[] arr) {
    int n = arr.length;
    for (int i = 0; i < n - 1; i++) {
        for (int j = 0; j < n - i - 1; j++) {
            if (arr[j] > arr[j + 1]) {
                int temp = arr[j];
                arr[j] = arr[j + 1];
                arr[j + 1] = temp;
            }
        }
    }
}

Calling bubbleSort(new int[]{5,3,8,2,1,6,4}); yields [1,2,3,4,5,6,8].

Article Generation

Prompt: “Write an article titled ‘What AI Has Brought to Humanity.’”

Response: A concise overview of AI applications across healthcare, finance, transportation, education, and entertainment, highlighting efficiency gains and new capabilities.

Translation

Prompt: Translate the English sentence “But with the passage of time, Hemingway has reached middle age, and his energy and creativity have also declined. In Hemingway's inner world, he was very upset, but he never lost hope and confidence.” into Chinese.

Response: “然而随着时间的推移，海明威已经步入中年，他的精力和创造力也不断下降，但他的内心却非常不安，却从未失去希望和信心。”

Dialogue

Prompt: “What are the most advanced machine‑learning algorithms today?”

Response: List of CNN, RNN (including LSTM and GRU), AutoEncoder, GAN, Transfer Learning, each with a brief description.

Training and Fine‑Tuning

Pre‑training Example (Yuan 2.0‑51B)

The loss curve demonstrates convergence, indicating that users can start from the Yuan 2.0 architecture and train on their own corpora to create a custom base model.

Fine‑tuning Example (LoRA on Yuan 2.0‑51B)

Applying LoRA adaptation yields a convergent loss curve, showing that domain‑specific fine‑tuning can be performed with limited GPU resources.