Build Your Own AI “Zhuge Liang” Chatbot with LLaMA Factory on Alibaba Cloud

Activity Overview

Address: https://developer.aliyun.com/topic/llamafactory

Time: July 25 – September 1

Task: Use PAI and LLaMA Factory to Chinese‑localize and role‑play fine‑tune Llama‑3, then deploy an AI “Zhuge Liang” Q&A bot available 24/7.

Popular Task

Complete the PAI × LLaMA Factory fine‑tuning experiment, upload a screenshot of the chatbot conversation, and receive a limited‑edition outdoor waist bag.

Invitation Challenge

Invite friends to complete the scenario experience and compete for prizes.

Tutorial Overview

LLaMA Factory is an open‑source low‑code fine‑tuning framework that integrates the most widely used techniques. It supports zero‑code fine‑tuning via a Web UI and has over 20 k GitHub stars.

This tutorial uses Meta AI’s open‑source Llama‑3 8B model and demonstrates how to Chinese‑localize and role‑play fine‑tune the model on the PAI platform.

1. Prepare Environment and Resources

1.1 Claim Free PAI‑DSW Trial

New users receive 5 000 CU·h of free trial resources. Alternatively, purchase a resource package (starting at ¥59).

1.2 Create a Default Workspace

Open the PAI console, select region (Beijing is recommended), deselect MaxCompute and DataWorks, and authorize the service role.

1.3 Open Notebook Gallery

In the console, go to Quick Start → Notebook Gallery and select “LLaMA Factory: Fine‑tune LLaMA‑3 for role‑play”. Open it in DSW and create a new DSW instance.

2. Create PAI‑DSW Instance

Give the instance a name (e.g., DSW_LlamaFactory) and select a GPU spec. Prefer ecs.gn7i-c8g1.2xlarge (A10 GPU). Use the official image modelscope:1.14.0-pytorch2.1.2-gpu-py310-cu121-ubuntu22.04. Keep other parameters default and confirm the order.

3. Install LLaMA Factory and Download Dataset

In the opened llama_factory.ipynb notebook, run the cells in the “Install LLaMA Factory” section, then run the cells in the “Download Dataset” section. Each cell’s ▶ button starts execution; a ✅ indicates success.

4. Zero‑Code Fine‑Tuning via Web UI

4.1 Launch Web UI

Run the launch command, click the returned URL, and open the Web UI.

4.2 Configure Parameters

Select the LLaMA‑3‑8B‑Chat model, keep the fine‑tuning method as lora , and set learning rate to 1e‑4, gradient accumulation to 2. Choose LoRA+ with a ratio of 16 and apply it to all linear layers. Set the output directory to train_llama3.

4.3 Start Training

Click “Start”. Training takes about 20 minutes; progress and loss curves appear in the UI. When the UI shows “Training completed”, the fine‑tuning succeeded.

5. Model Evaluation

Refresh the adapter, select the train_llama3 LoRA weights, then choose the “Evaluate & Predict” tab. Use the eval dataset, set output directory to eval_llama3, and start evaluation. ROUGE scores are displayed after ~5 minutes.

6. Interactive Chat

Switch to the “Chat” tab, ensure the adapter path is train_llama3, and load the model. You can now converse with the fine‑tuned model, which responds in the style of the historical figure Zhuge Liang.

7. Cleanup and Follow‑Up

If you no longer need the DSW instance, stop or delete it from the PAI console to avoid further charges. Remember to use any free trial resources before they expire, or recharge your account to keep the instance running.

Conclusion

The tutorial demonstrates how to combine Alibaba Cloud PAI and LLaMA Factory to fine‑tune Llama‑3 with LoRA, achieve Chinese Q&A and role‑play capabilities, and verify the results using ROUGE scores and manual testing. The same workflow can be applied to custom business datasets to build domain‑specific large language models.