Boost LLM Performance: Data Augmentation & Distillation with Qwen2

Large language models (LLMs) achieve impressive text generation, translation, and summarization but require massive computational resources; model distillation transfers knowledge to smaller models, dramatically lowering cost and enabling deployment in resource‑constrained scenarios.

This guide, based on the Qwen2 family, presents a complete workflow for data augmentation and distillation.

Prerequisites

Activate PAI (DLC, EAS) paid service and create a default workspace.

Create an OSS bucket for storing training data and model files.

Step 1: Prepare Instruction Data

Follow the data format and preparation strategy: collect at least hundreds of diverse, balanced instructions, clean abnormal entries, and ensure multilingual balance when needed.

Data Format

[
    {"instruction": "在2008年金融危机期间，各国政府采取了哪些主要措施来稳定金融市场？"},
    {"instruction": "在气候变化加剧的背景下，各国政府采取了哪些重要行动来推动可持续发展？"},
    {"instruction": "在2001年科技泡沫破裂期间，各国政府采取了哪些主要措施来支持经济复苏？"}
]

Optional Step: Instruction Augmentation

Use the provided instruction‑expansion models (Qwen2‑1.5B‑Instruct‑Exp or Qwen2‑7B‑Instruct‑Exp) via PAI‑QuickStart to automatically generate similar instructions, improving the generalization of the distilled model.

Deploy Augmentation Service

Select the desired augmentation model in QuickStart and deploy the service (default configuration can be modified).

Confirm the billing dialog.

Call the Service

Obtain the service endpoint and token from the model detail page (see image).

import argparse, json, requests
from typing import List

def post_http_request(prompt, system_prompt, host, authorization,
                      max_new_tokens, temperature, top_k, top_p):
    headers = {"User-Agent": "Test Client",
               "Authorization": f"{authorization}"}
    payload = {"prompt": prompt,
               "system_prompt": system_prompt,
               "top_k": top_k,
               "top_p": top_p,
               "temperature": temperature,
               "max_new_tokens": max_new_tokens,
               "do_sample": True,
               "eos_token_id": 151645}
    return requests.post(host, headers=headers, json=payload)

def get_response(response):
    return json.loads(response.content)["response"]
# Argument parsing and usage omitted for brevity

Optional Step: Instruction Optimization

Use Qwen2‑1.5B‑Instruct‑Refine or Qwen2‑7B‑Instruct‑Refine to rewrite instructions into more detailed forms, which yields richer model outputs.

Deploy Teacher Model and Generate Responses

Deploy the teacher Qwen2 model, feed the prepared (or augmented/optimized) instructions, and collect the generated responses for distillation.

Step 5: Distill Student Model

In QuickStart, select a smaller student model (e.g., Qwen2‑7B‑Instruct), configure hyper‑parameters, and start fine‑tuning using the instruction‑response pairs.

Key Hyper‑parameters

training_strategy : sft (standard supervised fine‑tuning).

learning_rate : 5e‑5 (float).

num_train_epochs : 1 (int).

per_device_train_batch_size : 1 (int).

seq_length : 128 (int, maximum token length).

lora_dim : 32 (int, LoRA dimension; >0 enables LoRA/QLoRA).

lora_alpha : 32 (int, LoRA scaling factor).

load_in_4bit : true (bool, load model in 4‑bit).

load_in_8bit : false (bool).

gradient_accumulation_steps : 8 (int).

apply_chat_template : true (bool, prepend default chat template).

system_prompt : "You are a helpful assistant" (string).

After training, click “Deploy” to publish the student model as an EAS online service.

Batch Processing Scripts

Python scripts are provided for batch instruction augmentation, optimization, and teacher‑student generation, all using the same post_http_request and get_response helpers.