10 Essential Large‑Model Fine‑Tuning Techniques for AI Product Managers

1. Full Finetuning (Full Finetuning)

Core principle : Update all model parameters to fully adapt to a new task.

Definition : Update every parameter of the model.

Goal : Maximize performance on the target task, at high computational cost.

Applicable scenario : When the downstream task differs greatly from the pre‑training objective (e.g., switching from language generation to text classification).

Finetuning steps :

Data preparation – collect and preprocess labeled data (cleaning, tokenization, etc.).

Model loading – load a pretrained model such as BERT or GPT with its weights.

Parameter configuration – set hyper‑parameters (learning rate e.g., 1e‑5, batch size, epochs).

Finetuning training – train end‑to‑end on the labeled data using an optimizer like AdamW to minimise task loss (e.g., cross‑entropy).

Evaluation & tuning – evaluate on a validation set (accuracy, F1) and adjust hyper‑parameters (learning‑rate decay, early stopping).

Deployment – save the best model for inference.

2. Frozen Layers Finetuning (Frozen Layers Finetuning)

Core principle : Only update the top‑most layers while keeping lower layers frozen.

Definition : Update parameters of the model’s top layers; freeze the rest.

Goal : Preserve pretrained low‑level features and reduce over‑fitting risk.

Applicable scenario : Tasks similar to the pre‑training task (e.g., text classification on a language‑model pretrained backbone).

Finetuning steps :

Load the pretrained model and freeze the lower layers (e.g., the first few Transformer layers).

Add task‑specific head layers (e.g., a fully‑connected classification head).

Configure a lower learning rate than full finetuning.

Train – only the top‑layer parameters are updated.

Evaluation & tuning – monitor validation performance and adjust the head or learning rate.

3. LoRA (Low‑Rank Adaptation)

Core principle : Simulate parameter changes via low‑rank decomposition, updating only a small set of low‑rank matrices.

Definition : Update a small low‑rank matrix ΔW = A·B while keeping the base weight W_base unchanged.

Goal : Reduce the number of updated parameters while maintaining model performance.

Finetuning steps :

Select weight matrices in key layers (e.g., attention or feed‑forward layers of a Transformer).

Initialize low‑rank matrices A ∈ ℝ^{d×r} and B ∈ ℝ^{r×d} (e.g., r = 8).

Compute ΔW = A·B.

Update the weight: W_new = W_base + ΔW.

Train – only A and B are optimized; W_base remains frozen.

Evaluate the finetuned model.

4. Prefix Tuning (Prefix Tuning)

Core principle : Introduce task‑specific prefix vectors that are concatenated with the input before feeding it to the model.

Definition : Add a prefix vector P to the input sequence, forming [P; X].

Goal : Guide model generation with minimal parameter updates.

Variants : Ptuning v2 uses discrete token embeddings for the prefix.

Finetuning steps :

Generate a fixed‑length prefix (e.g., 100 tokens) and initialise P ∈ ℝ^{L×d} (L = length, d = hidden dimension).

Concatenate P with the input sequence X to form [P; X].

Train – only the prefix parameters P are optimized; the rest of the model is frozen.

Inference – use the optimized prefix to generate task‑relevant outputs.

5. RLHF (Reinforcement Learning from Human Feedback)

Core principle : Combine supervised finetuning (SFT) with reinforcement learning, using human preference data to optimise model outputs.

Goal : Align model outputs with human values (e.g., for dialogue systems or content generation).

Finetuning steps :

SFT – train on labeled input‑output pairs.

Reward model training – collect human preference data (e.g., "Output A is better than Output B") and train a reward model to predict quality.

Reinforcement learning – apply policy‑gradient methods to maximise the reward model's score.

Iterative optimisation – repeat SFT and RL phases to progressively improve output quality.

6. Adapter (Adapter Finetuning)

Core principle : Insert lightweight adapter modules between existing layers.

Definition : Each adapter consists of two fully‑connected layers with a bottleneck (x → Linear1 → ReLU → Linear2).

Goal : Learn task‑specific features via the adapters while keeping the main model frozen.

Finetuning steps :

Insert adapters after each attention and feed‑forward layer.

Randomly initialise adapter weights.

Train – only adapter parameters are updated; the original model parameters stay frozen.

Merge output – adapter output is added to the original layer output via a residual connection.

7. QLoRA (Quantization + LoRA)

Core principle : Combine model quantisation (e.g., 4‑bit) with LoRA to lower memory and compute requirements.

Goal : Deploy extremely large models on edge devices.

Finetuning steps :

Model quantisation – use tools such as LLM.Q or Hugging Face bitsandbytes to compress weights to 4‑bit.

Insert LoRA – add low‑rank matrices A and B into the quantised model.

Train – only A and B are optimised; the quantised base weights remain frozen.

Inference acceleration – the quantised model consumes less VRAM during inference.

8. UPFT (Unsupervised Prefix Finetuning)

Core principle : Guide inference with an initial prefix without any labeled data.

Definition : Use a prefix to steer generation, relying on "prefix consistency" to reduce dependence on annotated data.

Goal : Minimise the need for labeled datasets.

Finetuning steps :

Design a task‑relevant prefix (e.g., "Solve a math problem:").

Generate constraints – force the model to produce tokens only after the prefix.

Unsupervised training – maximise coherence between the prefix and subsequent generated content.

Inference – generate outputs conditioned on the optimized prefix.

9. Quantization‑aware Finetuning (Quantization‑aware Finetuning)

Core principle : Preserve model performance while quantising by finetuning the low‑precision model.

Goal : Balance accuracy with resource consumption.

Finetuning steps :

Model quantisation – use TensorRT, DeepSpeed, etc., to convert the model to FP16, INT8, etc.

Finetuning configuration – set quantisation‑aware training parameters (e.g., dynamic range adjustment).

Train – finetune on the quantised model to optimise low‑precision parameters.

Deployment – the quantised model runs faster on edge hardware.

10. Hugging Face PEFT (Parameter‑Efficient Finetuning Library)

Core principle : Provide a unified interface that integrates multiple PEFT methods (LoRA, Adapter, Prefix Tuning, etc.).

Goal : Simplify the finetuning workflow.

Finetuning steps (example with LoRA) :

pip install transformers peft

from peft import LoraConfig, get_peft_model
model = AutoModel.from_pretrained("bertbaseuncased")

lora_config = LoraConfig(
    r=8,                # low‑rank dimension
    lora_alpha=16,
    target_modules=["query", "key", "value"],
    lora_dropout=0.1,
)
model = get_peft_model(model, lora_config)

from transformers import Trainer, TrainingArguments
training_args = TrainingArguments(
    output_dir="./results",
    learning_rate=1e-3,
    per_device_train_batch_size=4,
    num_train_epochs=3,
)
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=val_dataset,
)
trainer.train()

Selection Advice

Abundant resources – choose Full Finetuning or RLHF.

Lightweight needs – LoRA, QLoRA, or Prefix Tuning.

No labeled data – adopt UPFT.

Edge deployment – Quantization‑aware Finetuning or QLoRA.

Rapid development – use the Hugging Face PEFT library.

Comparison Table