Why Fine‑Tuning Large Models Is Now Ridiculously Easy

What is Unsloth?

Unsloth is an open‑source toolkit for fine‑tuning large models. It supports full‑parameter fine‑tuning (FFT), SFT, LoRA, QLoRA, pre‑training, FP8, and a wide range of model types, including text, vision, TTS, and embedding models. Quantized models such as Kimi K2.5, GLM‑4.7‑Flash, and MiniMax M2.1 are listed as examples of models that run with Unsloth.

Core advantages

Speed : up to 2× faster training.

Memory : GPU memory usage reduced by ~70% with negligible accuracy loss.

Model coverage : works with any model that runs on transformers, including quantized variants.

Reinforcement learning : supports GRPO, GSPO, DrGRPO, DAPO, PPO, DPO with up to 80% memory savings; inference‑model training possible with 5 GB GPU memory.

Zero accuracy loss : optimizations are exact, not approximate.

Multi‑platform deployment : export to GGUF, vLLM, SGLang, or Hugging Face.

Hardware compatibility : NVIDIA (V100 → RTX 50 series, Blackwell), AMD, Intel, DGX Spark.

Implementation : all kernels written in OpenAI’s Triton language with a custom manual back‑propagation engine.

Installation

Linux/WSL: pip install unsloth Windows (additional steps required): install NVIDIA driver, Visual Studio C++, CUDA Toolkit, PyTorch, then run the same pip install unsloth command.

Docker (zero‑configuration): docker pull unsloth/unsloth Upgrade to the latest version:

pip install --upgrade --force-reinstall --no-cache-dir unsloth unsloth_zoo

VS Code + Colab free fine‑tuning

Install the Google Colab extension in VS Code (Cmd+Shift+X or Ctrl+Shift+X, search "Colab").

Clone the notebook repository:

git clone https://github.com/unslothai/notebooks
cd notebooks

Open the desired notebook, e.g. nb/Qwen3_(4B)-GRPO.ipynb.

Select Kernel → Colab , authorize the Google account, and choose the free T4 GPU.

Click Run All and wait for training to finish.

Community‑trained models (Hugging Face)

TeichAI – GLM‑4.7‑Flash‑Claude‑Opus‑4.5‑High‑Reasoning‑Distill : 30 B model distilled from Claude 4.5 Opus, >65 000 downloads.

Zed – Qwen Coder fine‑tuned : 7 B model optimized for coding, runs smoothly on consumer‑grade GPUs.

DavidAU – Llama‑3.3‑8B fine‑tuned : multiple variants for role‑play, instruction following, and domain‑specific knowledge.

Supported training methods

MoE acceleration: 12× speed‑up and 35% memory reduction for MoE models (DeepSeek, GLM, Qwen).

GRPO reinforcement learning: enables training of inference models with only 5 GB GPU memory.

Extended context: on an 80 GB A100, a 20 B model can handle 500 k tokens (13× increase over standard pipelines).

FP8 reinforcement learning: allows GRPO on consumer GPUs such as RTX 4060.

Vision RL and TTS fine‑tuning: supports models like sesame/csm-1b and openai/whisper-large-v3.

Performance comparison (Alpaca benchmark)

Configuration: batch = 2, gradient accumulation = 4, LoRA rank = 32, QLoRA on all linear layers.

Llama 3.1 (8 B) : standard context ≈ 6 K tokens; Unsloth reaches 342 K tokens (≈ 57× increase).

Llama 3.3 (70 B) : on an 80 GB A100, standard context ≈ 6.8 K tokens; Unsloth reaches 89 K tokens (≈ 13× increase).

Gains are attributed to a collaboration with Apple on Cut Cross Entropy and Unsloth’s custom RoPE & MLP Triton kernels.

Quick‑start code example (QLoRA fine‑tuning Llama 3.1 8 B)

from unsloth import FastLanguageModel

# Load 4‑bit quantized model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/llama-3.1-8b-unsloth-bnb-4bit",
    max_seq_length = 2048,
    load_in_4bit = True,
)

# Add LoRA adapter
model = FastLanguageModel.get_peft_model(
    model,
    r = 32,            # LoRA rank
    lora_alpha = 32,
    lora_dropout = 0,
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
)

# Configure trainer (using HuggingFace TRL)
from trl import SFTTrainer
from transformers import TrainingArguments

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = your_dataset,  # your data
    args = TrainingArguments(
        per_device_train_batch_size = 2,
        gradient_accumulation_steps = 4,
        num_train_epochs = 1,
        learning_rate = 2e-4,
        output_dir = "outputs",
    ),
)

trainer.train()

# Export to GGUF for local inference (Ollama / llama.cpp)
model.save_pretrained_gguf("my_model", tokenizer, quantization_method = "q4_k_m")

Free notebook catalog (each URL is a direct Colab link)

OpenAI gpt‑oss (20 B) – SFT : https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/gpt-oss-(20B%20)-Fine-tuning.ipynb

OpenAI gpt‑oss (20 B) – GRPO : https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/gpt-oss-(20B%20)-GRPO.ipynb

Qwen3 (4 B) – GRPO : https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen3_(4B%20)-GRPO.ipynb

Qwen3 VL (8 B) – Vision GRPO : https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen3_VL_(8B%20)-Vision-GRPO.ipynb

Gemma3 (4 B) – Vision fine‑tuning : https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Gemma3_(4B%20)-Vision.ipynb

Gemma3N (4 B) – Conversational fine‑tuning : https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Gemma3N_(4B%20)-Conversational.ipynb

Llama 3.1 (8 B) – Alpaca fine‑tuning : https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B%20)-Alpaca.ipynb

Llama 3.2 (1 B/3 B) – Conversational fine‑tuning : https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.2_(1B_and_3B%20)-Conversational.ipynb

Orpheus (3 B) – TTS synthesis : https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Orpheus_(3B%20)-TTS.ipynb

FP8 Qwen3 (8 B) – FP8 GRPO : https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen3_8B_FP8_GRPO.ipynb

References

GitHub project: https://github.com/unslothai/unsloth

Documentation: https://unsloth.ai/docs

Free‑run notebook example (FP8 GRPO on Qwen3 8 B): https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen3_8B_FP8_GRPO.ipynb

Official notebook list: https://unsloth.ai/docs/get-started/unsloth-notebooks