How to Fine‑Tune LLMs in 2026: Overcome the 30‑40% Error Wall with GRPO and RULER

Every team that builds a product on top of large language models eventually encounters the same obstacle: despite detailed system prompts, few‑shot examples, and temperature tuning, the agent still makes mistakes 30‑40% of the time and never learns from those errors.

01. Fine‑tuning breaks the wall

If you use GPT or Claude you share the same model, capabilities, and cost with everyone else, giving no competitive edge. By fine‑tuning a smaller open‑source model on your specific task, you can outperform a model that is 100× larger while reducing both cost and latency.

02. SFT vs. Reinforcement Fine‑tuning

Most developers know Supervised Fine‑Tuning (SFT): collect input‑output pairs and let the model imitate them. SFT teaches the model *what to say* but not *how to succeed* in multi‑step, tool‑calling, or search‑driven agents. Reinforcement Fine‑Tuning (RFT) adds a reward signal so the model can discover optimal strategies through trial and error.

Analogy: SFT = reading a textbook (memorising known answers); RL = on‑the‑job training (learning from attempts, errors, and feedback).

03. How GRPO works

GRPO (Group Relative Policy Optimization) is the most popular RFT algorithm today. Instead of training a single model to assign absolute scores, GRPO generates multiple completions for each prompt and performs a relative ranking.

For each prompt the workflow is:

Sample a set of N completions from the current model.

Score each completion with a reward function.

Normalize scores within the group to compute a relative advantage over the group average.

Update the model to reinforce behaviours above the average and suppress those below.

Only the relative order matters; absolute values such as 0.3, 0.5, 0.7 or 30, 50, 70 are interchangeable.

04. ART: Agent Reinforcement Trainer

GRPO is powerful, but applying it to real‑world agents requires a suitable framework. ART (Agent Reinforcement Trainer) is a 100% open‑source library from OpenPipe that brings GRPO into any Python application.

Most RL libraries target simple chat bots (single input, single output). Real agents need tool calling, document search, and multi‑turn reasoning. ART provides:

Native support for tool calls and multi‑turn dialogues.

Integration with LangGraph, CrewAI, and ADK.

Efficient GPU utilisation during training.

Architecture: the client runs the agent code, records each action into a Trajectory (the full execution trace), and sends inference requests to the backend. The backend uses vLLM for fast inference and an Unsloth‑powered GRPO loop for training. After each training step a new LoRA checkpoint is automatically loaded into the inference server.

The full training loop repeats:

Client sends an inference request.

Backend generates model output.

Agent takes an action in the environment (tool call, search, etc.).

Environment returns a reward.

GRPO updates the model.

New LoRA checkpoint is loaded for the next inference.

Repeat, with each cycle improving the model slightly.

05. RULER: No hand‑crafted reward functions

Defining a good reward function is the hardest part of RL. RULER (Relative Universal LLM‑Elicited Rewards) eliminates this bottleneck by using an LLM‑as‑judge to compare multiple agent trajectories and produce a ranking, requiring no labeled data.

Key insights:

Asking an LLM to assign a raw 0‑10 score yields unstable results.

Asking it to pick the best trajectory among several attempts is far more reliable.

Because GRPO only needs relative scores, the absolute magnitude of the LLM‑judge’s output is irrelevant.

RULER workflow (three steps):

Generate N trajectories for a scenario.

Pass them to the LLM judge, which assigns a 0‑1 score to each.

Use these scores directly as the GRPO reward.

No reward function writing, no labeled data collection.

06. Putting it all together: a practical notebook

The author provides a runnable notebook that uses ART to train a 3B model via reinforcement learning, enabling it to operate on any MCP server. The notebook automatically:

Queries the server for available tools.

Generates a batch of tasks that exercise those tools.

Trains the model with automatic RULER evaluation on those tasks.

Additional examples are available in the ART GitHub repository.

Thank you for reading!