From LLM-as-Judge to Agent-as-Judge: A Technical Evolution

Problem: LLM as Judge

Using an LLM to score AI outputs by stuffing the content and a reference answer into a prompt works for tiny samples, but reliability drops dramatically when inputs are large, unstructured, or when the scoring rules become complex. Pure LLM‑only judging yields only about 60% agreement with human annotations.

Solution: Agent as Judge

The proposed remedy is to treat the LLM as a member of an evaluation team rather than the sole arbiter. The workflow combines precise scripts for deterministic metrics, LLM‑based semantic judgments, and engineering‑level cross‑checks.

Core components :

Sandbox (isolated execution environment) : each evaluation runs in a fresh cloud container equipped with Bash, Python, Node.js, and a file system. The sandbox is destroyed after use, providing isolation and the ability for the Agent to install dependencies and run scripts.

Agent (reasoning + action kernel) : an LLM‑driven autonomous entity that can invoke tools, make multi‑step decisions, and handle exceptions, effectively acting like a human evaluation engineer.

Skill (evaluation logic carrier) : a Markdown file that defines the Agent’s role, step sequence, output format, and tool permissions. Different Skills encode different evaluation scenarios, making the system plug‑and‑play.

Design principle: scripts produce facts, LLM judges facts

Deterministic metrics (e.g., CodeBLEU, coverage) are computed by scripts, guaranteeing reproducible results. Semantic judgments that require understanding are delegated to the LLM, but only after the scripts have supplied concrete evidence.

Three‑layer evaluation framework

Layer 1 – Deterministic computation : any metric that can be expressed as a formula or script is calculated directly. These anchors are indisputable and form the baseline.

Layer 2 – LLM semantic judgment (slice evaluation) : instead of a single holistic score, the LLM evaluates each sliced criterion (e.g., requirement coverage) and aggregates the results.

Layer 3 – Gate (result validator) : consistency and confidence gates cross‑check script outputs with LLM judgments. If CodeBLEU is 0.3 but the LLM gives an 8 for functional consistency, a contradiction is flagged and the Agent re‑examines the case.

Parallel SubAgents

The Skill model allows the main Agent to spawn SubAgents, each handling a narrow sub‑task (e.g., code review, knowledge retrieval, deterministic metric calculation). This parallelism reduces context length, avoids prompt contamination, and improves confidence.

Core assets: data + Skill

Evaluation data sets define *what* to evaluate, while Skills define *how* to evaluate. Both are versioned, diffable, and can be rolled back, turning the evaluation platform into an execution engine rather than hard‑coded logic.

Costs

Latency : sandbox cold‑start takes 3–5 minutes; a full Agent evaluation runs 5–15 minutes, totaling 10–20 minutes per sample.

Expense : each run consumes sandbox resources and multiple LLM API calls, making it suitable only for high‑stakes scenarios.

Engineering effort : writing effective Skills requires expertise; the same Skill can produce nondeterministic LLM judgments on repeated runs.

When to use

If the task requires running scripts, accessing files, or multi‑step reasoning, use Agent as Judge. For simple, purely textual checks (fluency, grammar, sentiment) the lightweight LLM‑as‑Judge approach remains faster and cheaper.

Practical outcomes

After adopting the Agent‑as‑Judge pipeline, consistency with human labels rose from ~60% to >85%, and “high‑score‑low‑quality” cases vanished. In 1.5 months the team completed over 800 incremental code‑generation evaluations across 35 knowledge bases. Parallel SubAgents improved stability and confidence, while data‑cleaning and scoring pipelines handled 70+ code‑generation tasks and enabled semi‑automated gold‑data set construction.

Overall, the Agent‑as‑Judge framework transforms evaluation from a single‑model scoring problem into a modular, version‑controlled workflow that can both assign reliable scores and surface concrete problem diagnostics.