Fine-Grained Evaluation of AI Agents: Designing a Comprehensive Testing Framework

1. Introduction

Good evaluations enable teams to ship AI agents confidently; without them problems surface only in production.

1.1 From “Usable” to “Useful”

When an agent is launched, users report wrong answers, slow responses, or irrelevant replies, but developers cannot pinpoint the root cause because end-to-end black-box testing only tells that the result is wrong.

1.2 Limitations of Traditional Evaluation

Metrics such as BLEU and ROUGE measure text similarity and are insufficient for agents that must understand intent, make decisions, call tools, and complete tasks. The article lists six concrete drawbacks: coarse granularity, hallucination blindness, ignoring intermediate steps, masking cost‑efficiency, lacking multi-turn coverage, and mismatch with real-world inputs.

2. Design of a Fine-Grained Evaluation System

The system follows three principles: architecture‑aligned evaluation, action‑oriented metrics, and product‑oriented sustainability. It evaluates the four core modules—Perception, Planning, Memory, and Tool—independently.

2.1 Why Module-Level Evaluation?

Example: a drop in task completion from 85 % to 60 % could be caused by perception mis‑recognizing intent, planning routing errors, memory loss, or tool‑parameter mistakes. Module-level metrics act like a white-box diagnostic.

2.2 Quality × Cost × Performance

Quality : correctness, completeness, faithfulness, hallucination rate, multi-turn completion.

Cost : model calls, tool calls, token consumption, latency of first token.

Performance : end-to-end latency and module-level latencies.

3. Metric Definitions

3.1 End-to-End Metrics (Six “User-Perceived” Indicators)

Task Completion Rate – does the agent solve the user’s request?

Multi-Turn Completion Rate – can the agent maintain context over 2‑5 turns?

Instruction-Following – does the output obey the required format?

Hallucination Rate – is the answer faithful to retrieved knowledge?

Abnormal-Input Handling – does the agent gracefully handle empty or malformed inputs?

User Satisfaction – derived from online feedback scores.

3.2 Perception Module Metrics

Intent Recognition Accuracy / Recall / Precision (three-way “tri-test”).

Multi-Intent Recognition Rate.

Fuzzy-Intent Clarification Rate.

Degradation Trigger Accuracy.

3.3 Planning Module Metrics

Routing Decision Accuracy (Skill vs. Knowledge-Base).

Tool-Call Decision Accuracy.

Planning Path Score (expert evaluation).

3.4 Memory Module Metrics

Short-Term Memory Retention Rate – does the agent recall entities from previous turns?

Long-Term Retrieval Precision / Recall – relevance of retrieved knowledge chunks.

Memory Decay Curve – retention over 5+ turns.

3.5 Tool Module Metrics

MCP & Skills Load Success Rate.

Tool-Call Accuracy, Success Rate, and Parameter Mapping Accuracy.

4. Evaluation Datasets

Eight dataset families are constructed to cover “basic coverage + targeted probes”: Basic Skills, Knowledge Q&A, Multi-Turn Dialogue, Abnormal Input, Tool Call, Multi-Intent, Fuzzy-Intent, and Long-Dialogue Decay. Each dataset includes a JSON schema with fields such as id, sceneCode, userInput, expectedSkill, expectedRoute, expectedIntent, and evaluationMode (E2E_MOCK or E2E_REAL).

Example of a basic-skill case:

{
  "id": "BF-TRACE-001",
  "sceneCode": "i18n-ic-trace-analyzer",
  "sceneName": "Trace Analysis",
  "userInput": "traceId:2116440e17706430209582423d0733",
  "expectedSkill": "i18n-ic-trace-analyzer",
  "expectedRoute": "SKILL_HIT",
  "expectedIntent": "i18n-ic-trace-analyzer",
  "datasetType": "BASIC_FUNCTION",
  "evalMode": "E2E_MOCK",
  "mockDataId": "MOCK-TRACE-001",
  "referenceOutput": "Agent should invoke the trace analysis tool..."
}

Mock data are stored separately and injected when evalMode is E2E_MOCK, ensuring reproducible results.

5. LLM-as-Judge Framework

Each metric is implemented as a “Judge Task” that receives the agent output, the execution trace, and the expected annotation. Tasks follow four design principles: single responsibility, chain‑of‑thought reasoning before decision, inclusion of positive and negative examples, and structured JSON output.

Binary-decision tasks (e.g., Task Completion) output {"passed":true,"reasoning":"..."}. Multi-label tasks (e.g., Multi-Intent) output matched/expected/missed lists. When a higher-level decision (e.g., routing) fails, downstream tasks that depend on that decision are marked as “error” and excluded from aggregation, preventing cascade failures.

6. Execution Engine

The engine orchestrates four steps for each test case: input construction (session ID, mock injection), full agent run (Perception → Planning → Memory → Tool → Generation), trace collection (module latencies, token counts, tool logs), and Judge evaluation. Multi-turn cases reuse a shared session ID and wait for persistence before sending the next turn.

Parallel execution uses three threads with per-case timeout of 120 s and up to two retries for transient errors.

7. Visualization Dashboard

The dashboard shows total cases, pass/fail counts, and three columns for Quality, Cost, and Performance. Radar charts display module-level quality rates; bar charts present token consumption and latency; progress bars highlight scenario-level pass rates.

8. Conclusions and Future Work

The fine-grained system transforms agent evaluation from a single black-box score into a diagnosable pipeline, enabling targeted improvements. Planned extensions include automatic triggering on code or model changes, longitudinal trend analysis, multi-model A/B testing, and continuous enrichment of the evaluation datasets from real-world failure cases.