Agent Evaluation Blind Spot: Set Coverage Misses Precondition Ordering

The author discovered a critical blind spot in an offline regression evaluation for a customer‑service Agent. In a damaged‑item refund case, the Agent called oms.get_order, policy.search, and then finance.issue_refund (50 USD), but never invoked attachment.inspect. The evaluator gave a score of 0.55 because the evidence set was incomplete (2/3 required reads) and a missing_evidence_cap rule capped the total.

By manually appending a successful attachment.inspect call after the refund, the same evaluator produced a perfect 1.0 — all five sub‑scores maxed, no caps triggered. The tool set was identical; only the order changed. This reveals that the evaluator uses simple set coverage: it checks whether each required tool name appears with ok=true anywhere in the trajectory, ignoring temporal precedence.

Four Tools Succeeded, but the Causal Order Was Wrong

The refund policy demands three pre‑conditions before a write: order facts, damage verification, and policy decision. A correct sequence is:

查订单 → 验附件 → 查政策 → 退款

The counter‑example sequence was:

查订单 → 查政策 → 退款 → 验附件

Both trajectories contain the same four successful tool calls, but the latter represents “refund first, find evidence later” — analogous to signing an approval after the transfer has already occurred.

Set Coverage Checks Presence, Not Precedence

The evaluator iterates the required read tools ( oms.get_order, attachment.inspect, policy.search) and marks each as satisfied if any call with ok=true exists in the full trace. The original trace scored 2/3 (0.666…) on evidence; after adding the late attachment inspection, evidence became 1.0. Because the refund had already executed, the missing_evidence_cap had reduced the original total from 0.933… to 0.55; once evidence reached 1.0, the cap disappeared and all sub‑scores (amount, policy, reply) also hit 1.0.

This set‑based approach loses ordering, object binding, and version information. For read‑only actions (search, summarise) order may only affect quality; for write actions (refund, place order, send message, change permissions) order is a safety rule.

Don’t Let Late Evidence Retroactively Authorise a Write

The fix is to move from a task‑level checklist to per‑write pre‑condition gates. Each write action declares its own required reads, and each read must have an action_index smaller than the write’s index. Using a monotonic sequence number from the event stream (not wall‑clock timestamps) avoids concurrency issues.

Object binding is equally critical: the attachment must belong to the current order, the verification result must match the current claim, and the policy must apply to the same market and intent. Otherwise an Agent could reuse another order’s attachment to satisfy the tool‑name checklist.

A structured PreconditionReceipt can capture this:

{
  "type": "PreconditionReceipt",
  "order_id": "O1",
  "attachment_id": "A1",
  "policy_id": "P",
  "damage_verified": true,
  "checked_at_seq": 2,
  "version": 3
}

The refund gateway consumes this receipt before execution, verifying status, object identity, version, and sequence. Receipts carry explicit states ( verified, stale, consumed) and are invalidated when the underlying order, attachment, or policy changes. A consumed receipt cannot be reused for another refund.

This gate belongs in the runtime or tool gateway. The evaluator can detect the error in replay, but cannot undo a sandbox ledger entry; the gateway prevents the write from reaching the executor in the first place. Both share the same invariant rules.

Before Launch, Run at Least Five Evidence Counter‑Examples

The author proposes five minimal regression tests for the evidence gate:

Missing attachment inspection → refund must be rejected.

Attachment inspection arrives after refund → cannot retroactively approve the side effect.

Inspection tool times out or fails → “called” does not count as valid evidence.

Attachment bound to a different order → even if result says damaged, it cannot authorise the current refund.

Attachment or policy updated → old version evidence must immediately become stale.

Tests must assert that the refund executor was never invoked, no ledger entry was created, the rejection reason points to the specific failed pre‑condition, and that after supplying valid evidence the flow can continue from the safe point rather than restarting the entire task.

Every high‑risk write tool must answer five questions: what pre‑conditions are needed? which object does each evidence bind to? before which step must the evidence appear? who performs the runtime check? how to prove the write never happened when a condition is missing?

Agent reliability is not about eventually calling all tools; it is about ensuring that evidence can block an undesired write before it occurs.