Verifying Agent Task Completion: Three Success Layers, Acceptance Criteria, and Timeout Recovery

When a user asks an Agent to research a technology, create a Feishu document, and share it, the Agent may return a link and say "document created." But the link might open to half‑written content, broken references, or missing permissions. Every step returned success, yet the user still didn't get what they asked for. This article breaks down how to prove a task is truly finished.

A Document Link Hides Multiple Independent Operations

The research assistant flow: model decides what to search, search tool returns candidates, web tool fetches pages, model organizes material, then calls the document tool with title and content. The document tool then creates a document object, converts content into platform‑accepted blocks, appends blocks in batches, and configures access permissions for the recipient. Only then does it return a link.

An earlier step's success does not guarantee a later step's success. Getting a document ID only means the object exists; subsequent block writes and permission settings each have their own results. For example, if the first batch of blocks writes successfully but the second fails, the remote document may contain only half the article. If error handling merely logs "document creation failed," the recovery logic won't know which document to continue writing; if it reports success based on the initial ID, it delivers a half‑finished product.

In an interview, saying "I integrated search and Feishu APIs" doesn't explain this delivery process. Detailing which operations can partially succeed and what state each step leaves makes the project verifiable.

The 'success' Field: Who Does It Guarantee For?

Many tool interfaces return success, data, error. This structure is convenient, but we must ask: who set success and what did they check?

Some executors set success: true whenever the function doesn't throw an exception. That only proves the function returned cleanly. Whether the payload contains a business error, whether the call merely queued an async task, or whether the data satisfies the current goal still needs further judgment.

We can split success into three layers:

Call layer – validates tool name and parameters.

Execution layer – verifies the service processed the request per its interface contract.

Task layer – confirms the result fulfills the user's requirement.

In the document example: title and body passing validation is call‑layer; create and append APIs returning contract‑compliant responses is execution‑layer; complete body, correct citations, and recipient access is task‑layer acceptance.

Numbers in tool output also need scrutiny. A field like blocks_created might only count blocks prepared locally for sending, not blocks confirmed saved remotely. It records a local plan, not independently verified evidence.

Therefore, I redesign the return structure to explicitly include: document ID, per‑batch write status, permission‑handling result, and subsequent acceptance results. A top‑level success can remain for upper layers, but its computation rule must be transparent and not drift across different layers.

Acceptance Criteria Must Be Defined Before Writing

Waiting for the model to say "done" and then having another model score it turns acceptance into a reading impression. The scorer may like the prose but miss that the remote document lacks the final section.

For this task, we can pre‑define concrete conditions:

Document title matches the current task.

Body contains the required solution comparison and conclusion.

Key citations trace back to supporting sources.

Specified recipient has the agreed access permission.

If the user asked for tables, code blocks, or images, add corresponding format conditions.

Each condition has its own check method; not all need a model.

After writing, read back the remote content using the saved document ID. For the generated and confirmed content, first normalize (platforms may adjust line breaks and whitespace; raw Markdown byte‑hash comparison would falsely flag normal conversions; checking only "body not empty" would miss massive omissions). Plain‑text read‑back detects missing content but not all formatting. If tables or code blocks are required, fetch the block structure and verify type and content. If the read API paginates, traverse all pages – don't treat the first page as the whole document.

Citations need a separate check: source existence only proves the link is reachable; whether the cited material actually supports the claim in the body requires comparing the retrieved source with the claim. A model can help with semantic checking, but its input must be the actual retrieved sources and the body, not just its own summary.

Permissions must be listed individually. The application reading the document with its own token proves the app can read; whether the recipient can access requires checking against the target user and agreed permission. When possible, verify from the recipient's identity. Don't equate the service account's capability with the user's.

Thus, acceptance cost scales with delivery requirements. A simple memo checks content and access; a structured report adds format checks; a research document used for decisions adds evidence verification for key conclusions. The check scope derives from the task requirements.

Acceptance must bind to the current version. A model once generated a complete body, but that doesn't prove the final submission is that version; a first read‑back pass doesn't guarantee later rewrites. Store a summary identifier of the expected content, the target document, and the read‑back timestamp in the task record. When content changes, only re‑check affected content and format, avoiding stamping old results onto a new version.

Timeouts: First Reconcile What Already Happened

Modify the example: document creation succeeds, but appending content times out. What next?

Immediately retrying "create and write" is dangerous. A second attempt might create another document; if the first append actually executed but the receipt was lost, replaying could duplicate the same content.

A timeout means the caller didn't wait for a result; it cannot conclude the remote side did nothing. Recovery must face an "unknown result" state.

At task start, establish a stable identifier for the business intent and record its relationship with the document ID. This identifier should cover the user/tenant and the specific task; if the user explicitly asks for a new document, that's a new intent and must not reuse the old document.

After obtaining the document ID, persist each batch's content identifier and progress. The recovery program first reads that document, confirms which content already exists, then decides to continue writing, wait, or escalate to human. Never mark a batch complete just because it was "prepared for sending" locally.

A subtle race condition: the first request might still be executing. Reading back and finding a segment missing doesn't automatically mean it's safe to re‑append. Need to combine downstream idempotency capabilities, operation status, or a verifiable reconciliation method to control replay; lacking those, pause automatic writes to avoid creating duplicates.

Message deduplication at the entry point solves only part of the problem. An in‑process cache remembering a message was processed doesn't survive restarts, and doesn't cover the model actively choosing the same write tool again. Business‑action deduplication and recovery must land on persistent state and downstream operation semantics.

When the Agent Stops, It Must Report Its State

Setting a max tool‑turn limit prevents infinite loops, but the limit only answers "how many turns at most," not "did the thing get done."

If the last turn exhausts the limit and the document still misses a section, the system should record it as incomplete, saving the gap and a recovery entry point. If content is complete but permission configuration failed, that's content‑done, delivery‑blocked. Only when all required conditions are met is the task aggregated as success.

The final reply should be generated from these states. Body saved but sharing failed → "document saved but access permission not yet configured." Write result unknown → "verifying, not promising delivery yet."

The model still handles language and task‑path adjustments. But the words "completed" must be backed by execution records and acceptance results. At run end, leave a brief record: target document, completed steps, pending items, acceptance evidence, and termination reason.

This record aids troubleshooting. When a user says a section is missing, you can directly confirm the remote version, write batches, and read‑back content without guessing which prompt was poorly written. Later evaluation can separate content‑quality issues from delivery issues.

Prove Delivery Control with Fault Injection Samples

To present this design as project experience, I want to see results under faulty inputs, not just a happy‑path demo.

First, build a small fault‑injection harness: replace the document client with a controllable mock service. Make creation succeed, content append timeout. Observe whether the next recovery reuses the original document and whether it reconciles remote content before proceeding. Then simulate "content already written but receipt lost" and check for duplicate appends.

Next, test complete content but sharing failure, confirming the overall status isn't mis‑recorded as success. Let the model repeatedly call the same tool until the turn limit is exhausted, verifying the system retains the incompletion reason. Then have the remote return empty body or miss the last batch, confirming acceptance catches it.

Every sample must examine actual state and remote side effects. Normal process exit or a model‑written explanation cannot replace these observations. The mock service makes fault timing repeatable; after integrating with the real platform, still verify pagination, permissions, and format conversion on controlled documents.

During evaluation, record "run completion rate" and "task acceptance pass rate" separately. The former reflects whether the system can produce a terminal state; the latter reflects whether delivery meets requirements. Further break down incompletions into content gaps, permission blocks, unknown results, and budget exhaustion. Only then can you decide whether to fix the write path, recovery strategy, or model planning. A single overall success rate mixes these problems together.

With this, an interview answer becomes concrete:

I split document delivery into creation, content writing, permission configuration, and acceptance. After the tool returns a link, I read back the remote content using the same document ID and verify against task requirements; format and recipient permissions are checked separately. On write timeout, I enter an unknown‑result state, reconcile using persisted progress and downstream status to avoid rebuilding from scratch or duplicate appends. Turn‑limit exhaustion is recorded as incomplete; success status is aggregated from acceptance conditions.

Every judgment in that answer should trace back to your own implementation and tests. Start by picking one tool that produces remote side effects, map its partial‑success states, then add a "request sent, receipt lost" fault sample. That teaches you more about the whole project than stacking more tools into the Agent.