Designing Reliable AI Agents: From a Single Prompt to Stable Delivery

Problem with a single over‑long prompt

Recent advances let AI agents understand goals, call tools, generate content, analyse files and even decompose tasks. Teams often assume that extending the prompt and adding more tool calls automatically yields a stronger agent. In practice demos may run, but production quickly becomes unstable: single‑step successes hide downstream failures, results vary across runs, and cost, permissions, compliance or collaboration make the system hard to explain. The issue is not model intelligence but that a complex software system is reduced to one long prompt.

When a task requires a complex agent

If a task has a single clear input, one tool call and a directly verifiable output, a simple prompt plus a script suffices. Complexity is needed when two or more of the following appear:

The user provides a vague goal; the system must infer intent and fill constraints.

Multiple inter‑dependent stages exist, where earlier steps affect later ones.

The workflow must choose among several tools, models or external services.

Intermediate steps can fail, each requiring a distinct recovery strategy.

Generating a file does not guarantee the result’s quality.

Costs, time, concurrency, permissions, privacy or content risks must be controlled.

Execution time is long and must support pause‑and‑resume.

Results need to be auditable, showing inputs, process, evidence and final conclusions.

These tasks share open semantics together with deterministic constraints. If the architecture does not answer questions such as “what counts as completion?”, “which steps are needed?”, “which can run in parallel?”, “how to retry or switch strategies?” and “how to verify the result?”, the answers end up forced into the prompt, turning the system from “intelligent” to “uncontrollable”.

Define invariants before features

Instead of starting with a feature list (models, tools, vector stores, etc.), first define system invariants that must never be broken. A generic set includes:

Only one role may modify the global runtime state.

A versioned, readable plan must be produced before execution.

Each stage must output a structured result, not free‑form text.

Final quality must be judged on evidence, not merely on a tool’s success flag.

Hard gate‑keeping rules cannot be silently overridden.

Repairs must have a scope, a budget and a maximum number of rounds.

Credentials, private data and external content must follow least‑privilege principles.

All terminations must have explicit reasons.

The final delivery must exactly match the evaluated candidate.

No role may delete evidence, lower thresholds or falsify success.

These invariants act as a system “constitution”. Models, tools and workflows can be swapped, but the invariants must remain stable.

Four cooperating planes

Control plane

The scheduler maintains the current state and decides which role to invoke next, handling retries, rollbacks, repairs, termination and final candidate selection. The key principle is single writer : only the scheduler may change the global state. Planners, evaluators and repairers can propose actions, but without scheduler authority they would cause duplicate executions, round‑count mismatches and “each thinks it succeeded” problems.

Decision plane

The planner turns vague goals into an executable contract; the evaluator judges whether the contract is met; the repairer maps problem codes to allowed repair actions. All roles may use large models, but their outputs must be structured and constrained by schema, rule versions and state permissions.

Execution plane

The executor receives a single work item from the task graph, not a free‑form requirement. A valid work item includes task ID, dependencies, input references, expected output, timeout, max retry count, idempotency key, allowed tools and acceptance criteria. The executor must obey two boundaries:

External outputs (web pages, files, model results, tool logs) are untrusted and must be format‑checked and security‑checked.

Tool success only means the call completed; it does not guarantee business goals. The executor reports facts without over‑claiming success.

Governance plane

The governance plane, often omitted in demos, determines long‑term operability. It includes rule versions, data contracts, evidence chains, permission boundaries, cost budgets, state persistence, provenance, audit logs and final atomic delivery. Without it, teams cannot answer three crucial questions after a run: why did the agent act this way, is the result trustworthy, and can the incident be post‑mortem analysed?

Closed‑loop workflow

Receive goal and boundaries.

Planner generates a frozen plan.

Executor runs the task graph to produce candidates.

System collects evidence from the final candidate.

Evaluator performs hard gate‑checks and quality scoring.

If the score meets the threshold, deliver.

If not and budget remains, repair the affected sub‑graph.

After repair, regenerate candidates and re‑evaluate.

If maximum rounds, budget exhaustion, a hard gate, or a user decision occurs, terminate explicitly.

The article distinguishes four kinds of retry to avoid conflating them:

Provider retry – short retry after a network or service failure.

Stage retry – re‑execution of a specific task node.

Repair round – a business‑level fix based on evaluation problem codes.

New run – a fresh execution when the goal or core constraints change.

Mixing these into a single retry_count quickly makes cost and state inexplicable. The system must also detect “ineffective repair loops” where consecutive rounds produce almost identical problem codes and scores, prompting a strategy switch, human intervention or early stop.

Partitioning probabilistic and deterministic work

Reliability comes from correctly partitioning capabilities: AI handles probabilistic tasks such as understanding vague intent, extracting hidden constraints, semantic classification, content generation, relevance/quality judgement, attribution and multi‑option trade‑offs. Deterministic tools handle tasks that must be provably correct, including file existence and hash checks, schema validation, state transition legality, resource limits, permission checks, formula calculations and idempotent writes.

Incremental development roadmap

Stage 1 – Define problem and boundaries: Clarify user goals, failure costs, hard gate‑keeping rules, permission limits, non‑automatable actions and human‑approval points.

Stage 2 – Contract first: Define core data structures such as Request, Plan, StageResult, Evidence, Evaluation, RepairAttempt and FinalDelivery.

Stage 3 – Minimal closed‑loop: Implement a single happy‑path that runs through planning, execution, evaluation and delivery without using file‑existence as a proxy for acceptance.

Stage 4 – Bounded repair: Support a small set of high‑frequency problem codes and whitelist repair actions, validating dependency failures, re‑evaluation, max rounds and selection of the best historical candidate.

Stage 5 – Rule and prompt engineering: Incorporate shared rules, role contracts, schemas, versions and hashes into the production pipeline.

Stage 6 – External capability expansion: Build detection and adaptation layers to distinguish unavailable capabilities, missing credentials, transient failures, invalid inputs and inherently impossible tasks.

Stage 7 – Real‑world regression: Test not only happy paths but also timeouts, partial successes, malformed outputs, ineffective repairs, budget exhaustion, concurrency conflicts, resume execution, permission errors and mismatched final deliveries.

Pre‑delivery checklist (10 questions)

Is there only one global state writer?

Is a versioned plan generated before execution?

Does each stage produce a unified structured result?

Is the final decision based on concrete evidence rather than intermediate success?

Are hard gate‑keeping rules non‑averagable?

Can problem codes be mapped to explicit, limited repair actions?

Do repairs only rebuild the affected dependency sub‑graph?

Are provider retries, stage retries, repair rounds and new runs clearly distinguished?

Do pass, block, budget‑exhausted and max‑round terminations each have explicit end states?

Does the final delivery match the evaluated candidate’s ID, version and hash?

If three or more items cannot be answered, the system is likely still a sophisticated demo rather than a production‑ready agent.

Conclusion

Building a mature AI agent is not a prompt‑crafting contest nor a tool‑count race. A trustworthy system must have a single state, explainable decisions, readable plans, recoverable execution, verifiable results, bounded repairs, limited cost, traceable permissions and no hidden failures. When the probabilistic model is confined within a disciplined closed‑loop of scheduling, planning, evidence, evaluation and bounded repair, AI uncertainty becomes usable production power.