Agent Engineering: 9 Layers That Turn Models Into Reliable Teammates
This article presents a nine-layer framework for building reliable AI agents—Prompt, Context, Skills, Verification, Loop, Memory, Harness, Graph, and Tokens—showing that model choice matters less than the engineering around it, with evidence from Anthropic, LangChain, and DeepMind.
Why the Model Is the Smallest Part
Anthropic measured what makes one research agent outperform another and found that token usage alone explains 80% of the performance gap. LangChain kept the model fixed and only adjusted the surrounding engineering—prompt, tools, and checks—raising its coding agent's Terminal Bench 2.0 score from 52.8% to 66.5%. The model is the most debated component, yet it is the smallest part of the system you actually build. Prompt engineering is only the first layer; eight more layers sit around it.
The Nine Layers in Three Phases
Input: What Goes In
Prompt – What you ask it to do. Rule: define what "done" looks like, then let the other eight layers handle the rest. Missing symptom: the agent does something reasonable but not what you wanted.
Context – What it can see. Every token competes for attention; more context is not better. Five actions solve most problems: delete unneeded items, put rules at the top and goal at the bottom, let the agent fetch files instead of pasting them, compress prior dialogue into notes, and offload heavy reading to sub-agents with their own context windows. Rule: every line in context must prove its usefulness for the current step. Missing symptom: the agent forgets rules given twenty messages ago.
Skills – What it knows how to do. A skill is a folder with a SKILL.md file describing when to use it, step-by-step commands, and any scripts that must produce identical results each run. Only the description (~100 tokens) loads eagerly; the rest loads on demand, so an agent can carry dozens of skills. Rule: the second time you explain the same task, admit you haven't turned it into a skill yet. Missing symptom: you paste the same instructions every week and still get different results.
Agent: How It Works
Verification – How it knows the task is done. Letting the agent grade its own work leads to frequent false "done" signals. Verification hands judgment to something the agent cannot argue with: tests, a running app, a metric, or an independent checker that never saw the reasoning. Every escaped error becomes a new check so the same mistake never ships again. Rule: the producer cannot be the judge. Missing symptom: you must read everything the agent produces, so you haven't saved time.
Loop – How it fixes failures. A loop has four parts: act, check, correct, stop. Each round makes one change; uses a check the agent cannot modify; logs what was kept and reverted in a file; and defines three exit conditions up front: done, stuck (e.g., three rounds no progress), budget exhausted. Without a check, "try again" is just a second guess—DeepMind showed that letting Llama-2 re-check its own answers on grade-school math dropped accuracy from 62.0% to 36.5%. Rule: one change, one check, one decision per round. Missing symptom: the agent stops at the first draft or runs all night making things worse.
Memory – What it retains across runs. Memory turns a one-time correction into a standing rule so you don't repeat it next week. Record lessons, not full transcripts; separate rules, facts, and events; load only what the current task needs; periodically promote notes to rules; prefer the latest facts; and have a human review weekly. Rule: if it won't change the agent's next behavior, don't record it. Reflexion, which writes self-summaries of failures and re-reads them, reached 91% on HumanEval versus 80% for GPT-4 alone. Missing symptom: you correct the same issue every Monday.
System: How It Scales
Harness – The environment the model runs in: workspace map, available tools, inviolable rules, checkers, memory, and stop conditions. This is why the same model delivers reliably for one team but says "done" and breaks things for another. Rule: when the agent fails, fix the harness before blaming the model. Missing symptom: Tuesday works, Wednesday breaks, nobody knows why.
Graph – Who does what. Before adding a second agent, draw the work: steps are nodes, handoffs are edges, each agent owns a slice. Most multi-agent failures happen at the graph level: two agents doing the same step, steps with no owner, one agent becoming a bottleneck. Parallel agents suit cleanly separable work (e.g., research); for sequential dependencies, parallelism often costs more than it yields. Rule: draw the work first, then recruit agents. Missing symptom: more agents, same output, bigger bill.
Tokens – What it all costs. A single agent uses ~4× the tokens of a chat session; a team uses ~15×. Six levers keep cost flat: measure tokens per task, cache invariant parts of each call, route simple steps to smaller models, delete unused context, set per-agent caps, and batch deferrable work. Rule: an agent without a cap isn't autonomous, it's unsupervised. Anthropic's multi-agent research system outperformed a single agent by 90.2% but consumed ~15× the tokens of one chat. Missing symptom: quota exhausted by Wednesday, no clue which agent burned it.
Build Order: Three Weeks, Left to Right
Most teams reverse the order: day one they deploy five agents, a team chat, and a dashboard—with zero checks. Build in this sequence:
Week 1 – Input : one agent, one weekly task, a prompt that defines "done", clean context, and the task written as a skill.
Week 2 – Agent : add an indisputable check, wrap the agent in a loop with the three exit conditions, and create a memory file for each correction.
Week 3 – System : only now add a second agent; first draw the graph, put both agents in the same harness, and set a token cap per agent.
Each phase stands on the previous one: a loop without checks just repeats errors; a team without a graph just parallels errors.
AGENT.md: One File, Nine Lines
A single file, one line per layer; only the bracketed parts change:
PROMPT: [task] is done when [completion criteria, one line] CONTEXT: for this task load [only these files], rules at top, goal at bottom SKILLS: when [trigger phrase] appears use [skill name] VERIFY: done only when [test / metric / live check] passes, run by [not your own checker] LOOP: one change per round, stop on [done] or [stuck 3 rounds] or [budget gone] MEMORY: every correction logs one line to [memory.md], newest facts first HARNESS: you may use [these tools], never [these 3 irreversible actions] GRAPH: you own [these steps], hand off to [next agent] when [condition] met TOKENS: max [N] rounds, [N] tokens per task; [simple steps] use small model
Nine lines, each blocking one failure path.
Diagnose by Symptom
When an agent underperforms, the symptom points to the layer to check first:
Does something reasonable but not what you wanted → Prompt
Forgot a rule from twenty messages ago → Context
You re-paste the same instructions weekly → Skills
Says "done" but it isn't → Verification
Stops at first draft or never stops → Loop
You correct the same thing every Monday → Memory
Worked Tuesday, broken Wednesday, no explanation → Harness
More agents, same output → Graph
Quota gone by Wednesday → Tokens
Nine symptoms, nine layers—and a bigger model fixes none of them.
Four Mistakes That Waste All Nine Layers
Waiting for the next model release. Every number above is a layer improvement. New models make working layers better; they cannot repair broken ones.
Starting with a team. Teams amplify everything a single agent does, including its errors and token bill.
Copying someone else's whole config. Their AGENT.md hardens their past mistakes. Yours must come from your own corrections.
Solving everything with longer prompts. Most failures live in layers 2–9. Longer prompts only overload layer 1.
Building all nine layers at once. Then no one knows which layer actually helped. Add one, measure, then add the next.
Where to Start Tonight
Pick the task you hand to an agent most often.
Read the symptom list above and find the layer that breaks first.
Open the corresponding article in this series and do its "Where to Start Tonight" section.
Copy the AGENT.md template, fill in the layers you already have, leave the rest blank.
Next week, fill in one more line.
Nine weeks later you'll have the agent others are still waiting for a new model to give them. Postscript: the model is rented; the layers are yours.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
