Context Engineering: Why More Context Makes Agents Worse — And 5 Fixes
This article explains why excessive context degrades AI agent performance, citing research on 'lost in the middle' and context rot, and presents five context engineering techniques — prune, order, retrieve, compress, delegate — with a practical CONTEXT.md template and a customer support case study reducing tokens from 180K to 7K and cost from $0.54 to $0.02 per reply.
Introduction: The Context Paradox
Google let Gemini play Pokémon for hundreds of hours. The longer it played, the worse it performed. After roughly 100,000 tokens of history, the agent stopped making new plans and simply repeated past actions. The model and game didn't change — only the amount of context the agent had to read before each action grew. Most users do the opposite: they paste entire documents, full chat histories, every tool, and every rule, then expect the agent to find the one sentence that matters. Usually it can't, and the more you give, the less it actually uses.
Why More Context Hurts
Information in the Middle Gets Lost
In the "Lost in the Middle" study, Stanford researchers placed the only document containing the answer at different positions in a long prompt. Models performed best when the document was at the start or end; when buried in the middle, GPT-3.5 performed worse than having no document at all.
Longer Inputs Degrade Reliability Even for Fixed-Difficulty Tasks
Chroma tested 18 models — including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3 — on tasks of identical difficulty. The only variable was input length. As input grew, answer reliability dropped.
Agents Start Copying Their Own Past Behavior
Google's Gemini 2.5 report describes the Pokémon agent: beyond 100k tokens it relied on long-term history, repeating old actions instead of planning anew. Anthropic calls this "context rot" and treats the context window as an attention budget: every added token consumes a slice.
The core insight: the context window is a budget, not a container. A larger window just lets you pay more for the same mistake.
Five Key Context Engineering Actions
1. Prune: Keep Only the Minimum High-Signal Tokens
Anthropic's principle: use the fewest high-signal tokens to get the result you want. Three categories almost never get pruned:
System prompts — cover every possible situation, most of which never occur in the current task.
Tools — the task needs 6, but 40 are provided. Anthropic's rule: if a human engineer can't say which tool to use, the agent can't either.
Examples — ten edge-case examples teach the model edge cases, not the task.
Principle: every line in context must prove its usefulness for the current step, not for some future step.
2. Order: Put Critical Content at the Start and End, Never in the Middle
Position determines attention. Put immutable rules at the top — they're read first and stay cached. Put the current goal and next step at the bottom, right before the agent acts. Manus (acquired by Meta) uses a continuously rewritten todo.md so the plan always sits at the context end, never sinking into the middle during a 50-step task.
Principle: anything the agent must never forget goes either at the start or repeated at the end.
3. Retrieve: Give the Agent a Map, Not the Whole Library
Old approach: preload everything — entire codebase, whole help center — just in case. Better: just-in-time retrieval. The agent keeps only lightweight references — file paths, links, search queries — and loads actual content only when a step truly needs it. Anthropic describes how Claude Code handles large codebases: it searches and uses head / tail to view file slices instead of stuffing whole files into context. Live retrieval also eliminates stale information: a document pasted in March stays March; a search today returns today's reality.
Principle: paste the map; let the agent fetch what the map points to.
4. Compress: Keep Notes, Not Full Transcripts
Long tasks inevitably fill the window, so content must be removed.
Discard raw tool outputs — once used, the agent doesn't need to re-read 4,000 lines; keep only the conclusion drawn.
Keep a notes file — Anthropic's own Pokémon-playing Claude kept running notes: steps trained on a route, levels still needed. Even after context reset, it could resume the plan. Same method works for any long task: let the agent read/write a NOTES.md outside the context window.
Keep failures — Manus found the opposite of common practice: leave failed actions and their errors in context. The model sees what went wrong and won't retry; if you erase them, it hits the same wall again.
Principle: compress successes into notes; keep failures verbatim in view.
5. Delegate: One Window, One Job
When a task requires reading large volumes, hand the reading to a sub-agent with a clean context window. In Anthropic's research system, each sub-agent can spend tens of thousands of tokens exploring, then return a compressed summary of 1,000–2,000 tokens. The main agent never sees the messy exploration, only the result. This is where delegation pays off: delegate reading, not decision-making. Decisions stay in one window.
Principle: if you need to read 50 pages for a one-paragraph answer, let a separate window read the 50 pages.
Worked Example: A Customer Support Inbox
Scenario: an agent replies to customer emails, with a 240-article help center, three years of tickets, and a policy change in March shortening refunds from 30 to 14 days.
Naïve approach : paste everything every call — entire help center, full customer history, 60 lines of rules — ~180k tokens. Result: after 200 emails in a week, 31 still promised 30-day refunds because old articles were still in context and longer.
Context-engineered approach :
Prune : keep 12 rules that apply to every email; move the rest to searchable articles.
Order : rules at top, customer email and reply goal at bottom, current refund policy pinned right above the email.
Retrieve : search help center, fetch max 3 articles per email, newest version only.
Compress : condense customer history into 10-line notes instead of loading 40 raw tickets.
Delegate : a sub-agent reads verbose order logs and returns a 5-line summary.
Result: per-reply context dropped from ~180k to ~7k tokens; policy-reference errors fell to zero; at $3 per million input tokens, cost per reply fell from ~$0.54 to ~$0.02.
CONTEXT.md Template
Fill once per task; only bracketed content changes:
ALWAYS FOLLOW (top): [rules that apply to every task, max 15 lines]<br>NEVER LOAD: [content to exclude: old versions, full history, raw logs]<br>RETRIEVE: search [folder / doc / site], return max [3] results, newest first<br>NOTES: read NOTES.md before each step, append one line after, keep failed attempts<br>DELEGATE: any reading task over [20] pages goes to a sub-agent, return [5] lines<br>BUDGET: [N]K tokens per call; if exceeded, compress then continue<br>PINNED (near bottom): [recently changed, must-use facts]<br>GOAL (last line): [the outcome this step must produce]
"Pinned" and "Goal" are deliberately placed at the end because the context tail is the region models miss least.
Diagnose Your Missing Link
When an agent degrades over time, symptoms point to the missing technique:
Follows rules in turn 1, violates by turn 30 → rules stuck in middle; move to top and repeat at end.
Repeats old actions instead of trying new ones → history too long; compress to notes.
Picks wrong tool → too many tools; keep only those the job needs.
Cites stale info → old docs still in context; retrieve latest and pin it.
Reads everything before acting → everything pre-pasted; provide paths and let it search.
Research, coding, writing all in one chat → one window doing five jobs; split reading out.
Six symptoms, six missing links — and a bigger context window fixes none of them.
Mistakes That Neutralize Context Engineering
"It has a 1M-token window, so paste everything" — larger window holds more, doesn't make the model read better (Chroma's data).
Changing the prompt start every call — adding timestamps or reordering tools invalidates cache for everything after. Manus calls cache-hit rate the single most important production metric; cached input costs a fraction of regular input.
Deleting error records — clean history looks tidy but teaches the agent nothing; it retries the same failed approach.
One giant system prompt for all tasks — billing rules mixed into writing context interfere with every answer.
Providing ten examples of the same type — Manus warns: the agent starts mimicking patterns instead of understanding the task.
No per-call budget — context grows until something breaks. Set a hard number; when exceeded, compress.
Where to Start Tonight
Find the agent that runs longer and performs worse.
Count how much it reads per call — most people never look.
Prune the system prompt to only rules that apply every time.
Copy CONTEXT.md, fill the brackets, pin the fact that keeps breaking.
Run the same task again; compare answer quality and bill.
Then look at what you used to paste in and ask how much of it was ever actually used.
P.S. Before you pay for a larger context window, try shrinking the context. It's cheaper and usually works better.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
