Anthropic Deep Dive: Context Engineering Lessons from Real‑World R&D

1. A Larger Window Can Be Messy

Anthropic defines context as everything the model sees: system instructions, tool definitions, MCP‑injected data, external resources, message history, and accumulated tool results. Each Agent action changes the material at hand, adding new code, test outcomes, or stale requirements. Without filtering, the window simply accumulates, and the most useful facts are often the ones that changed most recently. The paper calls the degradation of retrieval accuracy as "context rot" and notes that token limits differ across models, with no universal safe token line.

2. Information Has a Shelf Life

In practice, information can be grouped into three categories:

Stable boundaries – rarely change (e.g., API compatibility, approval requirements). These are best supplied at task start.

Current state – changes rapidly (e.g., current branch, diff, service status, latest test results). Before a critical step, the Agent should reread the authoritative source.

Raw evidence – large artifacts such as full logs, call chains, execution plans, design docs. Keep only a summary, range, and address; retrieve the full artifact only when needed.

The analogy is a shift handover: a permanent manual stays on the bench, live alerts are consulted in real time, and historic logs are fetched on demand.

3. Prompts Must Speak Clearly

Prompt engineering is hard because of granularity. Over‑specifying leads to brittle if tools or tasks change; under‑specifying leaves the model without actionable criteria. Anthropic calls the sweet spot the "Goldilocks zone": concrete goals, boundaries, and completion conditions, leaving room for the model to make on‑the‑fly judgments.

The recommended iterative method starts with the smallest prompt version, then adds commands and examples after real failures appear. An example for fixing a payment‑callback duplication issue is provided:

Goal: Fix duplicate callbacks causing double credit
Boundary: Do not modify external API; do not touch production data; deployment requires approval
Tools: Can read code, run tests, retrieve logs; on failure return evidence location
Completion: Regression tests pass; integration test results clear; provide Diff, logs, and rollback instructions

The prompt must capture the goal, boundaries, tools, and completion criteria; the format is illustrative, not a strict template.

4. Tool Returns Are Contracts

Tools act as contracts between the Agent and external systems. Anthropic requires contracts to be unambiguous, error‑handleable, and token‑efficient. A minimal contract should include status, conclusion, scope, evidence location, and next step. Example:

Status: failed
Conclusion: idempotent regression test still fails
Scope: unit tests run; integration tests not yet run
Evidence: artifacts/test-payment-callback.log
Next step: inspect first failing assertion, then decide whether to adjust transaction boundaries

Contracts must also convey failure modes (e.g., permission denied, incomplete data, timeout) so the Agent knows when a result is missing.

5. Retrieve Data Just‑In‑Time

Loading all potentially relevant documents at the start wastes token budget and introduces stale information. Instead, keep lightweight indexes (file paths, saved queries, URLs) and fetch full content only when the Agent truly needs it. Claude Code mixes pre‑loaded files (e.g., CLAUDE.md) with just‑in‑time grep or head / tail for large data analysis.

In a payment‑callback workflow, the schedule is:

Before modification: load API constraints, relevant code, existing regression cases.

Before test execution: reread current diff, test configuration, dependency state.

Before handoff: load latest test results, uncovered areas, rollback requirements.

Historical logs: keep query and evidence address, expand only when debugging.

Just‑in‑time retrieval adds latency and can misroute if file names or timestamps are wrong, so proper naming and cataloging are essential.

6. Long Tasks Need State Preservation

When a task spans minutes to hours, the context window cannot hold everything. Anthropic suggests three strategies:

Compression – retain architecture decisions, unresolved issues, implementation details; discard duplicate tool results; keep references to the five most‑recent files.

Structured notes – for tasks with clear milestones, record confirmed idempotent solutions, pending integration tests, and evidence locations so a new window can pick up where the previous left off.

Sub‑Agents – for well‑bounded research tasks, let a sub‑Agent handle detailed search while the main Agent receives a concise summary. This isolates noise but adds handoff cost.

Choosing a strategy should consider the cost of state transfer versus the benefit of keeping context concise.

7. Acceptance Must Inspect Process

Correct final answers do not guarantee a robust context setup. Evaluation should check:

Result correctness – does the task goal succeed?

State consistency – are new facts aligned with the plan?

Evidence completeness – can you trace back to tests, uncovered items, and raw logs?

Process controllability – are tool calls, human corrections, token usage, and latency stable?

Comparisons should focus on signal‑to‑noise ratio rather than raw token count; the model should see critical facts earlier and avoid reworking stale information.

8. Incremental Improvements

Start with three small changes:

Improve tool contracts to return "conclusion, scope, evidence, next step".

Add a handoff checklist capturing goal, current facts, unresolved items, and evidence locations.

Create a regression suite that varies only one variable at a time (e.g., tool return format, handoff template) and observe both outcome and process metrics.

After each change, monitor execution traces for new failure modes such as missing pagination support, lost state provenance, or inability to retrieve evidence.

Conclusion

Even with larger context windows, information freshness, priority, and evidence provenance must be actively managed. Treat context engineering as runtime information scheduling: let timely data flow in, update changed facts, set aside unused material, and retain raw evidence for audit. This keeps AI agents reliable throughout evolving development tasks.

References

Prithvi Rajasekaran, Ethan Dixon, Carly Ryan, Jeremy Hadfield. "Effective context engineering for AI agents" (Anthropic Applied AI team, 2025‑09‑29).

Agent Harness Context Management: Chat History or Work Set?

Claude Model Context Engineering Updates.

LLM Memory Survey: Repositioning Three Architectural Axes.