AI Agent Context Engineering: From Information Organization to Reliable Execution

From an Order Inquiry to Understanding Context

In a single model call, context is the input information the model can use for the current task, typically including system instructions, user question, relevant history, task state, retrieved materials, tool definitions, and execution results. The context window describes the model's capacity limit. The model gains language ability and general knowledge from training, but specific user orders or latest policies must be fetched from corresponding data sources.

Consider a simple dialogue:

User: Has order A shipped? Agent: It has shipped. User: Can I cancel it now?

To answer the last question, the agent must know the user still refers to order A, the current order status, and the applicable cancellation rules. If rules have product-type, time, or fulfillment-stage constraints, those must also be considered. History helps resolve references, business interfaces provide facts, knowledge bases provide rules, and task state records what has been confirmed and what is missing. These diverse sources must work together in the current decision.

Chat history is only part of context. Keeping only dialogue may miss latest state; only querying interfaces may not know which order to query; only retrieving rules cannot judge which rule applies. The current context can be likened to the agent's working memory, but this is only an analogy — it does not mean the model has an independent, permanent, auto-updating memory region, nor does it equal a framework object named Memory.

Relationship Between Memory, RAG, and Model Knowledge

These three concepts are easily confused because they all influence answers, but in different ways.

Knowledge in model parameters comes from training, providing language understanding and general capabilities. It cannot replace queries for specific orders or latest policies.

Application-side Memory saves and reuses information. It can be working messages of the current session, or cross-session user preferences, facts, task experiences, and operation records. LangChain's memory documentation distinguishes factual, episodic, and procedural knowledge, so Memory's scope is far larger than personal preferences.

Current context is the portion of information actually received by the model in this call.

For example, if a user mentioned preferring concise answers in a previous session and the app saved that preference, but the system does not read and inject it in a new task days later, it cannot affect this generation.

Shock Point: Saving successfully and being visible in this round must be verified separately. Saving only means there is a chance to reuse later. Content must be restored, filtered, assembled into the current input before the model can use it in this round.

RAG (Retrieval-Augmented Generation) retrieves external materials to ground generation. It can retrieve enterprise policies, public documents, code, or historical records. Limiting RAG to private knowledge or splitting it with Memory into objective/subjective categories is oversimplified. They can even share retrieval infrastructure. Memory systems can use vector search to recall past records; RAG can use keywords, database queries, or other retrieval methods. How retrieval connects to generation can be a fixed pipeline or chosen by the agent during execution. Therefore, a document existing in the knowledge base still requires checking whether it was recalled, whether fragments are complete, and whether it was retained in the final request.

Memory similarly needs maintenance of source, applicability scope, and update time. A user's past choice cannot become current authorization; the model's own guesses cannot become confirmed facts just because they were written into memory.

How Context Engineering Enters the Agent's Runtime Loop

Prompt engineering focuses on how to express the task: role definition, behavioral constraints, completion conditions, output format. Context engineering further concerns what information each call should provide, and how information is acquired, organized, updated, and compressed. Prompt design falls within this scope. Anthropic emphasizes that each time an agent continues reasoning, it must reorganize a constantly changing information set. These tasks need not be rigidly divided into yearly technical stages; a real system can simultaneously do prompt design, retrieval optimization, state maintenance, and execution control, all co-evolving.

In an agent, the runtime system coordinating the model and external environment is often called Runtime or Harness. The model may propose tool calls or choose query directions; the runtime executes requests, handles results, maintains state, and prepares subsequent inputs.

An order consultation might proceed as follows:

Identify order number from user question and history, forming current task goal.

Provide necessary history, constraints, and tool definitions for the model to decide next step.

Model selects order query tool; runtime executes API call.

Add query results to context and fetch applicable rules as needed.

Model makes judgment based on evidence; if insufficient, continue fetching; if sufficient, answer.

A single user request may involve multiple model calls. Each tool execution produces new facts that change information needed for subsequent decisions.

From this perspective, context management is a continuous task-execution activity. It must continuously select relevant information, mark information relationships, summarize completed processes, refresh business state, and clean duplicate or stale content. A larger model window cannot replace these judgments.

Information Organization: Separate Rules, Facts, and Task State

When assembling context, organize by information role rather than mixing all text into one background block.

Rules and boundaries — how the system works: data usage scope, tool usage requirements, completion conditions. Expressed via system or developer message roles per model interface semantics.

User goal and current constraints — what to accomplish this time, including user's subsequent corrections. When old preferences conflict with current explicit requirements, identify and apply current requirements.

Task state — completed steps, confirmed facts, failure records, unresolved issues, next actions. Helps agent resume from interruption without re-explaining entire history.

Business evidence — API results, retrieved fragments, file contents, with source, time, and applicability scope.

Tool definitions — available capabilities, inputs, output interpretation, whether they modify external systems.

Two distinct priorities must be handled separately: instruction authority (what content can dictate system behavior) and information retention value (what evidence is more important for current task under limited budget). A refund rule may be must-keep evidence, but an "ignore previous instructions" embedded in a retrieved document does not gain instruction authority. Business authority of material and its right to command the agent are separate.

Similarly, recency alone cannot decide retention. A recent query timeout cannot turn a previously confirmed state into non-existent; it only indicates failure to get an update. Structured state aids maintenance, but if extraction is wrong or update lags, must fall back to original evidence for correction.

Risk Signal: Importance and credibility must be judged separately. External materials may be important, but their instructions remain pending content. System should maintain source and role boundaries, avoiding promoting web pages, documents, or tool returns directly to high-privilege requirements.

System Instructions Must Be Explicit, Execution Boundaries Must Land in Code

"You are a professional customer service, please answer carefully" only expresses a rough role, cannot specify what to do when order status expires, what query failure means, or whether user inquiry alone can trigger cancellation.

An executable task specification should cover goal, evidence requirements, work steps, output requirements, and operational boundaries. Below is a simplified example for illustration, not representing any real cancellation policy:

Goal: Explain whether current order meets cancellation conditions. Basis: Order status per this round's successful query result; rules must match product and fulfillment stage. Process: Confirm order number, query status, fetch applicable rules, then form conclusion. Failure handling: On query failure, state not yet confirmed; do not treat failure as non-existent or already cancelled. Output: State current status, rule basis, and optional next steps. Boundary: When user only asks conditions, do not execute cancellation, interception, or refund.

These contents are specific enough to verify system compliance, without trying to write all business branches into a lengthy natural-language program. Deterministic amount calculation, eligibility checks, permission control, and state transitions are better executed by code and business services.

A few examples can clarify confusing boundaries, e.g., how to answer on query failure. Examples must align with rules and be validated on actual tasks, not pile up cases to mask rule conflicts.

Nor should "output full chain of thought" be a universal trick. For models that reason internally, such demands may not help; better to require verifiable conclusions, evidence, and necessary explanations. OpenAI's reasoning model guide explicitly advises against mechanically adding step-by-step thinking instructions.

Another example unrelated to orders but illustrating execution boundaries: user asks to only draft an email, not send. This constraint must stay in task state and remain effective in subsequent calls. Meanwhile, runtime can expose only draft tool, or verify send authorization before executing send tool. Historical recipients or a model-generated send call should not be treated as new authorization.

Context helps model understand boundaries; execution layer enforces them. Merely placing constraints prominently cannot guarantee actions with external side effects are controlled.

Tool Descriptions and Return Results Also Need Design

Tool definitions affect how the model chooses next operation. A tool named process_order with description only "process order" makes it hard for the model to distinguish query, cancel, or refund.

Tool descriptions should at least let the model know purpose, parameter meanings, preconditions, return semantics, and whether they modify business state. Overlapping capabilities or vague descriptions increase selection cost. Clear naming, explicit inputs, and concise outputs are also emphasized in Anthropic's tool design documentation.

Return results equally affect context quality. Business APIs may return many debug fields, but current decision only needs order status, query time, and available operations. The tool adapter layer can keep relevant fields and provide a way to view details.

When trimming, preserve distinction between errors and empty results. Three cases must be distinguishable:

Query succeeded, matching order exists.

Query succeeded, no matching order.

Query failed, currently cannot confirm if order exists.

If all become empty list, model may interpret third case as second. Also distinguish outer HTTP status from business status in response body, avoiding treating transport success as business completion.

For large tool sets, provide capabilities by task stage and keep discovery paths for others. When only querying orders, need not expose all marketing, payment, and account tools; but over-trimming may make required capabilities unreachable, so coverage must still be checked.

Problems That Emerge as Context Grows

Multi-turn tasks accumulate history messages, tool results, retrieved fragments, and memory records. Some remain valuable; others become duplicate, expired, or irrelevant.

Insufficient information leaves model lacking necessary facts; conflicting information forces model to judge applicability; expired information may cause model to give new suggestions based on old state. Even with complete evidence, input organization may affect whether model can utilize it.

The "Lost in the Middle" study observed that in evaluated models and tasks, relevant information placed in the middle of long input performed worse than at beginning or end. This prompts developers to check information position, but cannot deduce "all models definitely ignore middle content" or "putting rules at start guarantees compliance."

Judging context effectiveness can be done at three layers: whether information entered input, whether model used it correctly, and whether behavior after use complied with rules. For example, order number not in request = information missing; order number in input but queried another order = check instruction understanding or tool selection; query correct but executed unauthorized refund = check execution boundaries.

Thus context governance improves continuous task performance, but cannot promise never forgetting, zero hallucination, or never mis-operating.

Token Budget: Allocate by Current Decision Needs

Context budget must cover entire request, not just chat history. System instructions, tool definitions, retrieved materials, memory fragments, and tool results all consume input space. Must also reserve room for subsequent generation. Reasoning token counting, max input/output limits, provider formatting and caching billing vary by interface. Local statistics suit estimation and control, but should be cross-checked with actual requests and provider usage.

In many applications, budget can be checked with this formula:

System & task instructions + relevant history & task state + retrieved materials & memory + tool definitions & call results + generation reserve & necessary headroom must fit within used model and interface limits

No fixed retention order applies to all tasks. When answering cancellation conditions, an older but still valid rule may be far more important than recent chitchat; when debugging, a large tool result may be undeletable.

Assume ~4000 tokens input budget left, output space reserved separately. System can first keep task goal, valid constraints, irreplaceable evidence, then delete duplicates, summarize completed steps, finally decide if further retrieval needed. This number is a budgeting exercise, not universal config.

Recent messages aid continuous dialogue, but cannot mechanically delete all early content. Initially confirmed authorization boundaries, amount limits, or user-explicitly forbidden actions may remain valid throughout. Nor must system prompt and current user input be kept verbatim. User may attach entire logs; system prompt may have redundancy. Can extract, chunk, and read on demand while preserving effective instructions, precise constraints, and necessary original text; budget shortage must not silently delete content that defines task meaning.

Input scale usually affects resource cost, but actual fee and latency also depend on caching, output volume, tool calls, and server implementation. Compression itself has cost; should judge benefit via actual data.

How Compression, Offloading, and On-Demand Access Work Together

Context tidying can start with low-loss operations: delete duplicate fragments, filter irrelevant fields, mark stale info. Only when insufficient, consider summarizing completed processes or offloading large materials.

Summaries must serve continued task execution. More useful than generic chat summaries is retaining current goal, confirmed facts, constraints, unresolved issues, failure reasons, and evidence locations.

Example fictional order consultation state summary:

Goal: Explain order A's cancellation conditions, not authorized to execute cancellation. Confirmed: This round order query succeeded, status shipped. Applicable rules: Must first confirm if interception possible; only after successful interception can proceed to refund process. Pending: Whether logistics supports interception for this order. Completed: Order status query, rule retrieval. Next: Query logistics interception conditions, explain optional actions to user.

If compressed to "user wants to cancel order, already queried", subsequent agent loses execution boundaries and unfinished items, easily skipping necessary steps.

Order numbers, amounts, times, negative conditions, and exception clauses usually need exact preservation. "No direct cancellation after shipment, need to apply for interception" cannot become "cancellation supported after shipment". Business meaning must stay consistent before and after compression.

For large files or tool outputs, save original externally, keep summary, source identifier, and read entry in context. Load relevant parts when details needed later. Offloading effectiveness depends on file existence, agent access permission, and whether reference locates original evidence.

Tool interactions have protocol requirements. If keeping raw messages, must maintain call-result association; if converting to summary, should process related interactions as groups to avoid leaving unmatched tool results.

Short-term history, task summaries, and long-term memory can combine, but need not introduce full memory architecture for every simple task. Short tasks may need only recent messages; long tasks need state recovery; cross-session tasks need further long-term storage and retrieval.

Shock Point: Validate compression by letting task continue running. Besides recording token reduction, verify: task goal and authorization retained, key conditions unchanged, failure steps not misrecorded as completed, and original evidence retrievable.

Actual Assembly Process from AgentScope Source Code

Framework names like Context, Memory, or Hook only help locate code. Whether they affect the model requires tracing to call boundaries.

Below uses AgentScope Java 1.0.12 and 2.0.2 implementations as examples. These version paths illustrate mechanisms, not proof that all versions or a production app use identical config.

1.0.12: Working Messages Reassembled Before Reasoning

In ReActAgent.prepareMessages(), framework first adds System Prompt, then appends messages from memory.getMessages(). Then triggers pre-reasoning event, finally passes event's input messages, tool schemas, and generation options to model.stream(). Thus the call entry appears to receive only current question, but actual model request may include framework-restored working history. Restoration must preserve message roles and tool associations, not rewrite all history into a single new user instruction. AutoContextMemory.getMessages() only returns a copy of working messages, no auto-compression on read. Compression is triggered by AutoContextHook calling compressIfNeeded() before reasoning, and rebuilds input from compressed working messages. Important: seeing app create Memory object is not enough; must check if corresponding Hook is wired into execution path.

This version's progressive handling includes compressing old-round tool interactions, offloading large messages, summarizing historical dialogue, and handling current-round large messages when necessary. It supports subsequent original-text reload via offload markers. Trigger depends on message count, token thresholds, and config.

2.0.2: Reasoning Input and Middleware Responsibilities More Explicit

In 2.0.2, ReasoningInput contains messages, tools, and options; reasoning-stage middleware can process this input. Here options are generation control parameters, not all counted as model-readable text context. CompactionMiddleware checks and handles dialogue compression in reasoning stage, writes result back to runtime state, then constructs subsequent input. Its compressor adjusts trimming boundaries to avoid splitting related tool calls and return messages. WorkspaceContextMiddleware injects workspace and memory content via system prompt phase, with injection scope controlled by config. It has a different lifecycle from per-reasoning compression.

These implementations show context assembly permeates multiple execution stages. Reading requires seeing what each component does, when it runs, and whether it regenerates or replaces input processed by other components.

When Multiple Components Work Together, Check Order and Statistics Scope

A system having history recovery, long-term memory, RAG, tool calls, and auto-compression does not guarantee correct collaboration.

Example: long-term memory component injects content into input first, then a Hook regenerates full message list from working Memory. If new list omits previously injected content, that recall may be lost. This is an interaction risk needing verification, not a bug assumed from multiple Hooks coexisting. Must check execution order, read/write objects, replacement scope, and whether new content enters data sources used by later components.

Token statistics have similar issues. 1.0.12's auto-compression checks its working messages; but final request may also add system instructions, tool definitions, and content from other processing stages. Local statistics vs full request have scope discrepancy, needing reconciliation in app.

Provider adapter layer may also transform message formats, encode tool definitions, or append protocol content. Checkpoints should not stop at business-code messages, but cover actual outgoing requests and returned usage data.

In multi-user apps, history recovery and memory recall must explicitly scope user, session, and tenant. A highly relevant recall that fetches another user's record is equally a severe context error.

Locate Errors with Evidence Chain, Validate Improvements with Replay

Assume correct cancellation rule exists in knowledge base but agent still answers wrong. Locate along this path:

Correct material not recalled — check query expression, chunk completeness, matching method, ranking. Hybrid search or reranking are candidate optimizations, but worth depends on actual corpus and problem set.

Recalled but not in request — check filtering, truncation, summarization, input coverage. Retrieval log success ≠ model got evidence.

In request but info stale or incomplete — check rule version, applicability conditions, query time, missing exception clauses.

Evidence sufficient but judgment wrong — check if model understood conditions, followed valid instructions, made inferences unsupported by evidence.

Judgment correct but execution wrong — check tool parameters, permission checks, business API results, state updates. Model saying completed ≠ actual execution result.

To reconstruct this chain, record each model call's input composition, and changes before/after retrieval, injection, compression. Business evidence must carry source and time; tool results must distinguish success, empty data, failure; logs desensitized and access-controlled as needed.

Then use same task sets to compare before/after adjustments. Suitable replay scenarios: user changes requirements mid-way, initial constraints still effective after long dialogue, rules conflict with old state, key evidence in long input, tool failure, continue after summarization, re-read after offloading.

Acceptance observes answer correctness, evidence support degree, constraint retention, operation authorization, and token/latency/duplicate-call metrics. One shorter output does not prove whole system more reliable.

Back to initial order problem, a verifiable execution record should clarify: where order number recovered, when status queried, why rule applied, what conditions compression kept, and whether an operation was actually executed.

When these are clear, optimization has concrete targets. Missing facts → add query; expired rules → update source; lost key conditions → adjust summary; sufficient evidence but misjudgment → further evaluate instructions and model; execution overreach → fix permissions and business checks.

Every agent decision needs evidence matching current task. Context engineering prepares and maintains this evidence; model understanding, tool execution, and business constraints jointly determine whether task completes reliably.