How Context Engineering Powers Dynamic Business Data Assembly for LLM Agents

1. Six Core Context‑Engineering Practices Used by Leading Vendors

Production agents often crash or hallucinate because they lack a stable, clean "memory" environment. The following six engineering guidelines replace ad‑hoc prompting.

XML tag physical isolation : Wrap system instructions in <system_instruction> and external background data in <context_data> to prevent prompt‑injection and keep the model’s attention focused.

Hierarchical ordering : Place large knowledge‑base fragments at the top of the prompt and put the immediate action or core constraint at the very end, leveraging a strong recency effect to avoid the "Lost in the Middle" forgetting problem.

KV cache architecture and cost control : Store static company policies and API schema in a KV cache so that multi‑turn sessions only need to process tiny incremental updates.

Dynamic vector reranking : After RAG retrieval, run a lightweight cross‑encoder to score the retrieved snippets and keep only the top‑3 most relevant pieces for the LLM.

Memory‑state hierarchical async compression : When token usage exceeds a threshold, asynchronously invoke a short model to compress the long chat history into a concise <session_state_summary> segment.

Minimal viable few‑shot examples : Embed three real, perfect input‑output examples in the system context instead of long rule‑heavy prompts to improve execution robustness.

2. Practical Case: Refactoring an E‑commerce Refund‑Handling Agent

A naive prompt‑only solution supplies a static prompt such as "You are a polite Taobao customer service..." and frequently hallucinates because it lacks real‑time order status, logistics, and policy data. Applying the engineering guidelines, middleware builds a layered context before each LLM call. The pseudo‑XML below shows the five layers.

<!-- Layer 1: Static role and tool schema (cached in KV) -->
<tools_schema>
    [...fetch ERP and logistics API definitions...]
</tools_schema>

<!-- Layer 2: RAG‑driven dynamic business context -->
<dynamic_business_context>
    <order_status>Delivered_35_Days_Ago</order_status>
    <user_vip_tier>Silver</user_vip_tier>
    <faq_retrieved>
        [High‑score clause: Silver users cannot refund after 30 days]
    </faq_retrieved>
</dynamic_business_context>

<!-- Layer 3: Compressed session summary -->
<session_state_summary>
    User moved houses, missed try‑on period, second refund request was rejected, user is angry.
</session_state_summary>

<!-- Layer 4: Few‑shot examples -->
<few_shot_examples>
    ...
</few_shot_examples>

<!-- Layer 5: Current user query and execution rule -->
<user_query>
    我说你们怎么还不给我退款，我连衣服快递拆都没拆开过，快通过了！
</user_query>

<core_execution_rule>
    Examine <dynamic_business_context> for logistics and policy. Do NOT approve the refund. Return a calming refusal in JSON‑tool format.
</core_execution_rule>

3. Deep Dive: Context Engineering as a State Machine

In multi‑turn loops the system must not simply append raw dialogue. It must overwrite, compress, and cache information precisely, turning the agent into a strict state machine.

Dynamic fact overwrite : When the user asks a follow‑up, the previous <dynamic_business_context> is cleared and replaced with newly retrieved policy clauses, ensuring stale facts cannot mislead the model.

History compression layer : The original 1500‑token first‑round exchange is summarized asynchronously into a one‑sentence <session_state_summary>, preserving only the decision‑relevant gist.

Constant cache base : A 20 KB static policy document remains in KV cache, allowing the inference engine to fetch it in a few milliseconds regardless of conversation length, supporting high‑concurrency customer‑service workloads.

By repeatedly applying "filter‑clean → overwrite state → heavy compression", the LLM operates on a concise, up‑to‑date input slice, eliminating hallucination, reducing latency, and enabling reliable, scalable agent behavior.