How to Stop Large‑Model Coding Agents from Forgetting in Multi‑Turn Dialogues

Why Agents Appear to Forget

When a coding agent works for hours, the UI may show a full scrollable chat log, but the model can suddenly lose earlier conclusions. This is not model degradation; the history visible to the user is not the same as the context actually sent to the model each turn.

Three‑Layer Memory System

Interaction History – what the user sees; contains user messages, assistant replies, tool calls and results. It is not always sent directly to the model.

Request Projection – the exact slice of messages the model receives this turn; it consists of filtered, folded, externalized or summarized messages and is always sent to the model.

Logical Execution Chain – determines where the next turn should continue; includes activity branches, compression boundaries, parent/child relationships and runtime state, and influences the request indirectly.

Each layer serves a different audience and can diverge dramatically: a large log may remain visible in the UI while being omitted from the request projection, or a compressed segment may still exist on disk but be excluded from the logical chain.

Token Budget and Prompt Caching

The effective budget for history is not the raw window size W but B_{history}=W-R_{output}-R_{system}-R_{tools}-R_{safety}, where output reservation, system prompts, tool definitions and safety margin are subtracted. For coding agents, search results, build logs, long files and batch tool outputs are the biggest consumers.

Simply deleting old content reduces tokens but harms prompt‑caching, which reuses identical prefixes across turns. Changing a character early in the history can invalidate the cached computation, creating a tension between token reduction and cache stability.

Compression Is a Set of Projection Strategies

Large‑Result Externalization : keep the full result in external storage, retain only a preview, size, type and a reference in the request projection. The original stays on disk; the preview is stable to avoid breaking prefix cache.

Fine‑Grained Reclamation : discard cheap noise such as expired tool results. When the cache is cold, rewrite the local projection; when hot, preserve the prefix and let the backend trim old blocks.

Interval Clipping & Folding : remove or summarize low‑value middle history without deleting the original records; add a “projection commit” that tells the next request how to interpret past data.

Full Compression : when the window is near its hard limit, replace all history before a boundary with a concise “working‑state summary” that records goals, facts, decisions, pending items and constraints. The original messages stay on disk, but the logical chain starts from the summary.

Failure‑After Compression : if the server rejects a request for exceeding the context limit, compress once and retry, with a strict single‑retry limit to avoid infinite compression loops.

Protocol Integrity Beats Token Optimality

Context cannot be split arbitrarily. A tool call and its result form a protocol‑level unit; breaking them violates interface contracts. Therefore, boundary selection must first find a candidate that fits the budget, then expand outward to the nearest legal structural boundary, even if it slightly exceeds the nominal token limit.

Protocol integrity is higher priority than local token optimality; determinism outweighs one‑off compression ratio.

Event‑Log Model for Session Records

Instead of mutating a JSON array, store an append‑only event log where each event describes an immutable action. Example events:

{
  "type": "message_appended",
  "id": "m42",
  "parent": "m41",
  "role": "assistant"
}
{
  "type": "tool_result_externalized",
  "message": "m42",
  "artifact": "logs/run-17.txt"
}
{
  "type": "projection_committed",
  "hidden": ["m18", "m19", "m20"]
}
{
  "type": "compaction_boundary_created",
  "id": "c3",
  "parent": null,
  "summary": "..."
}
{
  "type": "runtime_state_checkpointed",
  "recent_files": ["src/index.ts"]
}

These events form a directed graph via parent pointers; projection rules decide which nodes enter the current request, and compression boundaries cut the default back‑track path. Runtime checkpoints store state beyond the chat text.

Resume vs. Fork

Resume : keep the original session ID, continue appending to the same log, inherit externalized results and projection rules, and aim to return to the original work context.

Fork : create a new session ID, start a new log, copy only the minimal state needed to interpret past history, and launch a new timeline.

Three Reusable System Principles

Prefix Decisions Must Be Repeatable : identical persisted records under the same configuration should produce byte‑stable request prefixes.

Structural Integrity Over Extreme Compression : never split tool calls or intra‑message relationships; budget control yields to protocol legality.

Disk Is Append‑Only, Logical Views Are Re‑creatable : physical records describe what happened; projection events describe how to view the past. Deletions and compressions are expressed as new events, preserving auditability and deterministic replay.

Conclusion: Controlled Reconstruction

Agent memory is not a simple short‑term buffer plus a long‑term store. It is a multi‑layer system where interaction history explains the past to humans, request projection fits the limited token window, and the logical execution chain decides the next action. True “remembering” means reliably reconstructing the necessary state under resource constraints, and “forgetting” often stems from mismatched layers rather than model failure.