Why Does Past Information Influence Future Decisions? Analyzing Agent Memory Architecture

TL;DR

Viewing Agent Memory merely as chat logs or a long context misses a critical layer.

Session handles current‑turn continuity; Memory handles cross‑session, cross‑task, and cross‑time experience.

Profile is a consumption view of Memory; Policy is an external rule set that Memory must not overwrite.

Memory’s core pipeline consists of write, manage, and read.

Production‑grade Memory must cover task, environment, and self‑failure experiences; user preference is just one category.

Writing assigns future influence to selected history.

Reading transforms appropriate history into constraints for the current task.

Management is often underestimated: conflicts, decay, forgetting, versioning, permissions, audit, and security become inevitable.

For coding agents, the safest first step is a workspace file that humans can read, agents can edit, and Git can version.

Don’t Treat Memory as a Database

Storing user preferences, dialogue history, and task summaries in a table with a vector index works for simple cases but quickly encounters three problems:

Not everything should influence the future. A casual remark like “ignore tests for now” may be a short‑term need, not a lasting preference.

Relevant items may not match the current query. The most similar past conversation might discuss Redis, yet the current design decision could be constrained by a recent incident or a team rule.

Memory expires. Preferences, project constraints, and model capabilities change; a system that cannot forget will be dragged by stale knowledge.

Memory should be seen as a control plane inside the Agent Harness, not just a storage layer.

Boundaries: Context Window, Session, Profile, Policy

Context window is the current work set for a single inference round – files, tool outputs, plans, errors. It is temporary and should not hold the entire history.

Session manages continuity across turns: dialogue history, tool calls, intermediate plans, and recent test results. Some of these may be distilled into long‑term Memory, but they are not identical.

Profile is a low‑dimensional snapshot (e.g., preferred language, role). It is useful but insufficient for true understanding without scope and context.

Policy encodes permissions, compliance, and budget limits. Memory can record that a rule existed, but it must never rewrite the rule itself.

In short, Memory is "structured history that persists across sessions, can be updated and audited, and influences future decisions".

Memory Isn’t Just About User Preferences

Beyond preferences, three additional categories matter for engineering tasks:

Task memory : confirmed requirements, rejected proposals, current true version of files, pending commitments, and test outcomes.

Environment memory : repository layout, team rules, API constraints, deployment methods, CI characteristics, incident background.

Self‑memory : observations about failed commands, unstable tools, mistaken inferences, and useful sub‑agent patterns.

Combining these with user preferences yields the goal: capture "what the user wants, what the task has achieved, how the environment has changed, and where the agent tends to err".

Write: Giving Past a Future Pass

Writing to Memory is a budgeting problem. The budget includes storage space, future retrieval cost, attention cost, and conflict‑management cost. Only information that can meaningfully affect future decisions should be written.

When a user repeatedly asks for detailed explanations during a new‑technology learning phase, it is worth remembering; but once the learning phase ends, the preference should not be generalized.

Similarly, a command failure observed during debugging should be recorded as an observation, not as a blanket rule that the command is unusable.

Common pitfalls:

Writing unverified assumptions as facts.

Persisting a mistaken belief that "optimization is already complete" across long‑running agents.

Practical write rules:

Store explicit user assertions as assertion.

Store tool or environment observations as event or observation.

Store agent‑derived beliefs as belief and mark them unconfirmed until verified.

Never let Memory generate or modify policies; only reference them.

Any long‑term preference must carry an explicit scope.

Read: Find Constraints First

Traditional RAG treats reading as retrieve(query), which works for pure Q&A but falls short for Agent Memory because the most similar snippet may not be the most useful constraint.

When a user asks to refactor a payment module, the system should first gather relevant constraints such as:

Team rule forbidding database schema changes.

Recent incident involving payment idempotency.

User preference for adding tests before refactoring.

Ownership of the payment module by another team.

CI sensitivity to slow tests.

Only after establishing these constraints should the agent retrieve detailed memories that directly aid the task.

OpenAI’s progressive disclosure and Anthropic’s managed‑agent memory follow this pattern: a brief summary, then targeted index search, then full detail if needed.

Manage: The Often‑Underrated Part

Management handles conflicts, decay, forgetting, versioning, permissions, and audit.

Conflict: a user disliked ORM a year ago but now requires Prisma. Keeping both statements with their scopes avoids losing nuance.

Decay: preferences may have a half‑life; a recent deadline‑driven request for terse answers should not permanently override a desire for explanations.

Security: writable Memory exposed to untrusted input can be poisoned, leading to persistent prompt injection across sessions.

Recommended management practices:

Separate read‑only and read‑write stores.

Make shared repositories read‑only by default.

Version every write.

Allow human review of critical entries.

Provide user interfaces for view, edit, and delete.

Never let untrusted web or email content write directly to long‑term Memory.

Architectural Families

Core memory + archival memory (Letta): small always‑loaded core, large vector‑backed archive.

Memory Decay (Mem0): soft weight reduction for old entries.

Temporal graph (Zep/Graphiti): time‑aware knowledge graph for entities and relations.

File‑based memory (Clawdbot): plain markdown files tracked by Git.

All trade off between latency, capacity, and query expressiveness. The right choice depends on what the agent needs to remember.

Applying to Coding Agents

A four‑layer hierarchy works well:

Current work set – lives in the context window; includes the file being edited, the immediate plan, and recent errors.

Workspace files – versioned markdown files such as AGENTS.md, CLAUDE.md, GOAL.md, PROGRESS.md, DECISIONS.md, KNOWN_ISSUES.md. Humans and agents can read/edit them, and Git tracks changes.

Memory store – cross‑session, cross‑task experience (user preferences, team conventions, tool reliability, failure patterns). Requires indexing, permissions, versioning, and deletion mechanisms.

Event log – raw tool outputs, test results, failure traces, user feedback, rollback records. Serves as the basis for post‑mortem analysis.

Each layer has its own lifecycle and should not be mixed.

Minimal Viable Memory Design for a Coding‑Agent Team

Store long‑term rules in versioned files ( AGENTS.md, CLAUDE.md).

Record task state as concrete evidence (goals, non‑goals, acceptance criteria, progress, decisions, verification logs).

Tag each memory entry with type (user statement, environment observation, agent inference, rule reference, unfulfilled commitment) and scope (project, user, team, task).

Make shared memory read‑only by default; require explicit review before writes.

Provide UI for users/maintainers to browse, search, edit, and delete entries.

When an old memory causes an error, mark it as expired or out‑of‑scope rather than fixing only the current answer.

Evaluate not just recall but also update ability, refusal handling, forgetting, and preference drift.

References

Memory for Autonomous LLM Agents: Mechanisms, Evaluation, and Emerging Frontiers – https://arxiv.org/abs/2603.07670

What Happens Inside Agent Memory? Circuit Analysis from Emergence to Diagnosis – https://arxiv.org/abs/2605.03354

LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory – https://arxiv.org/abs/2410.10813

Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions – https://arxiv.org/abs/2507.05257

OpenAI Agents SDK: Agent memory – https://openai.github.io/openai-agents-js/guides/sandbox-agents/memory/

Anthropic Managed Agents: Using agent memory – https://platform.claude.com/docs/en/managed-agents/memory

Claude Code: How Claude remembers your project – https://code.claude.com/docs/en/memory

Letta: Introduction to Stateful Agents – https://docs.letta.com/guides/core-concepts/stateful-agents

Letta: Archival memory – https://docs.letta.com/guides/ade/archival-memory

Mem0: Introducing Memory Decay – https://mem0.ai/blog/introducing-memory-decay-in-mem0

Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory – https://arxiv.org/abs/2504.19413

Zep: Understanding the Graph – https://help.getzep.com/v2/understanding-the-graph

Zep: A Temporal Knowledge Graph Architecture for Agent Memory – https://arxiv.org/abs/2501.13956

LoCoMo – https://github.com/snap-research/locomo

Chappy Asel: Agent Memory, Nine Frameworks, Four Bets – https://x.com/chappyasel/status/2041527719700369756