Unlocking AI Agent Memory: A Deep Dive into Forms, Functions, and Dynamics

Overview

The survey Memory in the Age of AI Agents: A Survey (arXiv:2512.13564) introduces a unified taxonomy for agent memory that spans three orthogonal dimensions:

Forms : where memory is stored.

Functions : what the memory is used for.

Dynamics : how memory is created, updated, and accessed.

Mathematical Formalization

An autonomous agent maintains a mutable memory state M_t. The paper defines three core operators that govern the memory lifecycle:

Formation : a selective function that converts the current experience (e.g., reasoning steps, tool outputs) into candidate memory entries.

Evolution : a set of operations that merge, de‑duplicate, resolve conflicts, and optionally forget entries, keeping the store coherent.

Retrieval : given a new task and observation, a retrieval function extracts the most relevant slice of M_t for the LLM to consume.

Forms of Memory (Where Memory Lives)

Token‑Level Memory

Memory stored as natural‑language text or discrete symbols in an external database. It is transparent, editable, and can be organized in different topologies:

Flat : chronological log, suitable for simple dialogues.

Planar / 2D : graph‑like structures (knowledge graphs) that enable associative reasoning.

Hierarchical / 3D : pyramid‑style abstraction layers (e.g., MemGPT) that support long‑term management.

Parametric Memory

Memory embedded directly in model weights. It is injected via fine‑tuning or model‑editing, making the knowledge instantly accessible without retrieval latency.

Pros : zero‑latency access.

Cons : high update cost, risk of catastrophic forgetting, and lack of interpretability.

Latent Memory

Intermediate representations stored as high‑dimensional vectors (embeddings) or KV‑cache entries. These are opaque to humans but enable fast similarity search and serve as a bridge between token‑level and parametric stores.

Features : compact, flexible, and well‑suited for multimodal tasks where, for example, an image is stored as a single embedding.

Functions of Memory (What Memory Is Used For)

Factual Memory

Ensures consistency by remembering user‑specific facts (name, preferences) and world facts (e.g., door status). This reduces hallucinations and keeps conversations coherent.

Experiential Memory

Enables agents to learn from past successes and failures. It is organized into three sub‑categories:

Case‑based : direct reuse of previous solutions.

Strategy‑based : abstract SOPs derived from multiple cases.

Skill‑based : translation of experience into executable code or API calls.

Working Memory

A limited cache that holds the current reasoning context. It dynamically compresses inputs, folds completed steps into summaries, and frees space for new subtasks.

Dynamics of Memory (How Memory Operates)

Formation

Semantic Summarization : compress long dialogues into concise abstracts.

Knowledge Distillation : extract explicit rules such as "user likes apples".

Structured Construction : organize information into knowledge graphs.

Evolution

Consolidation : merge short‑term fragments into long‑term memory.

Update : correct erroneous entries, often via conflict resolution in RAG pipelines.

Forgetting : prevent memory bloat through three strategies:

Time‑based forgetting – older items decay.

Value‑based forgetting – discard irrelevant chatter.

Frequency‑based forgetting – archive seldom‑used knowledge.

Retrieval

Timing : decide whether to query memory after every utterance or only when uncertainty arises (current trend: let the agent decide autonomously).

Strategy : move beyond pure keyword matching to hybrid approaches that combine keywords, vector similarity, and graph relationships.

Future Outlook

From Retrieval to Generation : future agents may generate memory fragments on‑the‑fly, mimicking human reconstructive recall.

Reinforcement Learning for Memory Management : heuristic rules for storing, updating, and forgetting will be replaced by RL policies that let agents learn optimal memory strategies.

Multimodal Memory : agents will store not only text but also images, audio, and other sensory data as embeddings.

Trustworthy Memory : as memory contains personal data, mechanisms for security, explainability, and user‑controlled editing become essential.

Conclusion

The survey provides a comprehensive technical framework for building dynamic, self‑evolving AI agents. It equips developers with concrete design choices—from token‑level stores to parametric updates—and highlights research directions that move beyond static retrieval‑augmented generation toward truly continuous cognition.