Unlocking AI Memory: A Comprehensive Survey of Theory, Architectures, and Future Trends

Survey Overview

Abstract: AI Memory is a key component for achieving general artificial intelligence (AGI) and for enabling agents to accumulate experience from past interactions, thereby improving subsequent decision‑making.

Scope: The survey covers theoretical foundations, a 4W taxonomy for classifying memory mechanisms, single‑agent and multi‑agent memory architectures, evaluation benchmarks, and emerging research directions.

4W Taxonomy: Memory is categorized along four orthogonal dimensions – When (lifecycle), What (type of information), How (storage format), and Which (modality).

Evolution: The field is moving from isolated single‑agent memory toward collaborative multi‑agent memory systems.

GitHub repository: https://github.com/BAI-LAB/Survey-on-AI-Memory
Project page: https://baijia.online/homepage/memory_survey.html

Theoretical Foundations

The survey links cognitive science, psychology, and neuroscience to AI memory. Classic models such as the Atkinson‑Shiffrin model and working‑memory frameworks are used to delineate the boundaries between Memory , Knowledge , Context , and Experience in intelligent agents.

LLM Memory: Implicit knowledge stored in model weights that drives prediction.

Agent Memory: Explicit data structures that support perception‑planning‑action loops for complex tasks.

AI Memory: A high‑level, lifelong, adaptive memory system that persists across sessions and modalities.

4W Memory Taxonomy

The taxonomy answers four orthogonal questions:

When (Lifecycle): Duration of storage, ranging from transient buffers to persistent cross‑session stores.

What (Type): Content categories – procedural skills, factual statements, meta‑cognitive reflections, and social models.

How (Storage): Representation formats – implicit weight‑based storage, explicit text, vector embeddings, or graph structures.

Which (Modality): Supported modalities – pure text, or multimodal combinations of image, audio, and video.

Single‑Agent Memory Architectures

Four major architectural paradigms are surveyed:

Hierarchical: Memory organized in layers from short‑term buffers to long‑term stores.

OS‑style: Memory treated as a file‑system‑like resource with explicit read/write APIs.

Cognitive‑evolutionary: Mechanisms that support self‑evolution, consolidation, and forgetting.

Graph/Temporal: Memory represented as knowledge graphs or time‑series structures for relational reasoning.

Core operations include Storage , Retrieval , and Updating (incremental, corrective, consolidation, and forgetting). Advanced capabilities extend beyond passive recall to self‑evolution and multimodal association.

Multi‑Agent Collaboration and Shared Memory

Memory sharing in multi‑agent systems aims to break isolated memory islands and enable collective intelligence. Two dimensions are highlighted:

Communication mechanisms:

Explicit – natural‑language messages or structured schemas that ensure interpretability.

Implicit – shared environment observations or internal state vectors for low‑latency coordination.

Memory‑sharing granularity:

Task‑level – aggregation of experiences across agents to evolve long‑term capabilities.

Step‑level – fine‑grained routing of contextual information during collaborative workflows.

Evaluation Dimensions

Four categories are proposed to assess memory effectiveness:

Memory Retrieval Capability: Accuracy of locating required information.

Dynamic Updating Capability: Ability to incorporate new data without catastrophic forgetting.

Advanced Cognitive Capability: Support for reasoning and planning based on stored memories.

System Efficiency: Resource consumption and latency of memory operations.

Practical Applications

Memory mechanisms empower two broad scenario categories:

Single‑agent applications: Continuous context retention, user‑preference personalization, and long‑term skill accumulation beyond fixed context windows.

Multi‑agent applications: Shared memory spaces that enable collaborative task execution, provenance tracking, and collective reasoning while addressing concurrency and access‑control challenges.

Future Outlook

Key challenges and research directions:

Architectural conflicts: Limited context windows of large language models, risk of catastrophic forgetting, and bottlenecks in multimodal integration.

Theoretical gaps: Incomplete dimensional understanding of memory, lack of standardized metrics for generalization and robustness, and immature frameworks for multi‑agent memory sharing.

Safety and operational complexity: Privacy leakage, inference risks, and static permission models unsuitable for dynamic collaborative environments.

Brain‑inspired modeling: Leveraging Complementary Learning Systems to balance stability and plasticity, and to create unified multimodal representations.

Memory‑to‑experience upgrade: Transforming unstructured logs into temporally and causally linked memories, enabling goal‑directed retrieval and closed‑loop self‑evolution.

Self‑evolving collective memory: Dynamic, context‑aware permission control, privacy protection, automatic deduplication, clustering, and consensus protocols for robust multi‑agent collaboration.

Advances in AI memory are expected to turn static large language models into lifelong, adaptive agents, thereby moving the field closer to AGI.