Where Does an Agent’s Long‑Term Memory Live? Update and Deletion Strategies Explained

1. Why Agents Need Memory

Using a corporate banking smart assistant as an example, the same query is asked repeatedly, leading to extra 3‑4 dialogue rounds without memory. Experiments show that adding a memory module lets the system recognize returning users, answer based on historical preferences, increasing satisfaction by 23% and reducing dialogue rounds by 2.1.

Memory is not a luxury but a core capability, encompassing what to store, how to store, how to retrieve, how to update, and how to delete.

2. Cognitive‑Science View: Three Memory Types

Human memory is divided into semantic, episodic and procedural memory; an agent can mirror this division, each with distinct storage and retrieval strategies.

Semantic Memory (general knowledge) : shared vector DB, low update frequency, pure semantic search. Example content: product terms, policies.

Episodic Memory (user‑specific) : per‑user vector DB, high update frequency, filter by user ID then semantic match. Example content: user preferences, consultation history.

Procedural Memory (how‑to) : stored in system prompts, medium update frequency, directly injected without retrieval. Example content: business flow, priority rules.

3. Dual‑Layer Architecture

Short‑Term Memory

Implemented as a sliding window that keeps the most recent N dialogue turns in the LLM context; it disappears after the session ends.

Long‑Term Memory

Persisted in a vector database (e.g., Milvus) with partitions based on memory type and user ID. Fields include memory ID, user ID, type, content, vector, timestamp, TTL and soft‑delete flag.

This design solves isolation (no data cross‑talk) and performance (user‑ID filter before vector search).

4. Memory Management Closed Loop

Step 1 Define – What to store?

Only high‑value information such as explicit user preferences, enterprise basics, key decisions and events; ordinary chit‑chat is discarded.

Step 2 Write – How to store?

After a conversation ends, an LLM extracts memory points and writes them to the appropriate store: episodic → user‑specific vector DB, semantic → shared knowledge base, procedural → system prompt.

Step 3 Manage – How to update or delete?

Use the open‑source framework Mem0 with four operations:

ADD : insert new information.

UPDATE : overwrite an existing entity when similarity exceeds a threshold (e.g., 0.85).

DELETE : remove expired or user‑requested data.

NOOP : no action if the information already exists.

Before writing, the LLM computes semantic similarity; if the score is above the threshold, the record is treated as the same entity and UPDATE is performed, preventing contradictory data.

Step 4 Read – How to use?

At the start of each dialogue:

Filter by user ID.

Perform vector similarity search and take the top‑5 most relevant memories.

Format them and inject into the system prompt.

Append the short‑term window and feed the combined context to the LLM.

5. Conflict, Expiration and Compliance

Conflict handling : compute similarity between new and existing memories; if above 0.85, treat as an update.

Expiration : assign a TTL to temporary information (e.g., monthly budget) and run scheduled soft‑deletion to avoid stale data.

Privacy / Right to be Forgotten : use soft‑delete flags to hide data from queries while retaining audit logs for regulatory review, satisfying compliance in finance and other regulated domains.

6. Common Misunderstanding: Memory vs RAG

Retrieval‑Augmented Generation (RAG) fetches generic knowledge, whereas Memory retrieves user‑specific history. Both coexist in a system but serve different questions.

7. Full‑Score Interview Answer

Summarize the five‑layer logic:

Classification : semantic / episodic / procedural, stored in separate partitions.

Architecture : short‑term sliding window + long‑term vector DB, partitioned by user ID.

Management : Mem0’s ADD/UPDATE/DELETE/NOOP with semantic deduplication and TTL cleanup.

Retrieval : user‑ID filter then semantic search, top‑5 injected into prompt.

Compliance : soft‑delete + audit log to meet the right‑to‑be‑forgotten.

Conclusion

Agent memory hinges on three dimensions—classification, architecture and management—and must address where memory lives, how it is retrieved, when it is updated and when it is deleted.