18 Must‑Know Agent Memory Interview Questions (How to Answer “What Did the User Say Yesterday?”)

Module 1: Fundamentals

The article starts by explaining why memory is required for large language model (LLM) agents. Because LLMs are stateless, each call starts with a fresh context, leading to three fatal problems: repeated self‑introduction, broken conversation continuity, and inability to learn from mistakes.

Interviewers often probe with the trap “Just put the whole history into the prompt?” The answer highlights three costs: token limits (e.g., GPT‑4o’s 128K window is quickly exceeded), linear token cost growth, and the "Lost in the Middle" effect where early tokens receive far less attention.

Module 2: Short‑Term Memory Implementations

Five short‑term memory strategies from LangChain are compared:

ConversationBufferMemory : stores the full dialog, no loss, but quickly exhausts tokens.

ConversationBufferWindowMemory : keeps the most recent K turns, cost‑controlled, but discards early context.

ConversationSummaryMemory : LLM‑generated summary, supports unlimited dialogs, introduces information loss.

ConversationSummaryBufferMemory : hybrid of recent full history plus summarized older turns; recommended for production.

ConversationTokenBufferMemory : truncates by token count, precise cost control, may cut meaningful content.

Example code for the production‑recommended ConversationSummaryBufferMemory:

from langchain.memory import ConversationSummaryBufferMemory
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")
memory = ConversationSummaryBufferMemory(
    llm=llm,
    max_token_limit=2000,  # ~20‑30% of model window
    return_messages=True,
    memory_key="chat_history",
)

The article also explains how to choose max_token_limit based on the model’s context window (e.g., 128 000 × 0.25 = 32 000 tokens for GPT‑4o).

Module 3: Long‑Term Memory Implementations

Three storage options are compared:

Vector Store : semantic retrieval via embeddings; simple but de‑duplication is hard.

Entity Memory : structured key‑value pairs extracted from dialogs; more interpretable but limited to extracted entities.

Knowledge Graph : triple‑based graph for complex relationships; powerful reasoning but high engineering cost.

Hybrid approaches combine vector stores with knowledge graphs for both semantic search and relational reasoning.

Example of LangChain ConversationEntityMemory:

from langchain.memory import ConversationEntityMemory
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")
memory = ConversationEntityMemory(llm=llm)
# After a turn, memory automatically extracts entities like {"X": "AI engineer at ByteDance"}

Module 4: Advanced Topics

Memory Compression – six strategies are listed, with Summary Compression being the most common. The article provides a concrete implementation of a time‑decay forgetting function based on the Ebbinghaus curve.

def should_forget(memory, current_time):
    days_elapsed = (current_time - memory["created_at"]).days
    access_count = memory["access_count"]
    retention = math.exp(-days_elapsed / (7 * (1 + access_count)))
    adjusted = retention * (memory["importance_score"] / 10)
    return adjusted < 0.1

Security Risks – three major risks are identified: PII leakage, cross‑user contamination, and prompt injection via memory. Defensive measures include pre‑write PII detection, namespace isolation per user, and marking memory as reference only.

PII detection example:

PII_PATTERNS = {
    "phone": r"1[3-9]\d{9}",
    "id_card": r"\d{17}[\dX]",
    "bank_card": r"\d{16,19}",
    "email": r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}",
}

def contains_pii(text: str) -> bool:
    for pattern in PII_PATTERNS.values():
        if re.search(pattern, text):
            return True
    return False

Self‑Evolving Memory – three paradigms are described: Reflexion (post‑task self‑analysis), Evo‑Memory (importance‑driven weight updates), and A‑MEM (agent‑driven abstraction of memories).

Module 5: Production Design and Scaling

The article presents a full system design for supporting one million users. Key points:

Hot tier (Redis + pgvector) for the most active 10 % of users (≈600 GB).

Warm tier (Milvus/Qdrant) for the next 30 % (≈1.8 TB).

Cold tier (S3) for the remaining 60 % (≈3.6 TB).

Metadata stored in PostgreSQL to filter by user, tier, importance, and recency before vector search.

Retrieval score combines recency, importance, and cosine similarity: Score = α·Recency + β·Importance + γ·Relevance.

Write path includes PII filtering, importance scoring, similarity‑based conflict detection (threshold 0.85), and optimistic‑lock versioning to avoid concurrent write races.

def write_shared_memory(key, value, agent_id, expected_version):
    current = store.get(key)
    if current["version"] != expected_version:
        raise ConflictError(f"Version mismatch: expected {expected_version}, got {current['version']}")
    store.put(key, {
        "value": value,
        "version": expected_version + 1,
        "last_modified_by": agent_id,
        "timestamp": datetime.now(),
    })

Cold‑start handling combines guided user onboarding (collecting a few key attributes) with group‑level memory transfer based on similar user profiles.

Framework selection advice:

Simple chatbots → LangChain Memory (Buffer/Summary).

Complex multi‑agent workflows → LangGraph with external store.

SaaS with multi‑tenant memory → Mem0 managed service.

Research‑grade, self‑managing memory → Letta (MemGPT).

Overall, the article equips candidates with a complete mental model of agent memory, concrete implementation snippets, trade‑off analyses, and real‑world pitfalls, enabling them to answer interview deep‑dive questions confidently.