When Should an LLM Agent Extract Memory? A Deep Dive into Trigger Strategies

1. The Core Problem: Memory Extraction Has a Cost

In an agent system, "memory extraction" means scanning the dialogue history with an LLM to decide which information should be persisted, which incurs an extra API call, cost, and latency. Triggering extraction after every turn would waste resources on trivial utterances such as "OK", "got it", or a simple "hello".

The challenge is to design a trigger strategy that balances cost control with coverage of important information.

2. Claude Code: Message‑Count Threshold + Coalescing

Trigger condition

The ExtractionCoordinator (see extractor.py:253‑328) runs after each REPL turn but only extracts when the number of new messages since the last extraction reaches MIN_NEW_MESSAGES = 4:

increment = len(messages) - self._watermark  # new messages since last extraction
if increment >= MIN_NEW_MESSAGES:          # MIN_NEW_MESSAGES = 4
    await extract_memories(messages, memory_dir, model)
    self._watermark = len(messages)

The _watermark records the total message count at the previous extraction; only when at least four new messages appear does the system call extract_memories(). Four messages roughly correspond to a meaningful exchange (at least two turns), making the extra LLM call worthwhile.

Coalescing: Preventing Concurrent Writes

When users send messages quickly, a new extraction request may arrive before the previous one finishes. Two naive approaches are rejected:

Queueing : serially processing each request would accumulate latency.

Debounce : dropping intermediate requests could lose important information.

Claude Code uses a coalescing state machine with three flags:

_running: bool = False   # is an extraction currently running
_dirty: bool = False     # did a new request arrive while running?
_watermark: int = 0      # message count at last extraction

The workflow (expressed as an ordered list) is:

New request arrives; if _running is True, set _dirty = True and return.

If _running is False, acquire a lock, set _running = True, and start extraction.

When extraction finishes, check _dirty. If False, exit.

If _dirty is True, clear the flag and run another extraction to handle messages that arrived during the previous run.

Repeat until _dirty becomes False.

This guarantees that every new message will eventually be scanned while ensuring that at most one extraction runs at any time.

3. Generative Agents: Importance‑Score‑Driven Reflection

Instead of counting messages, the 2023 Generative Agents paper (Stanford & Google, UIST 2023) assigns an importance score (1‑10) to each newly written memory. Scores accumulate in importance_sum. When the sum exceeds a threshold (150 in the paper), a reflection step runs:

importance_sum += new_memory.importance_score
if importance_sum >= REFLECTION_THRESHOLD:  # 150
    run_reflection(recent_memories[-100:])
    importance_sum = 0  # reset

Reflection asks the LLM to process the latest 100 memories and produce higher‑level insights (e.g., personality traits) that are then stored with higher importance scores. This event‑driven approach avoids extracting on trivial turns, but it requires an LLM call for every new memory to obtain the score, trading many small calls for fewer large ones.

Three‑Dimensional Retrieval Scoring

When retrieving memories, Generative Agents combine recency, importance, and relevance:

score = α * recency_score + β * importance_score + γ * relevance_score

recency_score : exponential decay, decay_rate ^ hours_passed (default decay_rate = 0.995).

importance_score : the score assigned at write time.

relevance_score : semantic similarity to the query.

This weighting prevents old but highly relevant memories from dominating when they are no longer contextually appropriate.

4. MemGPT: LLM‑Self‑Decided Memory Operations

MemGPT (UC Berkeley, NeurIPS 2023) treats memory like virtual memory. The LLM can call functions to manage two layers:

Main Context : the current window of tokens.

External Storage : an unlimited backing store.

Supported functions:

# LLM can invoke:
memory_search(query: str)          # retrieve from external storage
memory_insert(content: str)        # write to external storage
core_memory_replace(key, new_val) # update always‑present core memory
conversation_search(query: str)    # search dialogue history

When the LLM decides a piece of information is worth remembering, it calls memory_insert(); when it needs past information, it calls memory_search(). This removes any external trigger logic.

Drawbacks:

Non‑determinism : the LLM may forget to call the function or call it redundantly, leading to missed or duplicated writes.

Latency : each decision incurs an extra function‑call round‑trip, slowing real‑time responses.

In practice, this design suits batch‑processing agents where latency is less critical.

5. Mem0: CRUD‑Style Post‑Extraction Classification

Mem0 (2025) focuses on how to handle the result of an extraction rather than when to trigger it. After each extraction, the system classifies the outcome into one of four operations:

ADD    ← brand‑new information
UPDATE ← existing memory changed (e.g., new job title)
DELETE ← outdated or negated memory (e.g., "I no longer need mock")
NOOP   ← nothing valuable to store

NOOP acts as a quality filter: the extraction still runs (so the LLM evaluates the content), but if no valuable update is found, nothing is written, avoiding waste.

Empirical results reported by the authors show a 26 % accuracy boost, 91 % reduction in p95 latency, and >90 % token‑cost savings compared with a naïve OpenAI baseline, mainly because NOOP prevents unnecessary writes and UPDATE replaces full rewrites.

Compared with Claude Code’s pure count‑threshold, Mem0 adds a qualitative layer: the count decides *whether* to run, and the CRUD step decides *what* to do.

6. Engineering Trade‑offs and Scenario Guidance

Summarizing the four approaches:

Claude Code : deterministic, count‑based trigger, asynchronous background extraction, suitable for real‑time coding assistants where latency must be hidden.

Generative Agents : deterministic, importance‑score trigger, good for role‑play or social agents where information density varies.

MemGPT : non‑deterministic, LLM‑driven decisions, fits batch‑oriented, context‑heavy workloads but not interactive chat.

Mem0 : deterministic count trigger plus CRUD filtering, ideal for production‑grade dialogue agents that need cost efficiency and fine‑grained update semantics.

Choosing a strategy depends on whether the agent must respond instantly, handle high‑density events, or process large volumes of history offline.

7. How to Answer This in an Interview

Structure your response:

State the problem (≈15 s): "Memory extraction costs an LLM call, so we need a balance between cost and coverage."

Outline two main families of strategies (≈20 s): rule‑driven (Claude Code, Generative Agents) vs. LLM‑driven (MemGPT).

Explain post‑extraction handling (≈15 s): Mem0’s ADD/UPDATE/DELETE/NOOP model prevents waste.

Share your own practice (≈20 s): "We adopted a Claude‑Code‑style async trigger with a configurable message threshold, reducing API calls by ~60 % without hurting recall."

Conclusion

Memory extraction is costly; a well‑designed trigger strategy—whether count‑based, importance‑score‑based, LLM‑decided, or CRUD‑filtered—ensures that agents retain valuable information without exploding API usage. Understanding the trade‑offs lets engineers pick the right design for real‑time chat, social role‑play, or batch‑processing agents.