Hindsight: Evolving Agent Memory from History to Actionable Knowledge

Why Agents Repeat Mistakes Despite Long-Term Memory

Enterprises deploying data agents often encounter a puzzling failure: an agent corrected last week repeats the same error this week. The problem isn't storage — history, tool logs, and human corrections are saved — but that this information has not been organized into reusable knowledge . Traditional RAG-style memory chunks conversations, embeds them, and retrieves by similarity. This solves "can we find the information?" but not "how do we form experience from information?" Over time, new and outdated rules coexist in context, causing conflicts.

Hindsight's Architecture: Retain, Recall, Consolidation, Reflect

Hindsight extends memory beyond storage into four stages: Retain extracts structured facts, Consolidation merges facts into observations, Mental Models pre-organize key topics, and Recall/Reflect retrieve and reason over this hierarchy.

Retain: Structured Fact Extraction with Temporal Awareness

Instead of saving raw dialogue, Retain uses an LLM to extract World Facts (external rules, e.g., "net revenue deducts confirmed refunds effective Oct 1") and Experience Facts (agent's own actions, e.g., "agent generated wrong report, then corrected"). This separation prevents conflating external rules with agent execution history. Retain also links entities via semantic, temporal, and causal relations, distinguishing event time (when a rule takes effect) from ingestion time (when the system learned it), enabling accurate temporal queries.

Retain is not a fact validator; erroneous inputs can still be stored. Enterprises must decide which execution results enter long-term memory and which require business-system or human verification first.

Recall: Four-Path Hybrid Retrieval with RRF Fusion and Token Budget

Recall runs four parallel searches:

Semantic Search — meaning-based, tolerant of phrasing differences.

Keyword Search — exact matches for names, IDs, terms (supports BM25).

Graph Search — traverses entity relations (e.g., client → group → procurement policy).

Temporal Search — resolves relative time expressions ("last month") to precise windows.

Results are fused via Reciprocal Rank Fusion (RRF) , optionally re-scored by a Cross-Encoder. A Token Budget controls returned context size, letting developers trade retrieval depth for context window limits.

Consolidation: Evolving Observations and Mental Models

Consolidation runs automatically after Retain writes new facts. It groups related facts into Observations — evidence-backed judgments like "Client A consistently prioritizes stability and operational risk in technical evaluations." Observations cite supporting facts, quotes, and evidence counts, enabling traceability.

When new evidence contradicts (e.g., user switches from React to Vue), Consolidation revises rather than overwrites , producing "User previously favored React but has now switched to Vue." Original facts remain for audit.

Mental Models are developer-defined, topic-focused knowledge artifacts (e.g., "current client communication standards"). They are generated via Reflect and can auto-refresh when underlying knowledge changes. Reflect uses layered retrieval: Mental Models first, then Observations, then raw Facts, reducing repeated reasoning and token cost.

v1.1.0: Change Tracking, Scope Control, and Reliability

The October 21, 2026 v1.1.0 release adds:

Per-document change history — when a base fact is edited, appended, rewritten, or invalidated, dependent Observations are deleted and rebuilt; the history logs which operation triggered reconstruction.

Observation Scopes with tag wildcards — e.g., topic:* groups facts by matched theme tags, controlling which experiences consolidate together without pre-enumerating topics. Scopes govern consolidation boundaries, not access control (tenant isolation, permissions remain infrastructure concerns).

prefer_observations default — Recall can skip raw facts already covered by returned Observations, fitting more distinct information into the token budget.

Expanded model/embedding/reranker provider support, PDF/Markdown export, and fixes across Retain, Consolidation, Reflect, and background tasks.

Benchmarks and Production Reality

Hindsight v0.4.19 reported (March 2026) on specific benchmarks: LongMemEval 94.6%, LoCoMo 92.0%, PersonaMem 86.6%, LifeBench 71.5%. These reflect the project's test configuration, not v1.1.0 performance, and do not directly translate to enterprise task success rates. The team acknowledges current benchmarks focus on multi-turn QA, while real agent workflows involve tool calls, continuous decisions, multi-source knowledge, and cross-task experience. Critically, remembering a rule does not grant execution permission — an approval agent must still verify current permissions; a data agent must consult the live semantic layer for latest metric definitions. Authoritative business systems remain the source of truth.

Testing Recommendations for Enterprise Deployment

Instead of static QA benchmarks, test continuous learning: feed human corrections and observe stable adoption of new knowledge; change business rules over time and check for stale references; verify multi-agent sharing avoids unreasonable knowledge bleed. The value of long-term memory is not capacity but verified experience that works in the next task .

References: Hindsight GitHub repository, technical documentation, Observations and Mental Models design docs, v1.1.0 release announcement, Agent Memory Benchmark report, ACL 2026 paper.