Hindsight v1.1.0: From Vector Storage to Evolving Agent Knowledge Graphs

The article opens with a concrete problem: a Data Agent that correctly learns to subtract confirmed refunds from net revenue one week, but reverts to the wrong calculation the next week despite having stored the correction. The issue is not storage — vector databases retain the history — but that raw logs are not transformed into reusable knowledge.

Why Vector Memory Alone Fails

Traditional agent memory implementations treat history like RAG: chunk conversations, embed them, and retrieve by similarity. This solves "can we find the information?" but not "how do we form experience from information?" Over time, new and outdated rules coexist in the context window, causing conflicts. The real challenge is twofold: retrieve what happened, and know which experiences are still trustworthy.

Hindsight's Four-Stage Pipeline

Hindsight structures memory into Retain, Recall, Consolidation, and Reflect. Retain does not simply save raw dialogue; it calls an LLM to extract facts, entities, relations, and timestamps, organizing them into a Memory Bank. Crucially, it distinguishes World Facts (external rules, e.g., "net revenue deducts refunds from Oct 1") from Experience Facts (the agent's own actions, e.g., "agent generated wrong report, then corrected"). This separation prevents the agent's past mistakes from being mistaken for ground truth. Retain also links entities via semantic, temporal, and causal edges, and separates the event time (when a rule takes effect) from the ingestion time (when the system learned it), enabling correct temporal reasoning.

Recall: Four-Path Hybrid Retrieval

When a query arrives (e.g., "Why did client A change pricing last month?"), Hindsight runs four parallel searches: Semantic Search for meaning, Keyword Search (BM25) for exact identifiers, Graph Search to traverse entity relations (client → group → contract), and Temporal Search to isolate the relevant time window. Results are fused via Reciprocal Rank Fusion (RRF) , optionally re-ranked by a Cross-Encoder, and filtered by a configurable Token Budget so developers can trade retrieval depth for context size.

Consolidation: From Discrete Facts to Evolving Observations

After Retain writes new facts, a background Consolidation process analyzes their relationships with existing knowledge and generates or updates Observations — evidence-backed judgments such as "Client A consistently prioritizes stability and operational risk in technical evaluations." Unlike summaries, Observations cite supporting facts, original quotes, and evidence counts. When new evidence arrives (e.g., a user switches from React to Vue), Consolidation revises the Observation to "User previously favored React but has now switched to Vue" rather than overwriting or deleting history. This knowledge evolution preserves provenance for audit.

Mental Models: Curated Long-Term Knowledge

Observations are auto-generated; Mental Models are developer-defined targets (e.g., "current client communication protocol", "KPI definitions"). Reflect populates them by hierarchical retrieval: check Mental Models first, then Observations, then raw Facts. This avoids repeated reasoning and token spend. A freshness flag warns if a Mental Model is stale relative to underlying memory changes.

v1.1.0 Enhancements (Oct 21, 2026)

Per-document change history : when a base fact is edited, appended, rewritten, or invalidated, dependent Observations are deleted and rebuilt; the new log lets developers trace which operation triggered reconstruction.

Observation Scopes with tag wildcards (e.g., topic:*) control which facts consolidate together, preventing cross-client leakage while allowing shared themes.

prefer_observations flag reduces duplicate fact returns when an Observation already covers them.

Expanded model/embedding/reranker provider support, PDF/Markdown export, and reliability fixes across Retain, Consolidation, Reflect, and background tasks.

Benchmarks and Production Caveats

Hindsight v0.4.19 reported LongMemEval 94.6% , LoCoMo 92.0% , PersonaMem 86.6% , LifeBench 71.5% under the project's test configuration. The authors caution these are multi-turn QA benchmarks, not end-to-end agent task success rates. In production, an agent remembering a rule does not grant execution permission — authoritative business systems must still enforce permissions and real-time definitions. The article recommends testing with continuous correction loops, rule changes over time, and multi-agent shared-memory scenarios.

As of the October 23 snapshot, the GitHub repository had ~52.9k total stars and ~28.1k stars added that month , ranking fourth on a third-party monthly trend list. The project is open-source (GitHub) and accompanied by an ACL 2026 paper.

Key insight: valuable long-term memory is not about storing more history, but about letting verified experience drive the next task.