Dynamic Memory Forest: Precise Long‑Dialogue Tracking for Highly Coherent Responses

Problem Statement

Although large language models (LLMs) now support context windows of up to a million tokens, they still “fragment” when handling real‑world, open‑domain conversations that span many turns and topics. Existing long‑term memory approaches—summarization or fragmented retrieval—break the semantic bridges between distant turns, losing fine‑grained dialogue trajectories.

Inspiration from Cognitive Science

Human memory follows a consolidation‑then‑growth process: after an interaction, the brain clusters related utterances and forms synaptic connections that enable cross‑topic associations. The authors adopt this principle to design a Dynamic Memory Forest (DMF) that mimics consolidation, growth, and associative walking.

DMF Architecture

DMF converts unordered, sparse dialogue histories into a structured “memory forest” composed of multiple memory trees. The pipeline consists of four stages.

Stage 1 – Memory Consolidation

Soft clustering : After a session ends, utterance embeddings are reduced with UMAP and clustered with a Gaussian Mixture Model (GMM). Each utterance receives a probability distribution over topics, from which a Thematic Entropy is computed.

Bridge‑node identification : Utterances with high thematic entropy act as semantic bridges linking multiple topics (e.g., “camera” and “travel”). These become the key nodes for later cross‑topic jumps.

Stage 2 – Memory Growth

Memory‑tree construction : Clustered units are ordered by timestamp to form a dialogue chain. An LLM generates a concise summary for the root node, preserving temporal logic.

Group‑Relative‑Voting Optimization (GRVO) : When a new tree is created, each of its utterances votes for candidate nodes in existing trees based on cosine similarity. Only if the majority score exceeds a threshold does a graft occur, preventing spurious connections and ensuring high‑quality logical links.

Stage 3 – Entropy‑Driven Memory Walk (EDMW)

During inference, the user query triggers EDMW, which traverses the forest to find an optimal memory path. The walk balances global relevance and local coherence, and an entropy amplifier boosts the likelihood of visiting high‑entropy bridge nodes, enabling cross‑tree associative jumps.

Stage 4 – Response Generation

Visited nodes are sorted by global timestamp; the root‑node summaries are injected as hierarchical memory cues. The assembled structured context is fed to the LLM, which produces a response that reflects both recent and distant dialogue history.

Experimental Evaluation

Three human‑annotated long‑term open‑domain dialogue datasets—Conversation Chronicles (CC), Multi‑Session Chat (MSC), and GapChat (GC)—were used. Base models were GPT‑4o (128 K) and Gemini 2.5 (1 M). Baselines covered four mainstream strategies: Full History, Structured RAG (e.g., MemTree), Agentic Memory (e.g., MemoryOS), and Memory‑Enhanced Generation (e.g., THEANINE).

Automatic metrics show DMF consistently outperforms all baselines. Notably, the MAUVE score, which measures similarity to human style, improves dramatically, demonstrating that providing LLMs with structured dialogue trajectories is more effective than merely extending the raw context window.

Human‑rated case studies illustrate DMF’s associative capability: when asked about a “new oven,” the model first locates the “baking” tree, then jumps via a high‑entropy bridge node to the “fitness” tree, finally delivering a response that combines both cooking and weight‑loss considerations (“make sugar‑free cookies”).

Efficiency Analysis

Compared with methods that rebuild trees per turn (e.g., MemTree), DMF processes sessions in batch after each dialogue, resulting in the lowest total token consumption among dynamic‑maintenance approaches. Construction time and retrieval latency are also significantly reduced, making DMF suitable for real‑time, long‑range interactions.

Conclusion

DMF demonstrates that applying cognitive‑science principles—soft clustering, synaptic‑like grafting, and entropy‑driven associative walks—effectively solves the fragmentation problem of long‑term dialogue memory. The study reveals that, for extended conversations, the organization of memory structures and their relational trajectories matter far more than simply enlarging the context window.