Why Filesystem‑Based Memory Beats Expectations for LLM Agents – 5 Surprising Findings

Filesystem‑Based Memory for LLM Agents: Organization, Evolution, and Sustainability (arXiv:2607.26637, 2026) formalizes the industry‑default practice of storing an agent’s long‑term memory as a Markdown file hierarchy.

Memory Store Formalization

A memory store is a rooted file‑path tree. Each file contains three fields: the path, a one‑sentence description, and the body text. Folders have no content; their names together with Markdown headings inside files constitute the taxonomy visible to the agent before opening a file.

Three roles operate on the store:

Management Agent : creates, edits, merges, splits, moves, or deletes files and maintains the health of the store.

Search Agent : answers queries using the store and must cite file paths (and optionally line numbers).

Execution Agent : appears only in the skill‑memory scenario; it executes tasks and its trajectories are distilled by the Management Agent into reusable skill files.

All read/write actions go through a tool harness , which can be a set of simple file‑operation functions (e.g., ls, grep, view) or a sandboxed shell with regex and keyword ranking. The choice of tool set is an experimental variable.

Taxonomy Contracts for a “Good” Store

Same‑level entries must be distinguishable by name (and optional description) without opening the file.

Same‑level entries should be semantically related.

Parent folders must cover all child entries and vice‑versa.

Physical distance in the tree should reflect semantic relatedness.

Structure must serve retrieval; unnecessary nesting is a burden.

Memory Shapes Evaluated

Closed‑book (no memory).

Chunk retrieval (traditional RAG).

Verbatim dump (one file per turn, no processing).

Foldered sessions (LLM only creates folders, content unchanged).

Reorganized store (LLM rewrites and merges content).

Agent‑curated store (Management Agent decides what to write and how to organize).

Benchmarks

Four dialogue benchmarks (LoCoMo, PersonaMem‑32k/‑128k, REALTALK) and one embodied‑skill benchmark (ALFWorld, 140 tasks) were run with all six memory shapes, producing a dense experimental matrix.

Key Findings

RQ1 – Shape of a Self‑Organizing Store

Scaling material five‑fold (PersonaMem 32k → 128k) with a fixed Management Agent made the tree shallower: folders and files compressed into titles, and semantic distance shifted into internal headings. Swapping the Management Agent model produced dramatically different trees: small models created shallow, wide forests; large models generated deep trees with many cross‑references.

RQ2 – Is Organizing Worth It?

When material volume is large, organized stores cut retrieval cost by more than 50 % compared with the verbatim‑dump baseline. Accuracy does not uniformly improve; the cheapest “foldered sessions” baseline often matches or exceeds the fully autonomous Agent‑curated store, and the Agent‑curated store can be the worst performer on some benchmarks.

Two versions of the “reorganized” store were compared: an unconstrained “compression” version that silently drops details, and a “preserve‑all” version that enforces a hard rule to keep every fact. On long‑dialogue and real‑chat data, the preserve‑all version yields higher accuracy (the compression version’s accuracy drops by roughly 50 %); on dense PersonaMem data, the compression version performs better because the tighter store is easier to search.

RQ3 – Does a Stronger Model Use Memory Better?

Stronger Management Agents change the organization style but do not directly raise answer quality; answer quality depends on the Search (or Execution) Agent’s strength. An exception occurs when the Management Agent fails to record a state change—then a stronger model recovers about half of the loss.

RQ4 – Does Growing Memory Become Useless?

Within the experimental time span, the store does not degrade; it becomes more useful as experience accumulates. Most Management Agents let the taxonomy loosen as scale grows, and only the strongest model maintains all five contracts. Retrieval cost does not decrease with scale; the verbatim‑log baseline becomes increasingly expensive.

RQ5 – Impact of the Tool Set

Adding a single tool changes behavior modestly, but swapping the entire tool harness reshapes the store as dramatically as changing the model. In dialogue benchmarks, a more “fragment‑oriented” tool set yields a sparser but more inter‑linked store; in skill benchmarks the same swap produces a more consolidated store and better task performance.

Limitations and Open Questions

Current QA metrics ignore the organization of the store, making it hard to quantify the intrinsic value of structure.

Experiments cover only single‑dialogue or single‑task‑chain horizons; long‑term, multi‑month memory evolution remains untested.

Reference: https://arxiv.org/abs/2607.26637