Agent Wiki: Compiling Knowledge at Ingestion Time – Four Implementations Compared
This article analyzes Agent Wiki systems that compile knowledge from source documents into structured Markdown at ingestion time, contrasting with retrieval-augmented generation; it reviews four implementations (DeepWiki, AutoWiki, OpenWiki, GBrain), their shared three-layer architecture, limitations around scale and accuracy, and the critical distinction between document knowledge and user memory.
Core Idea: Compile at Ingestion, Not at Query Time
The common approach for handling large document collections is retrieval: documents are chunked, embedded, and relevant chunks are fetched per query. This method works but rebuilds answers from raw chunks each time, paying the same cost repeatedly. Agent Wikis shift this cost: the model reads sources once, writes results to Markdown pages, and keeps them updated as sources change. When new sources arrive, the model reads them, updates relevant pages, corrects summaries, and flags inconsistencies.
Both approaches are valid but differ in when cost is paid and what remains after a query. Every system shares three layers: (1) source documents (articles, papers, codebases) read but not modified; (2) a wiki written entirely by the model in Markdown, containing summaries, topic pages, and cross-links; (3) a schema file (e.g., CLAUDE.md or AGENTS.md) that defines wiki structure and maintenance tasks, making the model a competent wiki maintainer.
The system performs three operations: Ingestion – model reads a new source and writes data to relevant pages; Query – user asks the wiki, and a good answer can be written back as a new page; Check – model scans the wiki for contradictions, outdated information, and orphaned pages.
Why It Works
Human-maintained wikis become inaccurate over time because maintenance—fixing links, keeping summaries current, comparing new documents to existing pages—is endless, unrewarded work that busy teams abandon. Models don't get bored, don't forget links, and can edit fifteen files in one pass. The idea dates back to Vannevar Bush's 1945 Memex, a personal document store with cross-links; Bush didn't solve maintenance, but models do.
Origin of the Name
Andrej Karpathy's April 2026 GitHub Gist introduced the term "LLM Wiki." He described the usual method as "LLMs re-discover knowledge from scratch on every query. No accumulation." His approach compiles information instead of retrieving it: "Knowledge is compiled once and kept up to date, not re-derived at query time," yielding "a durable, compounding artifact." Karpathy emphasized that humans rarely write the wiki; the LLM writes and maintains it, using Obsidian as the IDE, the LLM as programmer, and the wiki as codebase. The gist also gave a scale limit: without embeddings, the method works "surprisingly well at medium scale (~100 sources, hundreds of pages) and avoids the need for embedding-based RAG infrastructure." For larger collections, add search (e.g., qmd, a local hybrid BM25/vector search engine with LLM reranking). The rule is about scale, not replacement: avoid retrieval infrastructure when the source set is small; add it when it grows.
What These Labs Actually Built
Cognition: DeepWiki – Wiki as Public Tool
Cognition applied the method to public GitHub repositories. Replacing github.com with deepwiki.com yields a wiki with architecture summary, file index, dependency graph, search, and links to source code. Over 50,000 largest public repos already have wikis, including MCP and LangChain. Crucially, the wiki is not the product; it's the retrieval infrastructure for the Devin agent, serving as the compilation layer beneath code search in Devin.
Factory: AutoWiki – Documentation as Build Artifact
Factory integrated the method into CI. Documentation must be a build artifact, not a separate project; it derives from source code, mirrors the codebase structure, and changes when the repo changes. Ingestion occurs in two passes: a structural scan (README, package manifests, CI config, entry points) followed by a semantic scan (routes, API endpoints, service classes, database schemas, feature flags). Work is distributed to specialized agents, each responsible for a repository slice with enough context to write a high-quality page, avoiding the known problem of a single agent failing on large repos. Correctness is enforced by infrastructure, not discipline: a /wiki command regenerates the wiki, and /install-wiki adds a CI workflow that regenerates on every push to the default branch, publishing to the repo's wiki tab.
LangChain: OpenWiki – From Code to Everything
LangChain released OpenWiki as an open-source CLI that writes and maintains agent documentation for codebases. OpenWiki Brains added two modes: Code Brain for repositories and Personal Brain for personal sources (Gmail, Notion, git, X, Hacker News, web search), consolidating all into a local Markdown wiki read by agents. This shifts the method from documenting a single repo to documenting a person's work. All teams made the same output decision: the output is not human-readable text but structured Markdown for LLM context, with headings, cross-links, and summaries, enabling agents to quickly locate relevant information—the wiki's reader is the model.
GBrain: Personal-Scale Open Source
GBrain applies the method to personal knowledge bases rather than codebases, using Markdown in a git repo with a schema file and auto-generating a topic link graph. It demonstrates the approach needs minimal infrastructure: no vector database, no services, just files maintained by the model, also readable by humans.
Technical Matrix
All four systems share the same structure: Markdown in git, a schema file, compile-at-ingestion, regeneration on source change, pages written for agent consumption. Four teams solving four different problems converged on the same structure, strongly suggesting it is correct. They differ in maintenance: Factory uses CI; the other three rely on manual command runs, so wiki correctness depends on the last run.
Where It Stops
Limit 1: Scale. Karpathy noted the no-embedding approach works up to ~100 sources; beyond that, a search engine (hybrid BM25/vector) must be added.
Limit 2: Accuracy. Compilation at ingestion risks dropping details from early summaries; every subsequent answer inherits that error. Retrieval from raw chunks avoids this, trading repeated compute for data-loss risk.
Limit 3: Staleness. A page is only as correct as its last update. Factory's CI-driven approach matters because a wrong wiki is worse than none—wrong information wears the format of truth.
Limit 4: Cost. Tokens are spent generating pages, possibly for pages no one reads, and checking unchanged pages.
Wiki Is Not Memory
The term "memory" is used ambiguously. First meaning: knowledge about a document set—what the wiki provides. It compiles your documents, codebases, or Gmail data and tells you what they contain. Second meaning: memory about a user—preferences, decisions, rejected approaches, results from other apps. This data is tied to a person, not a document set; it comes from interactions, not ingestion; and it requires per-user tasks: resolving contradictions, removing outdated info, preserving provenance, and deleting on request.
Wikis handle the first task; they cannot handle the second. Your Gmail wiki tells an agent what's in your Gmail, not that you changed a decision in a Tuesday conversation or that an approach failed for you. A memory layer (e.g., Mem0) handles the second: it stores memories per user_id, so memories follow the person across sessions, apps, and agents; it updates facts in place rather than appending new records. The two systems are complementary, not substitutes. The mistake is not using a wiki, but thinking a wiki gives you user memory.
Summary
The Agent Wiki idea is sound: compile knowledge once, keep it correct, don't rebuild per query. Maintenance that doomed human wikis is trivial for models. Four teams independently built the same structure in months—strong evidence. Three actions: compile documents into pages when the corpus is stable and frequently read; add retrieval as the corpus grows, per Karpathy's gist; distinguish document knowledge from user memory—wikis provide the former, not the latter.
References
Andrej Karpathy, LLM Wiki (GitHub Gist, April 2026) – https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
qmd: local hybrid BM25/vector search for Markdown – https://github.com/tobi/qmd
Cognition, DeepWiki: AI Documentation for Any Code Repo – https://cognition.com/blog/deepwiki
Devin Docs, DeepWiki – https://docs.devin.ai/work-with-devin/deepwiki
Factory, Introducing AutoWiki – https://factory.ai/news/wiki
Factory Documentation, AutoWiki Overview – https://docs.factory.ai/cli/features/wiki/overview
langchain-ai/openwiki (GitHub) – https://github.com/langchain-ai/openwiki
LangChain, Wiki Memory – https://www.langchain.com/blog/wiki-memory
garrytan/gbrain (GitHub) – https://github.com/garrytan/gbrain
Vannevar Bush, "As We May Think" (The Atlantic, 1945) – https://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/
Mem0 – https://mem0.ai/
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
