How LLM Wiki Transforms Personal Agent Knowledge Management

Starting with a Directory Structure

The LLM Wiki starts with a simple set of Markdown files. The directory layout must be clear to both the Agent and humans, indicating where raw sources, topic summaries, entity information, indexes, and logs reside.

Source pages preserve evidence, topic pages synthesize knowledge, entity pages track changes, index pages provide navigation, and log pages record update history. Separating these page types prevents the Agent from mistakenly treating source summaries as final conclusions.

wiki/
  sources/
    2026-04-04-karpathy-llm-wiki.md
    2026-05-07-competitor-pricing-review.md
  topics/
    llm-wiki.md
    rag.md
    agent-memory.md
  entities/
    products/
    people/
    projects/
  indexes/
    by-topic.md
    by-source.md
    open-questions.md
  logs/
    2026-05.md
  checks/
    stale-pages.md
    contradictions.md

Page Schema: The Skeleton of Knowledge Maintenance

Each page type has a stable set of fields so the Agent knows which parts are facts, which are synthesized, and which need review. Stable schemas improve reliable automated updates.

Source Page : core fields – title, url, date, summary, claims, evidence, related_topics; maintenance goal – retain source evidence and prevent distortion.

Topic Page : core fields – stable_view, key_points, sources, contradictions, open_questions; maintenance goal – create reusable topic-level understanding.

Entity Page : core fields – name, type, attributes, timeline, related_topics, latest_status; maintenance goal – track changes of people, products, projects.

Log Page : core fields – changed_pages, reason, source_added, risk, review_status; maintenance goal – make updates replayable and auditable.

Maintenance Process: Each Ingestion Is an Edit Task

Unlike traditional QA where the LLM answers and stops, LLM Wiki requires the Agent to perform a small edit cycle whenever new material arrives:

Read the new material and generate a sources/ page.

Scan the existing knowledge base to find affected topics/ and entities/ pages.

Update the topic page’s stable_view, evidence, contradictions, and open questions.

Refresh index pages and backlinks so future retrieval finds the new content.

Record the change in logs/, noting affected pages, risks, and review status.

A key risk is over‑reliance on the LLM: it may generate plausible‑looking but incorrect conclusions. The article recommends a human‑in‑the‑loop gate for the stable_view field—draft updates are allowed, but only after human confirmation does a view become reviewed.

The workflow also defines four page states: draft (free updates), reviewed (human‑approved), deprecated (out‑of‑date), and conflict (unresolved contradictions). High‑risk conclusions must stay in draft until verified.

Health Check: Prevent Knowledge Decay

The biggest danger is silent drift—LLM‑generated content gradually diverging from truth. Health checks run periodically (e.g., weekly) and look for structural warning signs without adding new knowledge.

No source for a conclusion : a topic page makes a strong claim without a supporting source – fix by adding a source, lowering confidence, or marking as open question.

Stale pages : pages not updated for a long time while their topic evolves quickly – trigger a review and update timestamps.

Contradiction gaps : two pages give opposite conclusions without cross‑linking – add a contradictions field and create a review task.

Broken index : new pages are missing from indexes, making them hard for the Agent to find – update by‑topic, by‑source, and backlink indexes.

Wiki vs. RAG: Division of Labor, Not Replacement

RAG (Retrieval‑Augmented Generation) follows a clear pipeline: ingest → chunk → index → retrieve → rerank → generate. It excels at scale and real‑time retrieval but struggles to accumulate stable, cross‑source judgments.

LLM Wiki, on the other hand, excels at research‑level material, project memory, competitor analysis, and long‑term decision making. The two can be combined: Wiki stores durable, synthesized knowledge; RAG fetches the latest evidence. The Agent first reads the relevant Wiki topic to build global context, then uses RAG to pull fresh snippets.

LLM Wiki : best for research documents, project memory, competitor analysis, personal Agent long‑term memory; focus on page schema, source constraints, backlinks, logs, health checks.

RAG : best for customer support, policy lookup, product docs, legal retrieval, code snippet search; focus on parsing, chunking, indexing, retrieval, reranking, citation.

Combined architecture : Wiki holds stable knowledge, RAG supplies up‑to‑date evidence; answers are generated after consulting both.

Limitations and Applicability Boundary

Four inherent limits must be considered:

LLM call cost : each new document triggers a full read‑update‑log cycle, consuming significantly more tokens than a single RAG retrieval.

Markdown file scale ceiling : a few hundred pages are fine with simple file‑system search; beyond a thousand pages, performance drops and vector or full‑text search becomes necessary.

Multi‑Agent write conflicts : without transactional locks, concurrent updates can clash; current community practice is serial updates with audit logs, which may not suit high‑throughput scenarios.

Hallucination accumulation : a single erroneous stable_view can be propagated and cited later, creating cascading bias. Health checks and human gates mitigate but do not eliminate this risk.

In practice, LLM Wiki shines when the data volume is in the low‑hundreds, updates occur daily or weekly, and conclusions require cross‑source synthesis. For thousands of documents with hourly updates demanding real‑time citations, RAG should dominate and Wiki should cover only the most valuable ~20% of knowledge.

Summary

Why LLM Wiki is needed: Personal agents using only RAG can retrieve fragments but cannot accumulate stable, synthesized judgments. LLM Wiki lets the agent distill each piece of material into reusable knowledge pages, forming long‑term memory.

How to build it: Start with a Markdown directory, define schemas for source, topic, entity, index, and log pages, implement an "ingest → update → log" pipeline, and add periodic health checks to guard against decay.

Wiki vs. RAG: They are complementary; Wiki manages durable knowledge and synthesis, while RAG handles fresh evidence retrieval. The best practice for personal agents is a hybrid architecture.

Limitations: High LLM invocation cost, Markdown scalability limits, write‑conflict risk with multiple agents, and hallucination accumulation require careful evaluation based on data size, update frequency, and accuracy needs.

Getting started: Deploy RAG for basic search, then create the first 3‑5 high‑value Wiki topics. Configure source links, status, confidence, and timestamps on each page. Change the answer flow to read Wiki first, then invoke RAG for the latest evidence. Add a human confirmation gate on critical stable_view fields and schedule weekly health checks.

Implementation Checklist

☐ Ingest raw material into RAG for basic search and citation.

☐ Select 3‑5 high‑value topics and create the initial Wiki pages.

☐ Configure each Wiki page with source links, status, confidence, and last‑updated timestamp.

☐ Change the agent’s answer flow to read the Wiki topic page before performing RAG retrieval.

☐ Set a human‑review gate on the stable_view field of critical topics.

☐ Schedule a weekly health‑check task to scan for decayed pages.

☐ Split any Wiki page exceeding 200 lines into sub‑pages.