PageIndex: Tree-Indexed RAG Achieves 98.7% on FinanceBench, No Vector DB Needed
PageIndex replaces vector-based RAG with a hierarchical tree index that lets LLMs reason through document structure like humans, achieving 98.7% accuracy on FinanceBench versus 50% for vector RAG, with indexing costs of $0.001 per page and no chunking or embedding required.
One-Line Summary
PageIndex is a vector-free, reasoning-based RAG engine — officially described as replicating human reading — inspired by AlphaGo, swapping vector similarity search for tree indexing plus LLM reasoning to deliver traceable, explainable, context-aware retrieval results.
Three Key Highlights
Highlight 1: No Vectors, No Chunking — Retrieval by Reasoning, Not Similarity
The difference from vector RAG is summarized as follows:
Index: Vector RAG uses a vector index; PageIndex uses a tree index .
Retrieval: Vector RAG uses semantic similarity search; PageIndex uses LLM reasoning along the tree .
Result: Vector RAG yields opaque, "mystical vibe retrieval"; PageIndex provides results traceable to explicit citations .
Context: Vector RAG only has the query embedding; PageIndex uses full context (conversation history, domain knowledge).
Retrieval is fundamentally changed: instead of computing vector distances to find the most similar, the LLM reasons about which page to turn to.
Highlight 2: Two-Stage Retrieval — Index with Base Model, Retrieve with Strongest Model
Stage 1: Index — Generate a tree-structured index for each document. Crucially, the tree structure is extracted directly from the document layout, not generated by an LLM ; the index model only does summarization and polishing, so a cheap base model suffices.
Stage 2: Retrieve — An LLM agent reasons along the tree to locate the correct section before answering. This step uses the strongest model you can afford, because retrieval precision is directly tied to model capability.
Separating the two stages makes the cost structure sensible: indexing is cheap and run once; retrieval is expensive but only reads the nodes that are actually hit.
Highlight 3: Cost Does Not Scale with Document Length, FinanceBench 98.7%
This is the hardest proof point. Native full-PDF input cost scales linearly , while PageIndex only reads the nodes hit during reasoning: a 52-page document costs 2.1× more with native input; 420 pages costs 16.6× more; 805 pages simply exceeds the context window.
Accuracy is also crushing: on the financial document QA benchmark FinanceBench , PageIndex achieves 98.7% SOTA , while vector RAG scores only 50% .
The accompanying PageIndex-OSS-Benchmark (34 PDFs, 1,945 pages, 62 questions) is publicly available in a separate repository for anyone to reproduce.
Three Key Design Decisions
Decision 1: Remove LLM from the critical path during indexing. The tree structure comes from the document's own layout information — the inherent table of contents — which does not require the model to understand and regenerate. The model only does summarization and polishing, so indexing cost can be pressed to $0.001/page — indexing a thousand-page textbook costs just over $1.
Decision 2: During retrieval, only read the hit nodes. The LLM's reasoning along the tree is equivalent to a human browsing a book: first look at the table of contents, then flip to a chapter, then scan to a section. Nodes that are not hit never enter the context, which is the root cause of cost not scaling with document length.
Decision 3: Local and Cloud tiered positioning. The open-source version targets text-based PDFs , local indexing + local storage, with citation precision at page level ; scanned documents, image-heavy PDFs, OCR, block-level citations , MCP server, and million-document File System are handled by PageIndex Cloud hosted service. The boundary is clear, no forced stuffing.
Installation and Configuration
pip install -U pageindexfrom pageindex import PageIndexClient
client = PageIndexClient(
index="gpt-5.6-luna", # build tree index, base model is enough
chat="gpt-5.6-sol" # reason along tree, use strongest model
)
doc_id = client.submit_document("report.pdf")["doc_id"]
answer = client.chat("What was the 2023 gross margin?", doc_id=doc_id)
print(answer)Indexing documents from 9 to 1,098 pages locally takes about 13 seconds to 4.5 minutes . For agent integration, official tools for OpenAI Agents SDK and Claude Agent SDK are provided, with support for multi-document retrieval, streaming, and citations.
Team Adoption Guidelines
Pilot on long-document scenarios first. Financial reports, legal filings, regulatory submissions, technical manuals, medical literature — these long, professional documents are PageIndex's home turf. Run A/B tests with real documents: vector RAG vs PageIndex, let accuracy + citation traceability speak, don't decide by gut feeling.
Calculate the cost envelope clearly. Indexing cost is low ($0.001/page, run once), but retrieval cost is tied to the chat model, and the official recommendation is to use the strongest model you can afford. Before rollout, define model budget and call-volume caps so long-document scenarios don't silently eat the LLM bill.
Turn traceability into a compliance asset. PageIndex citations trace back to page level, which fits audit, compliance, and regulatory QA scenarios better than vector RAG's mystical retrieval. Plug citation provenance into customer-facing workflows — that's its differentiated value over traditional RAG.
Scale with Cloud + File System. Dozens of documents on a single machine are fine with the local SDK; for million-document corpora, use PageIndex Cloud's file-level tree index layer, with OCR and image understanding in the cloud. Choose self-hosted or managed based on data sensitivity and scale.
Applicable Scenarios
Teams handling long, complex professional documents : QA and retrieval for financial reports, legal, regulatory filings, technical manuals, medical literature, academic textbooks.
RAG teams dissatisfied with vector retrieval's missed/false hits : seeking high precision + traceability + explainability as an alternative.
Teams that don't want to maintain a vector database : tree index is auto-generated from document layout, eliminating embedding, vector DB, and chunking entirely.
Pros, Cons, and Pitfalls
Pros
Novel and pragmatic approach: replaces similarity search with tree index + reasoning retrieval, directly hitting the "similar ≠ relevant" pain point.
Hard benchmark data: FinanceBench 98.7% SOTA vs vector RAG 50%, OSS benchmark public and reproducible.
Controllable cost, traceable: $0.001/page indexing, citations traceable to page/block level.
MIT licensed, pure Python, pip install ready, still iterating as of push date.
Potential Limitations
Local open-source version mainly targets text-based PDFs ; scanned or image-heavy documents require Cloud.
Core capability depends on LLM reasoning , so the chat model must be the strongest you can afford; retrieval cost is coupled to model capability.
Cloud features (OCR, block-level citations, MCP, File System) are commercial services requiring an API key.
Final Thoughts
If you're still hurting from vector retrieval answering the wrong question, PageIndex is worth a shot. Just remember: local version handles text-based PDFs; scanned docs and million-scale corpora need Cloud.
GitHub Repository:
https://github.com/VectifyAI/PageIndex
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
