Testing Four Non‑Vector RAG Approaches: BM25, GraphRAG, Tree Search, and Agentic Search

Vector‑based RAG optimizes semantic similarity but can miss logical constraints and regulatory requirements; chunking often breaks document coherence, leading to incorrect LLM reasoning.

A tiny test set of four JSON‑formatted enterprise documents (with pre‑extracted entities and relationships) is used to compare four non‑vector retrieval strategies.

Method 1: BM25 Lexical Retrieval

BM25 scores terms by frequency and inverse document frequency, similar to Google’s basic search. It excels at exact identifier lookup (e.g., error codes, clause numbers).

from rank_bm25 import BM25Okapi

tokenized_corpus = [doc['content'].lower().split() for doc in data]
bm25 = BM25Okapi(tokenized_corpus)
query = "error code E-4012"
tokenized_query = query.lower().split()
doc_scores = bm25.get_scores(tokenized_query)

Test query error code E-4012 returns the Refund Policy document with a score of 2.2774 and zero scores for other docs, demonstrating precise matching without noise. However, BM25 cannot match synonyms such as "OOM error" to an "out of memory" document.

Method 2: GraphRAG Knowledge‑Graph Traversal

When answers require multi‑hop reasoning across facts, a graph of entities and relationships is built and traversed.

import networkx as nx

G = nx.DiGraph()
for doc in data:
    for entity in doc["entities"]:
        G.add_node(entity, source_doc=doc['id'])
    for subj, pred, obj in doc["relationships"]:
        G.add_edge(subj, obj, relation=pred)

Query "What drove Revenue?" discovers the path [Product X] --(drove)--> [Revenue] and the supporting fact that Manufacturer Z provided components. Vector RAG would only retrieve a revenue‑related chunk without the causal context.

Graph construction requires clean, structured data; noisy PDFs or free‑form notes degrade entity extraction and render the graph ineffective.

Method 3: Tree Search (PageIndex) Document Navigation

For long, well‑structured documents (annual reports, contracts), the system parses a hierarchical tree (title → section → subsection) with short summaries, then lets an LLM read only the most promising nodes.

Parse the document into a tree of headings and short abstracts.

The LLM reads only the top‑level titles and abstracts.

Select the most likely node, "enter" it, read its children, and repeat until the relevant passage is reached.

Finally retrieve the full text and generate an answer.

Query "financial stability risks" navigates the tree, skips unrelated documents, and lands on the Annual Report section that mentions a new Q4 budget to mitigate identified risks, all without any embedding computation.

The approach provides clear traceability of why a node was chosen, which is mandatory in regulated environments, but it fails on untitled or unstructured sources and incurs extra LLM calls.

Method 4: Agentic Search (Self‑Researcher)

For complex, multi‑step questions with unknown answer locations, an autonomous agent plans a strategy, calls tools, evaluates results, and decides whether to continue searching.

def agent_loop(data, query):
    print("Agent: First, I'll search titles for 'Refund'.")
    results = tool_search_title(data, "Refund")

if not results:
    print("Agent: Nothing. Searching deeper in content.")
    results = tool_search_content(data, "Refund")
print(f"Agent: Got a result: {results[0]['title']}. Generating answer.")
return results[0]['content']

Running the agent on the query "What is your refund policy?" quickly finds the Refund Policy document and returns the policy text. Each reasoning step triggers a separate LLM call, making the method the most expensive and prone to infinite loops with weaker models.

Comparison and Selection Guidance

Do not pick a method based on hype; choose based on the failure mode observed in your system:

If retrieved documents are similar but incorrect, add BM25 or structured signals to improve precision.

If relevant documents are missing entirely, add vector retrieval to boost recall.

If complex queries return garbage, layer a re‑ranker or an Agent on top.

By 2026 most mature systems will be hybrid: vectors for semantic recall, BM25 for exact matching, a re‑ranker for quality, and reasoning‑based retrieval for the hardest queries.

The author recommends extracting 20–30 real production issues, running two contrasting methods on each, and comparing results—a one‑day evaluation that can save months of architectural debate.

Code Repository

The full implementation is provided in a ready‑to‑run GitLab repository. Each method lives in its own lightweight Python script and includes a SKILL.md descriptor for automatic discovery by AI agents.

# Clone and set up
git clone https://gitlab.com/public-ako74programmer/ai-projects/4-vectorless-rag-approaches.git
cd 4-vectorless-RAG-approaches
setup.bat          # creates venv and installs dependencies

# Activate the environment
venv\Scripts\activate
# Test individual methods
python .agents/skills/bm25-search/bm25.py "error code E-4012"
python .agents/skills/graphrag/graphrag.py "Revenue"
python .agents/skills/tree-search/tree_search.py "financial stability risks"
python .agents/skills/agentic-search/agentic_search.py "Refund"

Only two dependencies are required: rank_bm25 and networkx. Each script is a few dozen lines of readable Python, easy to adapt to other datasets.

Conclusion

Vector RAG works well but has production‑level limitations; non‑vector approaches are not silver bullets either. Combining them appropriately yields a system that not only finds information but finds the correct information.

Start with one method, identify its failure points, and incrementally add the next method that addresses those gaps.