How MIT’s RUBICON Cuts AI Agent Costs by 90% While Achieving 100% Accuracy

Problem

Enterprise AI deployments often fail because data is fragmented across heterogeneous systems (databases, document stores, email, web pages) with different query languages, schemas, and access controls. LLM‑centric agents that assume the model can understand and orchestrate all sources achieve high accuracy on clean academic benchmarks but suffer >50 % drop in real warehouses.

RUBICON Architecture

RUBICON restores control to the user via an Agentic Query Language (AQL) with three commands: FIND, FROM, WHERE. Users explicitly state what to retrieve, from which sources, and the condition in natural language. Wrappers translate each source into a uniform relational view; the LLM only translates the WHERE clause into source‑specific commands.

Example AQL

To list university professors who have won a Turing or Nobel prize, the user writes:

FIND professor_name, award
FROM wikipedia, university_warehouse
WHERE professor has won "Turing" OR "Nobel"

The LLM converts the natural‑language condition into queries for Wikipedia and the warehouse; wrappers expose both as tables, enabling deterministic joins.

Execution Modes

Interactive mode : each AQL command produces a spreadsheet‑like intermediate result that the user can inspect and correct before proceeding.

Compile mode : a sequence of AQL commands is compiled into an optimized query plan (similar to a traditional DB execution plan), reducing tool calls and cost.

Evaluation

The authors built a micro‑benchmark of seven queries, each requiring exactly two of five data sources (Wikipedia, a 97‑table anonymized university warehouse, a lab website, Gmail, and the LLM’s internal knowledge base). They evaluated three models:

OpenAI GPT‑5‑mini Google Gemini‑3‑flash‑preview Anthropic Claude‑Sonnet‑4.6 Each model was tested in two configurations: (1) vanilla chat mode (no tools) and (2) a LangChain ReAct agent with full tool access. Results:

Vanilla LLMs: 0 % accuracy.

LangChain ReAct agents: 0 % accuracy (systematic coordination failures such as missing required sources or incorrect joins).

RUBICON: 100 % accuracy across all seven queries.

Failure Analysis of Other Agents

Agents either omitted required sources or failed to join results correctly. For the professor‑award query, the LangChain agent fetched award winners from Wikipedia but never verified their professor status in the warehouse, producing many false positives. Granting the model more autonomy increased the failure surface and cost.

Cost and Latency Comparison

Average metrics (Table 3 in the paper):

Vanilla mode: < 80 input tokens, negligible cost.

ReAct agents: 20 k–470 k input tokens per query, up to 22 tool calls, cost up to $0.5, latency > 4 min.

RUBICON (GPT‑5‑mini): exactly two tool calls per query, token usage comparable to vanilla, minimal monetary footprint.

Query‑Plan Trade‑offs

For the professor‑award query two valid AQL plans were demonstrated:

Plan A – filter awards first (high selectivity) then validate professors, reducing downstream calls.

Plan B – retrieve all professors first and check each against Wikipedia, leading to linear growth in tool calls with the number of professors.

Because the user controls the plan, RUBICON can avoid the combinatorial explosion that current LLM agents cannot.

Contextual Findings

The paper cites an MIT report tracking > 300 enterprise AI projects, finding <5 % delivered measurable ROI. The authors argue that “old‑school” software engineering—clarifying data, managing interfaces, then adding intelligence—yields higher accuracy and lower cost than the prevailing AI‑centric hype.

Reference: https://arxiv.org/pdf/2604.21413