EvoOntology: Self-Evolving Ontology Layer Lets Data Agents Maintain Themselves

Introduction: The Ontology Maintenance Problem in Enterprise Data Agents

As Data Agents gain capabilities to write SQL, call APIs, and execute code, they still stumble on fundamental semantic mismatches: the same "customer" entity may differ across systems, "revenue" may map to order amount, recognized revenue, or cash collected, and risk events link to varying accounts and contracts. Ontology organizes business concepts, data objects, field relationships, and operational rules so agents understand what enterprise data actually represents. Palantir has long invested in this space, and recent Data Agent systems increasingly emphasize Semantic Layer, Knowledge Graph, and Ontology.

However, ontologies quickly become outdated as business evolves, schemas grow, field meanings shift, and agent usage patterns change. The core challenge: who continuously maintains the ontology after initial construction?

01: Stuffing Semantic Layer into Prompts Can Hurt Agent Performance

Current practice injects pre-built semantic layers (table schemas, field descriptions, business metrics, entity relations) into the agent's context. While helpful for small datasets, this approach fails at scale: the semantic layer grows complex, consumes context window, and injects irrelevant information for each task.

EvoOntology experiments on DDR-Bench with Claude-Sonnet-5 show that without ontology, Trajectory-Wise accuracy is 72.5%; injecting a static semantic layer drops it to 57.5%; EvoOntology raises it to 81.3%. On GPT-5.6-sol, baseline is 68.5%, static layer yields 65.5%, while EvoOntology reaches 93.5%. Across six tested models, EvoOntology improves Trajectory-Wise accuracy by an average of 17.8 percentage points over the no-ontology baseline.

The issue is not semantic layer's value but how agents consume it. Treating ontology as a massive prompt prefix forces agents to filter relevant knowledge each turn. EvoOntology instead exposes ontology as an independent MCP (Model Context Protocol) server that agents query on demand.

02: Ontology Becomes a Runtime Query Service with Three-Layer Architecture

EvoOntology structures ontology into three layers:

Content Layer : Stores business semantics — Terms (e.g., Revenue, Customer), Mappings (concept-to-table/field), Constraints (usage restrictions), and Evidence (supporting data samples). Each concept carries its exact field mapping, join conditions, applicability constraints, and the data evidence that justified the mapping.

Schema Layer : Manages ontology's own structure, allowing extension when new business relationships emerge.

Tool Layer : Defines how agents query the ontology. Agents receive only a lightweight Manifest initially, then use MCP browse/resolve tools to fetch relevant concepts, fields, mappings, and constraints during task execution.

This shifts ontology from a static prompt prefix to an independent runtime service within the agent harness.

03: Agent Execution Traces Drive Continuous Ontology Evolution

Initial Construction via Builder Agent : A Builder Agent ingests a batch of real tasks, extracts recurring entities, metrics, operations, and analysis conditions, then probes underlying data to validate mappings (e.g., checking field types, actual values, and relationships) before writing to ontology with supporting Evidence.

Evolution via Evolution Agent : As agents execute tasks, they leave execution traces. The Evolution Agent analyzes these traces for recurring failure patterns — wrong field mappings (Content Layer), retrieval failures despite correct knowledge (Tool Layer), or missing structural representations (Schema Layer). It then generates a localized Patch rather than rebuilding the entire ontology.

Validation Gate with Pairwise Evaluation and Safe Rollback : Candidate ontology and parent ontology are evaluated on the same verification tasks with identical model, decoding parameters, and execution budget. Only if the new version demonstrably outperforms the old is it accepted. Ablation on DDR-Bench confirms this gate's importance: full EvoOntology scores 89.5% Trajectory-Wise; removing the gate drops to 78.3%; removing attribution (error-to-layer diagnosis) also degrades performance.

The self-evolving loop is thus: run tasks → collect traces → detect failure patterns → locate layer → generate patch → validate via pairwise evaluation → accept or rollback.

04: More Ontology Usage Reduces Total Token Cost by 20%

On DDR-Bench, without ontology: 3.2K tokens/turn, 14.6 turns/task, 52.6K total tokens. With initial ontology: 4.1K tokens/turn, 11.2 turns, 50.4K total. After evolution: 4.6K tokens/turn, 8.4 turns, 42K total — a ~20% reduction in total tokens while Trajectory-Wise rises from 69.5% to 89.5% (averaged across GPT-5.5, GPT-5.6-sol, Claude-Sonnet-5, Claude-Opus-4.8).

Reason: without ontology, agents repeatedly explore (find tables, inspect fields, check values, attempt joins, infer business meaning). Ontology crystallizes verified results, eliminating redundant exploration. Compared to a ReAct baseline with Memory (retrieving past execution traces) which scores 75.8%, EvoOntology's 89.5% shows that restructuring experience into queryable, combinable structure outperforms raw history replay.

05: Same Business Knowledge Yields Model-Specific Ontologies

Four models evolved from the same initial ontology produced divergent final ontologies; pairwise Jaccard similarity of accepted Terms never exceeded 0.62. Cross-model transfer (using one model's evolved ontology for another) degrades performance by at least 6.6 percentage points, averaging 10.9 points loss. This indicates that how business knowledge should be organized, which fields to expose, and how tools return results depends on the specific model.

Accepted evolution improvements breakdown: Tool Layer 57%, Content Layer 34%, Schema Layer 9%. Over half the gains come from optimizing how agents use the ontology, not merely adding more knowledge — a shift from traditional ontology engineering focused on defining business objects and relations.

EvoOntology remains a research prototype evaluated on DDR-Bench, InsightBench, and BIRD. Real enterprise deployments face additional complexity: access control, business rule conflicts, multi-user editing, audit trails, and long-term version governance. The Parent/Candidate, validation, and rollback mechanisms demonstrate that "self-evolving" does not mean handing ontology entirely to agents, but automating a verifiable maintenance loop.

Ultimately, ontology in agent systems becomes a runtime component that evolves alongside agent work: agents use it to understand data, and their execution results continuously refine it. The question moves from "how to build an ontology for agents" to "how to let agents participate in maintaining it."

References

EvoOntology: A Self-Evolving Ontology Layer for Data Agents — https://arxiv.org/abs/2609.15779 RUC-DataLab / EvoOntology —

https://github.com/ruc-datalab/EvoOntology