Beyond the Lakehouse: How Databricks Is Targeting the Enterprise Agent Operating System

Over the past decade Databricks has driven enterprise data architecture from separate data lakes and warehouses toward a unified Lakehouse, adding ACID transactions, schema enforcement, scalable metadata and unified batch‑stream capabilities via Delta Lake. The next frontier, the "Agent operating system," is not a product name but an industry judgment that Databricks is assembling to provide context, state, tools, identity, permissions and execution boundaries for enterprise agents.

Four‑layer Agent Data Foundation

Layer 1 – Facts & State: Delta Lake and the Lakehouse continue to store large‑scale historical data and support streaming and analytical workloads. In January 2026 Databricks released Lakebase, a managed PostgreSQL service for low‑latency transactional use cases and online feature stores, enabling agents to read and write state close to business operations. Lakebase‑to‑Delta change data feed (Lakehouse Sync) is in Public Preview, so full replacement of existing transaction systems cannot be assumed.

Layer 2 – Business Semantics: Unity Catalog Business Semantics centers on Metric Views that define metrics, dimensions, relationships and calculation logic for shared KPI definitions. Genie Ontology, also in Public Preview, automatically extracts knowledge fragments (metric definitions, authoritative sources, business rules) from tables, queries, dashboards and documentation, assigning authority scores based on provenance, usage frequency and freshness.

Layer 3 – Runtime Governance: Unity Catalog is expanding to govern Agent Services, MCP Services, model services, functions and external connections as unified, governable objects. Unity AI Gateway routes model and MCP requests, enforcing access control, rate limiting, budgeting, guardrails, service policies and audit logging. These capabilities are still in Beta, representing a nascent control plane rather than a mature industry standard.

Layer 4 – Agent Construction & Execution: Agent Bricks provide the entry point for building, optimizing and orchestrating agents. The broader Databricks Agent stack supports multiple models and third‑party frameworks, integrates with MLflow for tracking and evaluation, leverages Databricks Apps for deployment, Unity Catalog for governance, and offers a Beta Managed Agent Memory for cross‑session long‑term memory. While not a full operating system on its own, the combination of these pieces begins to assume OS‑like responsibilities: resource management, state maintenance, identity enforcement and interaction constraints.

From Answering Questions to Constraining Actions

Traditional enterprise data pipelines store facts, BI tools generate reports, and humans act on insights. Even when analysis is flawed, the final business impact is usually driven by human decisions. Agents change this chain: they can not only surface insights (e.g., "which customers may churn") but also generate recommendations, create tasks, invoke CRM systems or trigger approvals. Consequently, semantic errors can propagate directly into erroneous actions.

To support agents, data platforms must ensure three things: reliable facts, correct interpretation of those facts by the agent, and appropriate permission to act. This requires a semantic layer to unify metric definitions, an ontology to model entities, relationships, rules and possible actions, an agent runtime to plan and invoke tools, and a governance system to decide which calls are allowed, whether human approval is needed, and how to audit the entire process.

Competitive Landscape

Databricks, Palantir, Snowflake and Microsoft Fabric all pursue similar goals but differ in implementation. Databricks focuses on open Lakehouse, data engineering and AI development, extending its platform with Agent Bricks, Unity Catalog Business Semantics, Genie Ontology and Unity AI Gateway. Palantir’s Ontology tightly integrates data, logic, actions and security, offering a mature enterprise action layer. Snowflake’s Cortex Agents (GA November 2025) provide a managed agent platform with Semantic Views, search, code execution and tool integration, emphasizing a governed data cloud. Microsoft Fabric combines Power BI semantic models, OneLake, Copilot Studio and Operations Agent (GA) to create short business‑to‑action paths, though its Ontology and integration remain in Preview.

The common thread is that enterprise agents need a governed business‑world model rather than raw tables. The competitive edge will be determined by who can define the facts agents see, explain their business meaning, and control the actions they may take.

Conclusion

Databricks cannot yet be called a mature "Agent operating system"—Genie Ontology, Unity AI Gateway, Agent Services and related MCP capabilities are still in preview, and its business‑process modeling lags behind Palantir. However, by extending from Delta Lake and Lakehouse to Lakebase and Agent Bricks, Databricks is expanding its responsibility boundary: it now manages not only data storage and computation but also the context, identity and governance that agents need to understand the enterprise and act within controlled conditions.

Future differentiation will shift from who can store more data or run faster queries to who can reliably provide the enterprise context that agents require and enforce the permissible actions when agents start altering business state.

References:

[1] Databricks: Agent Bricks at Data + AI Summit 2026 – overview of the agent platform and non‑core engineering challenges.

[2] Databricks: Unity Catalog Business Semantics – Metric Views and unified KPI semantics.

[3] Databricks: Genie Ontology – Public Preview for automatic knowledge extraction and authority scoring.

[4] Databricks: Unity AI Gateway – Beta for model/MCP routing, policies, budgeting and audit.

[5] Databricks: Lakebase GA release notes – Lakebase GA on 2026‑01‑22.

[6] Databricks: Lakebase Change Data Feed / Lakehouse Sync – Public Preview for continuous sync.

[7] Databricks: Managed Agent Memory – Beta for cross‑session long‑term memory.

[8] Databricks: Agent Services in Unity Catalog – Beta for registration, discovery and governance.

[9] Databricks: MCP Services – integration of external services with Unity Catalog and AI Gateway.

[10] Palantir: The Ontology system – unified modeling of data, logic, actions and security.

[11] Palantir: Action types – modifications to objects, attributes, relationships and side‑effects.

[12] Snowflake: Cortex Agents – managed agent platform with Semantic Views, Search, code execution and custom tools.

[13] Snowflake: Cortex Agents GA – availability announced 2025‑11‑04.

[14] Microsoft Fabric: Ontology Agent Integration – Preview for shared business context across agents.

[15] Microsoft Fabric: Operations Agent Actions – Teams notifications, Notebook, Pipeline, user‑defined functions and Power Automate.

[16] Microsoft Fabric: Operations Agent limitations – actions executed with creator‑delegated identity and permissions.