Helix Foundry: Graph & SQL Unite to Auto-Build Cross-Source Enterprise Ontology

Why Build an Enterprise Ontology Across 11 Datasets?

Helix Foundry's official example simulates a B2B SaaS company called Loopwell. It organizes data from PlanetScale, WorkOS, PostHog, Stripe, and S3 into 11 interrelated datasets covering accounts, users, organizations, identity connections, product events, subscriptions, billing, support tickets, and marketing campaigns. The example deliberately retains common real-world data issues: payment emails differ from login emails, support tickets lack account links, and marketing codes contain entry errors.

Analyzing a business customer's health requires joining data across these systems. When different systems use different identifiers for the same entity, the challenge shifts from writing SQL syntax to entity identity resolution . Incorrect joins can duplicate revenue or attribute user behavior to the wrong customer. Traditional data warehouses handle this via ETL, master data management, dimensional modeling, and semantic layers, but cross-system mapping still requires continuous engineering maintenance.

Helix Foundry uses an ontology to organize these relationships. Its ontology contains business object types, concrete object records, and associations between objects (e.g., Customer linked to Subscription, Organization linked to User via membership). These objects and relationship edges are stored in HelixDB and can be queried and browsed. However, the ontology's fidelity depends entirely on whether source data provides reliable identifiers and mapping evidence.

How Is the Ontology Automatically Built? AI Does Not Decide Entity Relations

Initialization in Helix Foundry proceeds through three steps: AI model selection (local or hosted), data source connection (databases, SaaS tools, files), and ontology generation. Connectors for PostgreSQL, MySQL, and Neo4j support schema discovery and snapshot import; SaaS connectors (Stripe, WorkOS, PostHog) adapt to each service's data objects. Users must explicitly select each table or dataset to import.

After import, Helix Foundry produces an ontology draft using deterministic relationship discovery . Based on data structure, candidate keys, and matchable association fields, it identifies records that likely belong to the same business entity and relationships between entities. Some connectors proactively normalize fields: the Stripe connector converts customer and subscription identifiers to _id fields; WorkOS preserves identifiers linking organizations and users; PostHog's distinct_id can connect product behavior to application users. These transformations give cross-source matching a more explicit foundation.

However, key matching has inherent boundaries . Two systems having a customer_id field does not prove they use the same coding scheme; identical emails do not guarantee the same legal entity or business account. The Loopwell example's email discrepancies, missing links, and erroneous codes illustrate that real data cannot be reliably merged on field names alone. Helix Foundry provides initial relationship suggestions but cannot automatically resolve all identity conflicts; complex associations still require explicit business mapping rules.

AI's role in this phase is narrower than the project name might suggest. Official documentation states that initial ontology suggestions are computed by deterministic logic . Once the model is ready, AI can rewrite summaries, entity descriptions, and relationship names in the background, but it does not determine the initial structure. Users can rename or exclude unsuitable entities before confirming the build. This prevents unverified LLM inferences from being written directly into the business object network.

The confirmed build process materializes validated objects and relationship edges as records in HelixDB, publishing the workspace in stages. This goes beyond a simple data dictionary or relationship suggestion. For subsequent AI-generated SQL pipelines, object types, and relationship definitions, the system enforces schema checks, query execution validation, failure repair, and audit mechanisms. Formal release checks the input snapshot version, proposal hash, and current release version to prevent overwriting new versions with definitions validated against stale data.

The engineering value of this workflow is connecting otherwise scattered operations — data ingestion, relationship discovery, model definition, and publishing — into a repeatable pipeline. Users remain responsible for confirming business semantics, but the system automates repeatable structure discovery, object generation, and publishing.

HelixDB + DuckDB: How Graph Database and OLAP Collaborate

After ontology construction, Helix Foundry must enable analytical queries on business relationships. A graph database alone can show connections between customers, organizations, and subscriptions but still needs to handle event aggregation, revenue calculation, and time-window computations. Helix Foundry assigns these two workloads to HelixDB and DuckDB respectively.

According to the official architecture, HelixDB stores workspace metadata, business objects, relationship edges, tasks, and release states. DuckDB executes SQL and relational transformations on declared data inputs. Parquet stores immutable snapshots created after data import.

The system comprises an App, background Workers, an independent Executor, HelixDB, and Ollama. Data entering the local workspace is organized into versioned snapshots used by subsequent SQL pipelines, object construction, and analytical queries.

Continuing the Loopwell scenario: once the system identifies relationships between customers, organizations, and subscription records, an analyst needs to calculate regional revenue, subscription status, and product activity. Object relationships clarify which records can be joined; DuckDB reads the corresponding snapshots to perform filtering, joins, grouping, and aggregation. The analytical answer is tied to the actual SQL executed and the input snapshot version. Critically, HelixDB and DuckDB collaborate within the same workflow — this does not mean the project has implemented a unified query engine that arbitrarily mixes graph traversal and SQL.

Parquet snapshots are central to this architecture. Traditional natural-language data analysis often suffers from unexplained day-to-day result changes: the agent answers the same question yesterday and today, numbers differ, but it's unclear whether the difference stems from the model, the SQL, or underlying data updates. Helix Foundry records the data versions used for analysis, enabling users to trace which input version a query ran against. Data refresh has a defined mechanism: PostgreSQL and MySQL can trigger sync via native change capture when permissions and conditions allow; otherwise snapshots refresh on a schedule. The current implementation merges committed changes into new full snapshots rather than streaming each CDC event directly into the ontology in real time.

The graph-SQL combination also addresses execution safety. Helix Foundry does not let model-generated SQL directly access all local files or database credentials. The independent Executor loads only approved data snapshots, disables external access and extension installation, and restricts generated queries to a single SELECT statement. By default, at most two DuckDB tasks run concurrently, each with a 512 MB memory budget, spilling to disk if configured. Connector credentials are isolated from the SQL execution environment, reducing the risk of erroneous queries touching sensitive resources.

On October 10, 2026, the project added a larger workspace local test. On an M1 Pro with Docker allocated 8 CPUs and 8 GB RAM, test data included 30,000 customer records, 70,000 order records, and approximately 67,700 relationships. HelixDB v0.0.12 completed object and relationship publishing in approximately 64 seconds . This test demonstrates the project's practical publish path at that data scale but does not include all enterprise data ingestion, model inference, and continuous operating costs, nor can it directly extrapolate production throughput limits.

32 Customers, 240 Orders: What Did the Data Agent Actually Achieve?

If the first three sections address data organization, the Data Agent's value lies in whether it can perform reliable analytical tasks on that data. Helix Foundry's official validation report provides a fairly complete reference case using a local qwen3:4b model, building an analysis workspace around customer revenue and overdue orders.

In this validation, the system generated 3 data catalog entries, 1 SQL pipeline, 2 ontology object types (Customer and Order), 1 relationship type, and a dashboard with two components. Initial generation was not one-shot; automatic validation mechanisms repaired ontology and dashboard drafts. After a final 10-check suite, results passed API audit and were published. The model was invoked 6 times , consuming 14,516 tokens , and took approximately 153 seconds from phase one to a reviewable proposal, plus additional publishing time.

The published result contained 32 Customer objects, 240 Order objects , and 240 relationships connecting them, covering all 32 customers. The system also computed region-aggregated revenue and identified 35 overdue orders. Crucially, post-publish queries are bound to explicitly declared data snapshot versions. Another local verification showed that starting from a specific order record, the agent could choose a response task, execute SQL, and return the 35 overdue orders with source references.

The significance of these numbers is not the computational magnitude of 240 orders, but that they validate a complete path: from raw data to catalog and object definitions, to SQL generation, relationship validation, analytical execution, and result publishing. Users can explore data via object relationships or ask business questions directly. Compared to systems that only do Text-to-SQL, Helix Foundry adds reusable entity relationships and a validation process built around data versioning.

Of course, this result is a reference validation done by the project team in a local development environment, not an external enterprise production case. The data scale is limited, and model-generated definitions underwent automatic repair. Even with all programmatic checks passing, it does not prove that metrics like "revenue," "overdue orders," or "high-value customers" conform to a specific enterprise's business rules. The project acknowledges that syntax and execution validation cannot replace business semantic review. For a Data Agent, this is precisely the key capability boundary: execution success can be verified programmatically; business interpretation still requires an explicit accountable party.

Compared to Traditional Semantic Layers, What's Missing?

Helix Foundry is easily understood as a tool that automatically generates a semantic layer, but its focus differs from mature solutions like dbt MetricFlow. With dbt Semantic Layer, data teams define reusable metrics and join paths via Entities, Dimensions, and Metrics; MetricFlow organizes joins based on the semantic model, avoiding some duplicate calculations and erroneous joins. It does more than display metrics — it also handles semantic relationships between entities.

Helix Foundry's standout capability is building an actual business object network from multiple external sources. Connectors fetch data, deterministic programs propose entities and associations, HelixDB materializes objects, and DuckDB runs analysis. It is especially suited for scenarios where a unified warehouse model is not yet complete but rapid exploration of cross-source object relationships is desired. However, after establishing customer-subscription links, one still must define how "active customer" is calculated, whether refunds offset revenue, and how subscription changes are attributed to reporting periods. These are metric governance questions that cannot be solved by graph relationships alone.

Real-world deployment has clear limits. The project positions Helix Foundry as an executable single-server MVP , local-first, with single-user access patterns. It does not provide a full enterprise multi-tenant collaboration system and should not be exposed to public networks by default. The current version does not support writing back to source systems, distributed transactions across external systems, or distributed computing. Complex SQL joins still require business review, and pagination/browsing for large-scale ontologies needs improvement. Official backup, restore, and upgrade scripts rewritten on October 10 have not yet undergone full validation in a real Compose environment.

Another easily overlooked issue: the ontology itself requires ongoing maintenance. New data sources, field drift, and changing customer identity mapping rules can all affect existing entity relationships. Helix Foundry provides Schema Drift auditing , version checks, manual edit preservation, re-materialization, and definition rollback. It can re-validate old definitions and materialize them against current snapshots, but rollback cannot be understood as restoring all source systems' historical states. For long-term business object evolution, permission isolation, and cross-department metric governance, these mechanisms still need more complex real-world testing.

Helix Foundry's technical approach is worth watching: it places data ingestion, entity relationship discovery, ontology materialization, SQL computation, and AI analysis into a single local workflow, constraining model behavior with deterministic programs, snapshot versioning, and human review. For enterprise Data Agents, this offers an engineering direction for further validation. What ultimately determines production readiness remains cross-source identity matching accuracy, maintenance cost of complex business rules, and overall system stability as data scales.

References Helix Foundry Official GitHub (Apache 2.0): https://github.com/HelixDB/helix-foundry HelixFoundry Technical Guide: Data Source Connections, Ontology, AI, Architecture & Security: https://github.com/HelixDB/helix-foundry/blob/main/docs/GUIDE.md Guided Setup: Ontology Suggestions, Review & Build/Publish Flow: https://github.com/HelixDB/helix-foundry/blob/main/docs/onboarding.md Implementation and Validation: Local Model Validation, Object/Relationship Counts, Performance & Project Boundaries: https://github.com/HelixDB/helix-foundry/blob/main/docs/VALIDATION.md dbt MetricFlow Official Docs: Semantic Models, Metrics, Entities & Join Mechanics: https://docs.getdbt.com/docs/build/about-metricflow