Helix Foundry Open-Sources Graph-SQL Pipeline for Cross-Source Enterprise Ontology

Why an Enterprise Ontology Across 11 Datasets

Helix Foundry's official example simulates a B2B SaaS company called Loopwell. It organizes data from PlanetScale, WorkOS, PostHog, Stripe, and S3 into 11 interlinked datasets covering accounts, users, organizations, identity connections, product events, subscriptions, billing, support tickets, and marketing campaigns. The example deliberately retains common real-world data issues: payment emails differ from login emails, support tickets lack account links, and marketing codes contain entry errors.

Analyzing a single enterprise customer's health requires joining data across these systems. When different systems use different identifiers for the same entity, the challenge shifts from writing SQL syntax to entity identity resolution . Incorrect joins can duplicate revenue or attribute user behavior to the wrong customer.

Traditional data warehouses handle this via ETL, master data management, dimensional modeling, and semantic layers. However, each new SaaS tool requires re-checking business keys, foreign keys, granularity, and metric definitions — work that is hard to complete once for companies frequently adopting new tools.

Automatic Ontology Construction: AI Does Not Decide Entity Relationships

Helix Foundry's initialization splits into AI, Data, and Ontology steps. Users choose local or hosted models, then connect databases, SaaS tools, or files, explicitly selecting tables and datasets to import. Connectors for PostgreSQL, MySQL, and Neo4j support schema discovery and snapshot import; Stripe, WorkOS, and PostHog connectors adapt to each service's data objects.

After import, Helix Foundry generates an ontology draft using deterministic relationship discovery : it examines data structures, candidate keys, and matchable association fields to identify records that likely belong to the same business entity and relationships between entities. Some connectors normalize field formats — e.g., the Stripe connector converts customer and subscription identifiers into corresponding _id fields, WorkOS preserves identifiers linking organizations and users, and PostHog's distinct IDs connect product behavior to application users.

Key matching has natural boundaries. Two systems having a customer_id does not prove they use the same coding scheme; identical emails do not guarantee the same legal entity or business account. The Loopwell example's email discrepancies, missing links, and erroneous codes illustrate that real data cannot be reliably merged on field names alone. Helix Foundry provides initial relationship suggestions but cannot automatically resolve all identity conflicts; complex associations still require explicit business mapping rules.

AI's role is narrower than the project name might suggest. Official documentation states that initial ontology suggestions are produced by deterministic logic . Once the model is ready, AI can rewrite summaries, entity descriptions, and relationship names in the background, but it does not decide the initial structure. Users can rename or exclude unsuitable entities before confirming the build. This prevents unverified LLM inferences from entering the business object network.

Confirmed builds materialize validated objects and relationship edges as records in HelixDB, publishing workspaces in stages. The system adds schema checks, query execution validation, failure repair, and audit mechanisms for subsequent AI-generated SQL pipelines, object types, and relationship definitions. Formal publishing verifies input snapshot versions, proposal hashes, and current release versions to prevent overwriting new versions with definitions validated against stale data.

HelixDB + DuckDB: Graph Database and OLAP Collaboration

After ontology construction, Helix Foundry must enable business relationships to participate in actual analysis. A graph database alone can show connections between customers, organizations, and subscriptions but still needs to handle event aggregation, revenue calculation, and time-window computations. Helix Foundry assigns these roles separately: HelixDB stores workspace metadata, business objects, relationship edges, tasks, and release states; DuckDB executes SQL and relational transformations on declared data inputs; Parquet stores immutable snapshots created after data import.

The system comprises an App, background Workers, an independent Executor, HelixDB, and Ollama. Data entering the local workspace is organized into versioned snapshots used by subsequent SQL pipelines, object construction, and analytical queries.

In the Loopwell customer analysis scenario, once the system identifies relationships between customers, organizations, and subscription records, analysts need to calculate regional revenue, subscription status, and product activity. Object relationships clarify which records can be joined; DuckDB reads the corresponding snapshots to perform filtering, joins, grouping, and aggregation. Analysis answers link back to the actual SQL executed and the input snapshot version. Importantly, HelixDB and DuckDB collaborate within the same workflow , but this does not mean the project has implemented a unified query engine that arbitrarily mixes graph traversal with SQL.

Parquet snapshots are critical for reproducibility. Traditional natural-language data analysis often suffers from unexplained day-to-day number changes — whether due to model drift, SQL changes, or underlying data updates. Helix Foundry records the data versions used in analysis, letting users trace which input version a query ran against. Data refresh mechanisms exist: PostgreSQL and MySQL can trigger sync via native change capture when permissions and conditions allow; otherwise snapshots refresh on a schedule. The current implementation merges committed changes into new full snapshots rather than streaming each CDC event directly into the ontology in real time.

Execution safety is also addressed. Helix Foundry prevents model-generated SQL from accessing all local files or database credentials. The independent Executor loads only approved data snapshots, disables external access and extension installation, and restricts generated queries to a single SELECT statement. By default, at most two DuckDB tasks run concurrently, each with a 512 MB memory budget, spilling to disk if configured. Connector credentials are isolated from the SQL execution environment, reducing the risk of erroneous queries touching sensitive resources.

An official large-workspace test on October 10, 2026, ran on an M1 Pro with Docker allocated 8 CPUs and 8 GB RAM. Test data contained 30,000 customer records, 70,000 order records, and approximately 67,700 relationships. HelixDB v0.0.12 completed object and relationship publishing in approximately 64 seconds . This demonstrates the publish path at that data scale but does not include all enterprise data ingestion, model inference, or continuous operating costs, and cannot be directly extrapolated to production throughput limits.

32 Customers, 240 Orders: What the Data Agent Actually Achieved

Helix Foundry's official validation report provides a relatively complete reference case using a local qwen3:4b model to build an analysis workspace around customer revenue and delayed orders.

In this validation, the system generated 3 data catalog entries, 1 SQL pipeline, 2 ontology object types (Customer and Order), 1 relationship type, and a dashboard with 2 components. Initial generation was not perfect; automatic validation mechanisms repaired ontology and dashboard drafts. After a final 10 checks, results passed API audit and release. The model was invoked 6 times, consuming 14,516 tokens, and produced an auditable proposal in approximately 153 seconds (plus publishing time).

The release contained 32 Customer objects, 240 Order objects, and 240 relationships connecting them , covering all 32 customers. The system computed region-aggregated revenue and identified 35 delayed orders. Crucially, post-release queries bind to explicitly declared data snapshot versions. Another local test showed that starting from a specific order record, the Agent could choose a response task, execute SQL, and return the 35 delayed orders with source references.

These numbers matter not for their computational scale but because they verify a complete path: from raw data to catalog and object definitions, to SQL generation, relationship validation, analysis execution, and result publishing. Users can explore data via object relationships or ask business questions directly. Compared to pure Text-to-SQL systems, Helix Foundry adds reusable entity relationships and a validation process built around data versions.

This validation was performed by the project team in a local development environment, not an external enterprise production case. Data scale is limited, and model-generated definitions underwent automatic repair. Even with all programmatic checks passing, it does not prove that metrics like "revenue," "delayed orders," or "high-value customers" conform to any specific enterprise's business rules. The project acknowledges that syntax and execution validation cannot replace business semantic review — a key capability boundary for Data Agents: execution success can be programmatically verified, but business interpretation still requires an accountable human.

Gaps Compared to Traditional Semantic Layers

Helix Foundry is easily mistaken for an auto-generating semantic layer tool, but its focus differs from mature solutions like dbt MetricFlow. dbt's semantic layer lets data teams define reusable metrics and join paths via Entities, Dimensions, and Metrics; MetricFlow organizes joins based on the semantic model, avoiding some duplicate calculations and erroneous joins. It handles semantic relationships between entities, not just metric display.

Helix Foundry's standout capability is building an actual business object network from multiple external sources. Connectors fetch data, deterministic programs propose entities and associations, HelixDB materializes objects, and DuckDB runs analysis. It suits scenarios where a unified warehouse model is incomplete but rapid exploration of cross-source object relationships is desired. However, after linking customers to subscriptions, one must still define how to count "active customers," whether refunds offset revenue, and how subscription changes attribute to statistical periods. These metric governance problems cannot be solved by graph relationships alone.

Deployment limitations are explicit. Helix Foundry is positioned as an executable single-server MVP , local-first, with single-user access patterns. It lacks a full enterprise multi-tenant collaboration system and should not be exposed to public networks by default. The current version does not support writing back to source systems, distributed transactions across external systems, or distributed computing. Complex SQL joins still require business review, and large-scale ontology pagination and browsing need improvement. Backup, restore, and upgrade scripts rewritten on October 10 have not yet undergone full validation in a real Compose environment.

An often-overlooked issue: the ontology itself requires ongoing maintenance. New data sources, field drift, and changing customer identity mapping rules can affect existing entity relationships. Helix Foundry provides Schema Drift auditing , version checks, manual edit preservation, re-materialization, and definition rollback. It can re-validate old definitions against current snapshots but cannot treat rollback as restoring all source systems' historical states. For long-term business object evolution, permission isolation, and cross-department metric governance, these mechanisms still need more complex real-world testing.

Helix Foundry's technical approach is worth watching: it places data ingestion, entity relationship discovery, ontology materialization, SQL computation, and AI analysis into a single local workflow, constraining model behavior with deterministic programs, snapshot versioning, and human audit. For enterprise Data Agents, this offers an engineering direction for further validation. Whether it reaches production ultimately depends on cross-source identity matching accuracy, the maintenance cost of complex business rules, and system stability as data scales.

References: Helix Foundry official GitHub (Apache 2.0): https://github.com/HelixDB/helix-foundry HelixFoundry Technical Guide: data source connections, ontology, AI, architecture, and security mechanisms ( https://github.com/HelixDB/helix-foundry/blob/main/docs/GUIDE.md ) Guided Setup: ontology suggestions, audit, and build/release process ( https://github.com/HelixDB/helix-foundry/blob/main/docs/onboarding.md ) Implementation and Validation: local model validation, object/relationship counts, performance, and project boundaries ( https://github.com/HelixDB/helix-foundry/blob/main/docs/VALIDATION.md ) dbt MetricFlow official documentation: semantic models, metrics, entities, and join mechanisms ( https://docs.getdbt.com/docs/build/about-metricflow )