Why Data Agent Stalls at 70% and Hits 95% Only With a Semantic Layer

Event Overview

On July 4 in Beijing, the Data for AI community hosted a four‑hour meetup with four technical talks, an 80‑minute roundtable, and three lightning talks, drawing nearly 100 on‑site attendees and over 4,000 online viewers.

Core Challenge

When agents become the primary consumers of data, traditional data stacks—designed for human users—cannot keep up. Without a semantic (context) layer, Data Agent accuracy hovers around 70%; adding a robust semantic layer can push accuracy to the 95% threshold that customers consider production‑ready.

Keynote: Engine‑Centric to Metadata‑Centric

JP Du (Datastrato founder, Apache Gravitino project lead) traced AI‑question‑answering evolution from NL2SQL to ChatBI to full‑task Data Agent. He argued that the data architecture must move from engine‑centric to metadata‑centric, because stronger models alone cannot close the accuracy gap; the missing piece is precise metadata and contextual understanding.

Six‑Round Convergence (Liu Binhui)

Round 1 : Ingest full schema, docs, and historic SQL – accuracy ~70% and unstable.

Round 2 : Build a trustworthy asset scorecard (owner, freshness, sensitivity) to select the right table among dozens.

Round 3 : Map business jargon (e.g., “effective sales”) into semantic metrics, dimensions, and entities.

Round 4 : Introduce a verified metric mechanism so only vetted metrics follow templates.

Round 5 : Add pre‑execution checks (permissions, limits, intent compliance) before running SQL.

Round 6 : Close the feedback loop to continuously correct metric definitions, schema drift, and business description errors.

Metadata Evolution (Tao Qing)

OceanBase expert Tao Qing described the shift from storage‑only systems to cognitive data bases. He presented the open‑source project ContextSeek , which acts as a “context socket” that layers context items (raw → extracted → knowledge → skill) and employs a “dream” mechanism to aggregate and evolve context while treating all dream‑generated assets as low‑confidence until reinforced by feedback.

Unified Metadata at Scale (Ding Tianbao)

China Mobile’s technical lead Ding Tianbao shared the deployment of Apache Gravitino 1.2 across nine data centers with tens of thousands of nodes. After implementing Kerberos trust, connector development, and predicate push‑down, file caching, and dynamic partition pruning, cross‑cluster query performance improved dramatically (Parquet 4.47×, ORC 3.93× faster).

Roundtable Highlights

“When the consumer changes from human to agent, the business model of selling a better software is no longer sufficient.”

Panelists debated engine load (agents generate 10‑100× more queries than humans), the claim that “SaaS is dead” in the agent era, and the role of Forward Deployed Engineers (FDE) as a transitional versus long‑term function. They also outlined three approaches to feeding agents: document‑based, structured (semantic layer/RAG), and parameter‑based (model fine‑tuning).

Takeaways

To achieve production‑grade reliability, start with a small, well‑scoped semantic layer (5‑10 core assets, 10‑20 real questions), iterate through the six‑round convergence process, and continuously refine context via feedback loops. Building trustworthy metadata and reducing entropy are essential for the next generation of agentic data systems.