How Semantic Assets and Agents Redefine Data Development in the AI Era

Why the AI Era Demands a New Data Development Paradigm

Large language models can generate code, but enterprises need results that are correct, explainable, consistent, and governed. The AI era exposes the hidden cost of semantic drift: existing data pipelines tolerate ambiguous meanings, which AI now amplifies at scale.

Limitations of the Old Physical‑Table + SQL Workflow

The traditional process relies on humans to:

Understand the business question.

Translate metrics (e.g., "active users", "revenue") into concrete calculation rules.

Locate the correct tables and fields among hundreds of tables.

Write and iteratively align SQL with the business side.

This model suffers from three root causes:

Physical tables describe storage structures, not business meaning.

SQL captures the computation process, not a stable semantic definition.

Documentation can explain knowledge but cannot be executed or validated directly.

What Is Data Semantic Engineering?

Data semantics consists of four categories:

Business concepts : revenue, active users, retention, GMV.

Entity objects : customer, order, product, supplier.

Analysis dimensions : time, region, channel, category.

Calculation rules : definitions, aggregation methods, filter conditions.

Engineering these semantics means turning them into assets that are definable, composable, bindable, verifiable, searchable, executable, and governable.

Core Components of Semantic Engineering

Semantic Layer becomes the governed entry point for AI agents, pre‑defining metrics, dimensions, entities, relationships, and access rules so that an agent can select from this catalog instead of writing ad‑hoc SQL. Example configuration:

cubes:
  - name: orders
    sql_table: analytics.fct_orders
    dimensions:
      - name: region
        sql: region
        type: string
      - name: created_at
        sql: created_at
        type: time
    measures:
      - name: revenue
        title: "Net Revenue"
        sql: "amount - refund_amount"
        type: sum
        filters:
          - sql: "status != 'test'"
    access_policy:
      - role: regional_manager
        row_level:
          filters:
            - member: region
              operator: equals
              values: ["{{ user.region }}"]

Logical Table abstracts the connection between physical storage and higher‑level semantics, isolating physical changes, separating business processes, and enabling semantic reuse.

Metrics, Dimensions, and Binding Relationships are organized into a declarative graph, making joins, aggregations, and filters explicit and testable.

CTE Adaptation Layer handles non‑standard tables, temporary business logic, performance optimizations, and secondary abstractions.

Additional Guarantees

Harness : a workflow framework that adds process constraints and validation for agents (e.g., DeepSeek Harness).

Engineering Validation includes naming conventions, expression validation, binding integrity checks, logical‑to‑physical table connectivity, and change‑control approvals.

Summary and Example Workflow

The role of data engineers shifts from handling each query to orchestrating agent‑driven, governed workflows.

Example query: “What is the net revenue trend for East China over the past four quarters?” The new workflow consists of three steps:

Discover : list available governed metrics and dimensions.

Select : request specific measures, dimensions, filters, and time range.

Execute : the semantic layer compiles the request into a deterministic, governance‑aware SQL and runs it.

Agent‑issued JSON intent:

{
  "tool": "query_metrics",
  "arguments": {
    "measures": ["orders.revenue"],
    "dimensions": ["orders.region"],
    "timeDimensions": [{
      "dimension": "orders.created_at",
      "granularity": "quarter",
      "dateRange": "last 4 quarters"
    }],
    "filters": [{
      "member": "orders.region",
      "operator": "equals",
      "values": ["East China"]
    }]
  }
}

The semantic layer enriches this intent with joins, aggregations, and row‑level permissions, producing a single, governed SQL statement for execution.