Building High-Quality AI Datasets: From Business Scenarios to Semantic Data Products

Step 1: Define Business Scenarios and AI Tasks

High-quality dataset construction must begin with the business problem AI needs to solve, not with existing data inventory. The author proposes a value narrative template to constrain scope:

通过某种 AI 能力，帮助某个角色，
在某个业务场景中，
从当前状态转变为目标状态，
并用明确指标验证结果。

For example, instead of a generic "equipment failure dataset," the goal becomes helping maintenance personnel identify temperature sensor anomalies, issue warnings before risk escalates, and validate with precision, recall, and lead-time metrics. The more specific the scenario, the easier it is to determine data scope, labeling rules, and quality standards.

Step 2: Define Datasets with Business Semantic Models

After fixing the task, the next step is to describe the business world of that scenario. A business semantic model acts as a business specification diagram that captures not only objects but also their processes, state changes, governing rules, and supporting evidence. At minimum it must clarify:

有哪些业务对象；
对象之间有什么关系；
发生了哪些事件和过程；
对象会经历哪些状态变化；
哪些规则约束业务动作；
哪些数据能够证明业务事实；
AI 可以给出哪些判断或建议。

In the equipment anomaly example, the model expresses relationships among equipment, measurement points, pipelines, heating processes, temperature changes, alarms, maintenance records, and process rules — not just a sensor table. This model prevents data preparation from being limited by existing table structures and allows structured records, documents, logs, images, audio/video, and time-series data to jointly represent a single business fact. Modeling notations such as OPM (Object-Process Methodology) or ontologies are optional tools, not mandatory technical requirements.

Step 3: Find Data Evidence and Complete Semantic Mapping

With the semantic model in place, data sources are inventoried. The focus shifts from "how many tables/files" to:

哪个系统记录了业务对象；
哪个字段表达对象状态；
哪份文档记录了业务过程；
哪组时序数据反映状态变化；
哪个结果经过人工确认；
哪些记录可以互相验证。

A business object is often scattered across master tables, detail tables, process tables, logs, and documents. Mapping aims to identify the business meaning each source expresses and link them to unified objects, events, states, rules, and evidence — not merely to join tables. Source system, collection time, processing steps, responsible party, and trust level are recorded simultaneously, forming a data lineage that serves as both a technical processing chain and a business-fact evidence chain.

Step 4: Process Different Data Products per AI Task

Enterprises should not attempt a single "universal high-quality dataset." The same scenario data, based on a unified semantic model, can be processed into distinct data products for different AI tasks:

Training datasets: focus on sample coverage, label quality, class balance, noise and bias.

Evaluation datasets: focus on typical scenarios, hard cases, edge cases, ground truth, and evaluation stability.

RAG datasets: focus on content authority, chunk context, version timeliness, and citation traceability.

Agent context data products: focus on object state, business rules, permission boundaries, tool parameters, and action results.

These products share business objects, semantic definitions, and raw evidence but require separate processing pipelines and acceptance criteria. For instance, a single equipment maintenance record can become a training sample, a typical case in the evaluation set, an entry in the RAG knowledge base, or a semantic service that lets an Agent judge equipment status — each with different chunking, labeling, permission, versioning, and update strategies.

Step 5: Validate and Publish as Data Products

Validation goes beyond basic data quality (completeness, accuracy, consistency) and must cover four dimensions:

Basic data quality: missing values, duplicates, anomalies, format, timeliness, consistency.

Business semantic quality: object mapping, concept definitions, states, relationships, labels against business rules.

AI task effectiveness: whether models learn more accurately, RAG retrieves more correctly, Agents understand state and invoke tools properly.

Security and compliance quality: authorization scope, sensitive information, masking policies, usage restrictions, access permissions, audit requirements.

A dataset with clean format but wrong semantics will cause AI to produce stable errors; good effectiveness but unauthorized use still disqualifies it as high-quality. Upon passing validation, the published artifact includes data content, metadata, semantic definitions, processing and labeling rules, quality reports, lineage, permissions, version, owner, and update mechanism, plus explicit applicable and prohibited scenarios. Publication formats vary: files, tables, features, vector indexes, knowledge bases, or APIs wrapped via MCP tools for controlled dynamic access. All forms remain governed by unified semantics, permissions, versioning, and audit mechanisms.

Step 6: Continuous Iteration via AI Runtime Feedback

Deployment is not the end. Model mispredictions, RAG misses, Agent invocation failures, and business-user corrections are critical quality signals. These feedback loops feed back into dataset operations to:

补充缺失场景；
修正错误标签；
增加难例和边界样本；
更新过期知识；
调整语义映射；
优化权限和工具约束；
形成新的数据集版本。

This creates a true closed loop:

业务运行产生数据
  ↓
数据集支撑 AI
  ↓
AI 进入业务场景
  ↓
业务反馈暴露问题
  ↓
数据集持续更新

High quality is not achieved by a one-time check at build time but is operated continuously through business and AI usage.

Supplementary: Tool Selection for Implementation

Multiple tool categories collaborate, but tool choice must serve the construction steps, not dictate the solution:

业务建模工具用于梳理对象、过程、状态和规则；
数据集成与加工工具用于接入、转换和关联多源数据；
数据目录、质量、血缘和权限工具用于治理与审计；
标注、向量检索、知识库、语义服务和 API 工具用于按不同 AI 任务发布数据产品。

Graph databases, Data Fabric, ontology platforms, annotation platforms, and vector databases may all be suitable components, but no single product is mandatory for every scenario. Define tasks, semantics, and acceptance criteria first, then select tools that fill current capability gaps.

Summary

The rational path is:

从业务场景定义任务，
用业务语义模型描述对象、过程、状态和规则，
把多源数据映射为可追溯的业务证据，
按训练、评测、RAG 和 Agent 的不同要求加工，
完成数据、语义、效果和合规验证后发布，
再通过运行反馈持续迭代。

The final deliverable is not a batch of data but a semantic data product that sustainably supports AI applications.