Skip to content
From 'Just Query' to Verified Delivery: Goal-Constrained Ontology Construction in 5 Steps
Fundamentals 21 min read

From 'Just Query' to Verified Delivery: Goal-Constrained Ontology Construction in 5 Steps

This article outlines a five-step methodology for goal-constrained ontology construction, using an equipment management example to demonstrate how to align stakeholder expectations, define competency questions, reuse existing definitions, build minimal models with provenance, validate with normal and edge-case samples, and deliver version-aligned artifacts with explicit limitations and responsibilities.

Data Bricklaying Diary
Data Bricklaying Diary
Data Bricklaying Diary

The article opens with a concrete misalignment: an equipment department wants to "organize equipment knowledge so an assistant can query later," but technicians interpret this as linking ledgers and maintenance records for ID and type queries, while dispatchers expect "tell me which equipment can be used tomorrow." Both sides say they want an ontology, yet their acceptance criteria differ fundamentally.

Step 1: Turn "query later" into a verifiable agreement

Before starting, ask the future users: who queries, what they query, what they do with the result, and what is explicitly out of scope. In the example, the parties agree to organize identity, type, and historical status records for a specified inventory scope, support historical queries with traceability to source records, but not judge current usability or dispatch eligibility.

Acceptance criteria are fixed at this stage: can records be retrieved by confirmed identity and classification? Can historical status trace to its source? Are missing items and conflicts displayed per agreement? Which queries must pass, and which known limitations are accepted? Data cutoff date, classification version, terminology authority, and recipient are also agreed. If the user insists on current-availability filtering in this round, the scope must expand to include status timeliness, exception handling, and runtime feedback — or the scope must shrink to match capacity.

The agreement is not immutable; new types or adjusted status definitions can be evaluated for later versions, but a shift to researching unknown failure causes would require a separate exploration cycle, not a quiet expansion of the current acceptance.

Step 2: List the questions to be answered (Competency Questions)

Around the agreed historical query purpose, write concrete questions and record what the model and data must retain and how each will be verified with samples.

Competency Questions (CQs)

Question: Are the ledger and maintenance records referring to the same equipment?

What model and data must retain: ID scope, cross-source identity mapping and its basis

How to verify in samples: Verified records link; unverified retain source, no auto-merge

Question: Can a query for "equipment" retrieve both cranes and transport vehicles?

What model and data must retain: Classification used this round and equipment type membership

How to verify in samples: Check against confirmed classification; both types retrievable

Question: What status was recorded in an inventory, and what is the basis?

What model and data must retain: Status record, corresponding equipment, inventory timestamp, source

How to verify in samples: Result traces to original record; historical status not presented as current

Question: How should missing or conflicting data be displayed?

What model and data must retain: Existing content, missing items, differing source accounts

How to verify in samples: Show what is missing, where inconsistencies lie, who must verify

These are Competency Questions (CQs) , used in Ontology Development 101 (Noy & McGuinness) to scope and later validate the ontology. The number of questions is not fixed; start with key questions from the agreed purpose and edge cases prone to misinterpretation. If the purpose itself is still fuzzy, use the exploratory approach described in the next article. The CQs remain the benchmark through acceptance: for each, record the source material, expected result and basis, actual result, and any deviation — then trace back to definitions, data, or code.

Step 3: Verify existing definitions first, then fill gaps

Inventory enterprise equipment classifications, master data, inventory rules, and terminology. Check versions and applicability. Existing ontologies or industry vocabularies can be referenced but must be version-checked. For example, "crane is a type of equipment" already has a clear classification basis — no need for an LLM to guess. Two systems both using "equipment ID" but with different numbering scopes cannot be directly used as a global unique identifier.

Same ID does not prove same equipment
Same ID does not prove same equipment

Local ID equality is only a clue; cross-source identity linkage requires evidence. The diagram shows possibilities, not conclusions.

When discrepancies arise, discuss concrete terms: are two names for the same thing, or one name with two meanings? The responsible business definer confirms with evidence. Unresolved items keep both statements and sources; no forced merge. Record: term meaning, basis, scope, confirmer, and what remains unconfirmed. Out-of-scope items (lease settlement, job priority) are not expanded.

Step 4: Build a minimal model that aligns sample data

With questions and evidence, define the necessary concepts, relations, and constraints. The example needs equipment, equipment type, inventory status record, and source. Equipment has a type; a record describes an equipment at a timestamp with a source. Model granularity is driven by Step 1 questions.

Crucially, distinguish definition ("crane is a equipment" — type relation), entity (the physical crane), and record ("this crane was recorded as under repair in an inventory"). The record content is not today's status, nor does its existence prove the real-world situation is verified.

Type, entity, and record expressed separately
Type, entity, and record expressed separately

Type definitions, real-world objects, and records about objects carry different meanings; historical records cannot replace current judgments.

Map a few approved, traceable data records into the model. Specify which fields represent identity, status, time, and source. One source field may not map to a single ontology concept. Example: warehouse ledger entry "A-01, crane, recorded as under repair in an inventory." Mapping retains the warehouse system's ID scope; after identity confirmation, link related records to the equipment. Record type follows existing classification; "under repair" with its timestamp and source is kept as this record's content. Missing timestamp is flagged as missing, not defaulted to ingestion date.

Business confirms term meanings; technical implements model, mapping, and query to honor those meanings. Early discussion uses spreadsheets and samples; formal delivery moves into agreed tools later. AI can assist with terminology organization, relation candidates, or document extraction, but candidates must be verified. Existing structured fields can be mapped directly — no need to convert to natural language and re-extract.

Step 5: Check with samples, find issues, go back and fix

Do not demo only clean, complete records. Missing, conflicting, and easily mis-linked cases must enter the test suite. Agree expected handling for each scenario:

Sample Situations and Expected Handling

Sample situation: Complete identity and classification within inventory scope

Expected handling: Query returns equipment, type, history, and source per agreement

Sample situation: Two sources use same ID, identity not verified

Expected handling: Retain source and pending status; no auto-merge into single equipment

Sample situation: Record lacks timestamp to determine status time

Expected handling: Show existing content and missing item; do not fabricate current status

Sample situation: Two sources conflict on status at same timestamp

Expected handling: Retain conflict and sources; assign for verification per agreement

Sample situation: User asks "can it be dispatched today?"

Expected handling: Explain current data and scope insufficient; historical query ≠ dispatch permission

Expectations are confirmed by relevant personnel against sources, then compared with actual results. Where possible, hold out unused materials for blind testing to avoid overfitting to demo records. When a problem appears, decide whether to fix definition, data, or code; then re-run the original question and related samples. Unresolved items are documented with what is missing and what judgments remain unsupported. Detailed troubleshooting is covered in a later article.

Return to the upfront completion criteria: if key queries still fail, the round cannot be accepted per the original agreement; if data is missing but displayed per agreement, the round may meet its requirements. New issues are evaluated for inclusion — not every issue expands the model, and requirements are not quietly lowered to close the project. Definitions and mappings can be revised at any time, not only after a full waterfall pass. Check conclusions are recorded separately: a tool reporting "model file passed" must specify what it checked; query results must be actually executed. Source record accuracy is verified externally; the program cannot prove its own inputs.

No universal threshold (e.g., "85% coverage") replaces case-by-case judgment: which item was missed and what consequence it carries matters more than an aggregate score.

Step 6: Deliver version-aligned artifacts, agree on follow-up responsibilities

Deliverables include the model, material list, definitions with confirmation bases, actual mappings and queries used, check results, and unresolved items. At handover, the recipient runs the Step 1 queries against the delivered system, compares with expectations, and records satisfied, unsatisfied, and accepted limitations. A single clean-record demo is insufficient.

Record the exact versions of model, data, and program together. If delivering a running service, query from the actual entry point to confirm the version in use and that results match expectations. Call-path, cache, and switching tracing are covered in a later article.

The recipient must know who to contact for errors, how to verify and re-check, and whether new data or new use cases fall within this scope. Responsible persons, delivery date, and handling deadlines are written with real names and dates. Once scope is fixed, checks and handover complete, the round can close per agreement. Ongoing service maintenance, daily data ingestion with expiry checks, and anomaly handling are separate agreements.

Construction card: capture the round agreement in one place

The preceding content consolidates into a single construction card used from kickoff to handover. The table below shows the example entries (roles instead of names).

Construction Card Fields

Project: Equipment archive example agreement

Construction orientation: This round uses goal-constrained approach; new research questions recorded separately, do not replace current completion criteria

Purpose & boundary: Query identity, type, and historical status records within specified inventory scope; do not judge current availability or permit dispatch

Materials & versions: List approved ledgers, maintenance records, cutoff date, classification version; link model, mapping, and query versions

Core questions: Identity sameness, classification query method, status timestamp, basis, handling of insufficient materials

Completion criteria: Key queries and source checks pass; missing items and conflicts displayed per agreement; remaining unresolved items accepted with explicit limitations by recipient, otherwise continue processing or renegotiate scope

Check & results: Verify with confirmed normal, missing, and conflict samples; record actual results, failures, and unexecuted items

Handling responsibility: Equipment owner confirms classification and meaning; data provider verifies records; technical lead handles mapping and query; assign to specific persons in project

Delivery & follow-up: Confirm recipient, delivery date, error-reporting entry, handling deadline; new inventory scope or current filtering use case evaluated separately

Pack delivery version and limitations together
Pack delivery version and limitations together

Delivery must let the recipient find the adopted versions, verification bases, and uses not yet committed.

The card lives from kickoff to handover: record agreements, add evidence and results as you go, finally check what remains incomplete. Missing data is written as missing — no guessing to fill cells. With this card, "organize first, query later" becomes a concrete agreement: which queries this round, how to verify, who takes over. Future expansion has a baseline to return to.

The next article shifts to public opinion analysis: when the topic moves from brand discussion to celebrity gossip, how to research and model iteratively.

Reference

Natalya F. Noy, Deborah L. McGuinness, Ontology Development 101: A Guide to Creating Your First Ontology , Section 3 and Steps 1–2. Used for competency questions, revisable scope, ontology reuse, and iterative evaluation notes. The five-step workflow, equipment example, check expectations, and construction card are the author's suggestions, not a prescribed delivery process from the guide.

https://protege.stanford.edu/publications/ontology_development/ontology101-noy-mcguinness.html
Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

data integrationknowledge graphacceptance criteriacompetency questionsprovenance trackingontology constructiongoal-constrained modelingversion alignment
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.