Beyond Consistency Checks: A 7-Layer Acceptance Framework for Ontology Models

The previous article discussed how ontology models enter runtime systems: freeze the model baseline, generate or link data constraints, mapping configurations, rule references, action contracts, and semantic services as engineering artifacts, then pass them through consistency testing as a release unit.

However, passing release only proves the artifacts meet basic entry conditions for the target environment, not that they work correctly together under real business conditions. Many ontology projects still limit acceptance to two actions: running an ontology reasoner to confirm no logical conflicts, and organizing business expert reviews to confirm concepts and relationships look reasonable. Both checks are important but far from sufficient.

Ontology model acceptance is not accepting an OWL file or an ontology diagram, but verifying whether a specific model release unit can stably answer agreed business questions, correctly handle boundary cases, and provide traceable evidence for judgments and actions in specified data, rules, runtime implementation, and test environments.

What Consistency Reasoning and Expert Review Each Prove

Ontology reasoners mainly check whether the model is satisfiable under declared formal semantics, and whether contradictions exist among types, classifications, and logical axioms. They can detect issues like "a class defined as mutually exclusive types" or "constraint combinations making a class impossible to exist."

But absence of contradictions does not guarantee correct business definitions. An erroneous risk classification rule with internally consistent logic passes reasoning; a structurally correct ontology may map to wrong data fields.

Expert review addresses a different problem: whether object boundaries, term meanings, relationships, states, and rule semantics match business understanding. It identifies business errors formal tools cannot judge, but typically relies on static documents, limited examples, and personal experience, struggling to cover runtime situations like missing data, rule boundaries, interface failures, and repeated execution.

Comparison of Acceptance Methods

Consistency Reasoning : Can prove no specific logical conflicts found under declared formal semantics. Cannot prove alone: business definitions correct, real data trustworthy, mappings accurate, actions safe.

Expert Review : Can prove business personnel accept current model expression and term meanings. Cannot prove alone: all boundaries covered, engineering implementation matches model, results repeatable.

Layered Acceptance : Can prove model release unit satisfies agreed behavior in specified data, rules, and environment. Cannot prove alone: project has achieved long-term ROI or has full scale-up conditions.

Therefore, consistency reasoning and expert review should be part of acceptance evidence, not the endpoint of the entire acceptance process.

First Define the Acceptance Object, Not Just "the Ontology"

When the model is still being edited, data changes daily, rules are not locked, and the test environment is unclear, any "acceptance passed" cannot be reproduced. At minimum, one acceptance must lock:

Capability question baseline and acceptance scope

Model release unit identifier and model baseline version

Data mapping version, test dataset or data snapshot, and observation time

Rule, policy, and action contract versions

Target runtime implementation, dependent services, and test environment

Input, expected result, and evidence requirements for each test scenario

The acceptance object here is the "release unit and its target runtime implementation." Verifying only the release package cannot detect wrong rules or old mappings used after deployment; verifying only runtime results cannot explain which model version produced the result.

The previous consistency test acts like a release gate, answering "are these engineering artifacts from the same baseline and consumable by target components"; this acceptance further answers "do they behave correctly together under agreed business conditions."

Layer 1: Capability Questions and Acceptance Boundaries

Previously established capability questions are the entry point for model acceptance. Each high-priority capability question must enter the acceptance matrix, linked to model elements, data evidence, rule implementation, test scenarios, and expected results.

Acceptance cannot just prepare one normal example that yields a correct answer. At minimum it must cover:

Whether affirmative result is returned when conditions hold

Whether negative result is returned when conditions do not hold

Whether "insufficient evidence" is returned when key data is missing or expired

Whether high-risk judgments stop when multiple sources conflict

Whether questions beyond model and data boundaries explicitly indicate unanswerable

"Able to query a result" is not an acceptance standard. The result must conform to agreed type, carry necessary evidence, and remain unknown in unknown situations rather than forcing a judgment to give an answer.

Expected results must not be reverse-generated by the system under test. They should come from current regulations, confirmed decision tables, verified historical dispositions, or gold-standard samples jointly annotated by business experts (confirmed standard answer sets); disputed samples must be adjudicated first, not adjudicated while viewing system output.

Layer 2: Semantic Model and Constraints

This layer checks whether business meanings confirmed by experts have become computable, verifiable semantic constraints.

Key points include:

Whether the same business object has a stable, unique, cross-system linkable identity

Whether object types, attributes, relationship directions, and cardinalities match business definitions

Whether states are mutually exclusive, and state changes obey allowed pre-states and event conditions

Whether time, version, validity period, and applicability scope are explicitly expressed

Whether mandatory evidence, value ranges, and relationship integrity can be validated

Which conclusions use open-world assumption, which business judgments need explicit closed boundaries

The last point is easily overlooked. Under open-world semantics, "no dispute records found" does not automatically equal "confirmed no dispute." If business decisions require confirming non-existence, data coverage scope, query time, and evidence completeness conditions must be stated; otherwise the system can only return unknown or insufficient evidence.

This layer can combine ontology reasoning, SHACL, Schema validation, and model-level assertions, but tool names are not acceptance targets. The real goal is verifying whether business constraints are accurately expressed and actually effective.

Layer 3: Real Data, Mapping, and Evidence

Even a formally correct model produces wrong results if data mapping is wrong. This layer uses representative real data, or controlled data preserving real semantics and anomaly distributions, to verify:

Whether the same object's identifiers across different systems can be correctly merged

Whether attributes, states, and relationships map to correct data sources

Whether documents, interfaces, logs, and manual confirmations together form traceable evidence

Whether data missing, latency, duplication, and conflicts are correctly identified

Whether data freshness satisfies the judgment time point of current capability questions

Whether business conclusions can trace back to source records, mapping rules, and observation times

Using only manually crafted ideal examples by modelers usually only verifies the model "can work," not that it "does work" against real data. Nor is it necessary to copy all production data for acceptance; the key is test data must cover real object identities, business distributions, and anomaly patterns, while meeting desensitization and permission requirements.

Layer 4: Rules, Reasoning, and Unknown States

Business rules may be executed by ontology axioms, decision tables, rule engines, or business code. Acceptance should not require all rules to be implemented in OWL or SWRL, but should verify the same rule semantics are correctly realized in the actual execution carrier.

Each key rule must at least check:

Whether applicable objects, preconditions, and effective times are correct

Whether boundary values, combination conditions, and priorities match policies

What conflict resolution strategy is used when multiple rules conflict

Whether rule versions and input facts can be recorded

Whether results are "satisfied," "not satisfied," "undecidable," or "evidence conflict"

Whether existing capability questions and test cases regress after rule changes

In enterprise decisions, "no rule matched," "proven not satisfied," and "cannot judge due to insufficient evidence" are three different outcomes. If the model and runtime unify them as false, formal queries may succeed but business meaning is lost.

Layer 5: Exception Paths and Action Contracts

If the current model only provides query and explanation capabilities, this layer can be marked not applicable; but if the model claims to support Agent suggestions or trigger business actions, action boundaries must be accepted.

What needs verification is not "whether the interface calls successfully," but:

Whether the current role has permission to request and execute the action

Whether action preconditions and required evidence are satisfied

Which actions can only suggest, which require human confirmation

Whether duplicate requests are prevented from duplicate side effects via idempotency mechanisms

Whether timeouts, retries, and partial failures cause duplicate tickets or error states

Whether execution failures can pause, compensate, or fall back to human handling

Whether action requests, approvals, execution results, and failure reasons are fully logged

Correct answers to capability questions do not guarantee subsequent actions are safe. Query or explanation failures can usually fall back to human handling; credit limit adjustments, payments, customer notifications, and other actions produce real business side effects, so they must use stricter evidence, permission, and transaction thresholds.

Layer 6: Version Tracking and Result Reproduction

A live demo producing correct results once does not constitute stable acceptance evidence. When the same input cannot be reproduced later, the team cannot judge whether differences come from model, data, rules, services, or environment changes.

Each key acceptance record must at least link:

Capability question and test case identifiers

Release unit and model baseline version

Data snapshot, observation time, and evidence references

Rule or policy version

Action contract and dependent service versions

If the chain includes LLMs: base model, prompt, Agent orchestration, tool description versions, and key reasoning parameter values

Test environment, execution time, actual results, and log trace identifiers

Reproduction does not require returning the production world to the past, but requires retaining sufficient versions, inputs, and evidence so the team can replay the same test or explain why current results differ from historical ones.

Acceptance chains containing LLMs must distinguish deterministic results of semantic constraints from non-deterministic outputs of generative models. The former can use exact assertions; the latter should run repeatedly on a fixed evaluation set, checking whether structured conclusions, evidence citations, tool selections, and safety boundaries meet thresholds, rather than requiring identical text every time.

Layer 7: Controlled Business Outcomes

Layer 1 addresses "what to verify," this layer answers "do these results truly support business personnel in completing tasks."

After the first six layers pass, actual business owners must confirm in controlled scenarios: do the objects, judgments, evidence, and candidate actions given by the system truly support the originally agreed business tasks.

This layer can check:

Whether coverage and accuracy of key business questions meet acceptance thresholds

Whether distributions of misjudgments, missed judgments, and "insufficient evidence" conform to risk boundaries

Whether evidence presentation suffices for business personnel to cross-check

Whether human takeover points appear at correct positions

Whether results are more stable or timely than previous methods within agreed scenarios and sample ranges

This verifies whether the model release unit achieves agreed business usability in the current scenario, not a final project value assessment. Long-term adoption, overall ROI, operational costs, and cross-scenario replication remain subsequent project evaluations.

Running a Full Acceptance with a Credit Warning Case

Continuing the "identify overdue risk and form credit disposal suggestions" scenario, the following core test scenarios can be designed:

Test Scenarios

Normal overdue : Payment past due date, contract and dispute evidence complete, confirmed not in grace period and no valid dispute → Enter credit warning, return payment, contract, rule version, and calculation basis

Still in grace period : Overdue days less than valid contract's grace period → Do not enter warning, return applicable contract version and grace period evidence

Existing accepted dispute : Dispute linked to current payment and in valid accepted status → Must not auto-enter high-risk disposal, transfer to human review or agreed process

Dispute evidence expired : Dispute status exceeds allowed freshness, cannot confirm current status → Return "insufficient evidence," block high-risk judgment and action

Unauthorized action request : Ordinary business role requests customer credit limit adjustment → Reject request, record permission basis, do not call execution service

Duplicate action request : Same decision and business intent submitted repeatedly → Return existing processing result, no duplicate tasks or side effects

These cases are not six isolated interface tests. Every result should trace back to the same release unit's object definitions, data mappings, rule versions, evidence, and action contracts.

For example, even if the "normal overdue" case successfully enters warning, if it references an already expired credit policy or cannot explain the dispute data query time, it should still not be judged as passed.

Acceptance Cannot Be Signed Off by Ontology Team Alone

Layered acceptance involves different responsibility boundaries and cannot be concluded by ontology engineers or platform teams substituting for all roles.

Acceptance Content and Primary Confirmation Roles

Capability questions, expected results, business boundaries : Business owner, regulation or policy owner

Objects, relationships, states, semantic constraints : Business experts, knowledge engineers, ontology engineers

Data mapping, quality, evidence sources : Data owner, source system owner

Rule implementation, action contracts, security boundaries : Rule owner, application owner, security or compliance personnel

Automated testing, version tracking, result reproduction : Test team, platform or operations team

Controlled business outcomes and final usage scope : Business owner, risk owner

Not every role must participate in every test case, but key conclusions must be confirmed by those who truly hold corresponding business, data, rule, and risk responsibilities. Otherwise, even a complete acceptance report may just be the technical team's self-proof of their own implementation.

Acceptance Should Produce a Report, Not Just a Signature

Complete acceptance deliverables must include at least:

Acceptance scope, exclusions, and applicable scenarios

Release unit, model, data, rules, contracts, and environment versions

Capability question and layered test case coverage matrix

Expected results, actual results, evidence, and logs for each test

Defects, residual risks, waiver reasons, owners, and closure deadlines

Final conclusion and permitted usage boundaries

Acceptance conclusions can be three categories:

Pass: All key capability questions and high-risk boundaries pass, remaining issues do not affect declared usage scope

Conditional Pass: Only isolated non-critical issues exist, with explicit conditions, owners, and closure deadlines

Fail: Blocking issues remain in key business questions, evidence chains, permission actions, or result reproduction

Blocking defects in security, permissions, key evidence, and high-risk actions cannot be turned into conditional pass with a mere "will optimize later."

Parts suitable for automation — model reasoning, constraint validation, mapping tests, rule regression, contract tests — should enter CI or model release pipelines; business semantics confirmation, complex exception judgments, and controlled business outcomes still require business experts and owners. Layered acceptance does not cancel expert review, but places expert judgment into a repeatable, traceable evidence system.

Summary

The acceptance object of an ontology model is not a diagram or a file, but a specific versioned model release unit and its target runtime implementation.

Complete acceptance should unfold along an evidence chain:

Capability questions and boundaries → Semantic model and constraints → Real data and evidence → Rules and unknown states → Exception paths and action contracts → Version tracking and result reproduction → Controlled business results

Consistency reasoning proves the model has no certain formal contradictions; expert review proves business personnel accept the current expression; only layered acceptance can further prove this model suite, within its declared business scope, facing real data and runtime boundaries, still stably does the right things.

Acceptance conclusions are only valid for the bound scope, versions, data, and environment. Whether model, mapping, rules, or action contracts change later requires local regression or re-acceptance, to be decided by change impact analysis — the topic of the next article.

Ontology models are not acceptable just because they "look reasonable"; they must prove they are truly usable with reproducible business questions, real evidence, and runtime results.