Start with One Scenario or Build Full Ontology? Preventing Pilot Silos

Deliver a Concrete Business Result in the First Phase

The article opens with a procurement example: a supplier qualification assistant that checks material-category qualifications and expiry dates. Months later, a supplier performance analysis team discovers a mismatch — the qualification assistant records individual legal entities, while performance analysis rolls up to group level. This forces both teams to stop and reconcile what "supplier" means.

One reaction is to demand a full enterprise ontology upfront, but that risks endless meetings and delayed delivery. The recommended approach: deliver one verifiable capability first, while explicitly capturing shared objects, applicable scope, and change responsibilities. The scope is narrowed: the assistant verifies whether a specific company's qualifications meet a specific purchase requirement for a specific material category and purchasing organization, citing the evidence. Performance, settlement, and logistics are deferred.

Following Ontology Development 101 (Noy & McGuinness), the team defines the purpose, questions to answer, users, and maintainers, and checks for reusable existing ontologies. The output is a concrete answer: "Qualifications satisfy this requirement, based on…" or "Missing proof X, pending re-check." A vague "supplier qualified" tag is rejected because it conflates qualification, on-time delivery, and approval-to-order — leaving the next team to guess.

Agreements That Cannot Wait for the Second Team

Even though group-level performance risk is out of scope for phase one, certain decisions must be recorded now:

Object identity: Reuse verified company identifiers; group is a separate object linked by evidenced affiliation. A group company's qualifications do not automatically apply to its sisters.

Phase-one question: Verify only the supply scope and qualifications required for this purchase. Do not equate qualification pass with overall fitness.

Shared vs. local definitions: Company identity is shared; qualification requirements are tagged with organization, material category, and validity period.

Model division: Qualification check and performance evaluation are separate models that reference the same company entity.

Versioning and usage: Record definitions, data mappings, service versions, and consuming applications.

Maintenance and disputes: Assign specific roles (not just departments) for shared definitions, scenario-specific definitions, and conflict resolution.

These decisions are captured in an Ontology Architecture Decision Table , inspired by Architecture Decision Records (ADR) (Nygard, 2011). Each entry includes context, rationale, and consequences. The table is maintained from design through delivery, archived with the corresponding model, code, and tests, and made accessible to business stakeholders.

Use the Second Scenario as a Sandbox Simulation

After the first model stabilizes, the performance analysis team brings sample cases for a joint walkthrough — not to build the second system, but to test three things:

Do both sides recognize the same legal entity?

Can each side understand the other's result scope?

Will new evaluation logic corrupt existing definitions?

Example: A company's qualifications are current, but its past delivery performance was poor. Both facts can coexist; the performance team adds an evaluation with explicit time window and order scope, without changing the qualification result to "unqualified." Procurement then applies its own rules to decide.

Group and company views coexist: qualification checks target the transacting company; performance analysis aggregates by a clearly defined group boundary and affiliation snapshot. The ontology (or a relational model) must preserve these distinctions. The simulation exposes integration issues early without loading phase-one with phase-two work.

When Definitions Change: Tracking Affected Consumers

Later, material qualification rules change. A simple rule update is insufficient; the team must confirm:

Which purchasing organizations are affected?

When does the new rule take effect?

Do in-flight purchase requests follow the old or new rule?

Downstream consumers — qualification assistant, procurement dashboards, report filters, application decision branches, agent tool descriptions — must be validated. Historical queries must reproduce the original basis; re-evaluations are stored separately, not overwritten.

Example: A new proof requirement is added. The interface still returns "qualification passed" without error, but the underlying criteria have shifted. Consumers must verify their logic still holds.

Impact analysis follows a dependency chain: "Qualification definition → Verification service → Assistant, dashboards, filter conditions." The dependency registry shows who uses the definition; impact assessment determines which calls, results, and in-flight tasks need adjustment. Unregistered applications are discovered via interface and code scans — absence from the catalog does not mean no impact.

Where business permits, a migration window keeps the old version for history, but the old version must be blocked from producing valid conclusions for current checks once the new rule is mandatory. A mere "version outdated" warning is insufficient.

When to Stop the First Phase

Early governance can be handled by existing business and data owners; formal coordination bodies are added only when real cross-scenario conflicts arise. However, temporary IDs, unconfirmed definitions, and ownerless mapping tables must be resolved before they become dependencies for other applications.

Phase-one completeness is validated against a set of representative procurement samples agreed upfront: complete materials yield correct verification, non-conforming materials surface reasons, missing materials are flagged for follow-up. The sandbox simulation with the second scenario must reveal no unresolved identity or definition conflicts. Simulation exposes problems but does not prove cross-scenario reuse is production-ready.

Unneeded definitions can be deferred, but any relationship whose absence would break a judgment cannot be omitted for "lightweight" reasons. The number of classes and relations is not the criterion for phase completion.

Summary

Phase one does not need to model the entire enterprise. Doing the immediate qualification check well, while making the model understandable, discoverable, and extensible for the next team, already establishes a useful foundation.

Scope is set but data is incomplete — can we govern and pilot simultaneously? The next article addresses this concrete question.

References

Natalya F. Noy, Deborah L. McGuinness, Ontology Development 101: A Guide to Creating Your First Ontology , Steps 1–2. Used for scoping from questions and checking reusable ontologies; the procurement case and governance conventions are the author's design.

https://protege.stanford.edu/publications/ontology_development/ontology101.pdf

Michael Nygard, Documenting Architecture Decisions , 2011. Used for the practice of recording decision context, decision, status, and consequences; the article's decision table is a summary tailored to the pilot scenario, not a full ADR template.

https://www.cognitect.com/blog/2011/11/15/documenting-architecture-decisions