Is Your Ontology Project Worth It? Measure Value by Reduced Rework, Not Graph Size

The article opens with a scenario: an enterprise builds an ontology to assist verification of broadband installation cancellations. At acceptance, the project team shows a growing graph of orders, addresses, network resources, and work orders. The business owner asks whether the system actually reduces the phone calls and manual cross-checks needed to understand why a cancellation occurred and who should handle it next. If senior staff must still be called into a meeting room to trace each case manually, the system has not replaced a single step.

Define the Concrete Task to Be Improved

Vague goals like “improve business intelligence” cannot be accepted. A testable task is: “clarify a single installation cancellation and hand a substantiated verification result to the correct role.” The article illustrates a cancellation where the reason code reads “resource issue,” the resource system shows the building covered, but the work order notes “on-site construction blocked.” Cross-checking field records and customer logs reveals the real blocker was the in-home wiring path, requiring coordination of construction conditions—not network expansion. Sending it to the expansion team would be a misrouting.

The ontology can make explicit the relationships among order, address, resource, and work-order records, distinguishing states and causes. Query and rule services then assemble evidence into a verification recommendation. Where field records are missing, the system should flag the gap rather than guess along graph edges. Following Noy and McGuinness ( Ontology Development 101 ), the author lists competency questions: which order, which address, what condition was missing at that time, where is the evidence, and who receives the result. Business acceptance must go further: did these answers reduce back-and-forth verification, and can the recipient act on them? Node counts and query latency measure content volume and performance, not whether the work improved.

Attribute Time Savings to the Right Cause

Assume average manual effort per cancellation drops from 20 minutes to 9 minutes. The 11-minute reduction cannot be wholly credited to the ontology. Concurrently, the project fixed address coding, linked construction records, and clarified assignment responsibilities—changes that would save time even without an ontology. To isolate the ontology’s marginal contribution, the article proposes comparing three approaches under comparable tasks, personnel, and quality requirements:

Original process: 20 minutes – staff query multiple systems, chase missing information, then compile results.

Fixed data and interfaces: 12 minutes – keep existing queries, rules, or reports but eliminate system-switching and record-hunting.

Add shared ontology and supporting services: 9 minutes – centralize shared object relationships, state meanings, and judgment conditions for reuse by relevant applications. The 3-minute difference is the extra improvement that demands further verification.

Both improved approaches must use the same confirmed business definitions, data scope, permissions, time points, and optimization opportunities. The ontology variant must not be given expert interpretations denied to the other side. Real projects rarely separate this cleanly; modeling may have uncovered the address-link errors that triggered data fixes. Document that chain, but without controlled counterfactual evidence, report the whole solution’s benefit rather than inventing a percentage attribution. Also, if verification is slightly slower but markedly reduces misrouting and subsequent rework, the investment may still be justified—evaluate error consequences and added costs together.

End-to-End Acceptance: From Query to Correctly Received Result

A report generated in seconds is useless if the recipient must re-verify everything. Acceptance must span from verification initiation to the result being correctly accepted—not stop at “report generated.” A repeat verification round after the report erases earlier time savings; necessary human review should be retained and counted as input. Whether the customer eventually gets broadband installed is a separate tracking item; do not equate verification completion rate with installation problem resolution rate.

A usable verification result must at minimum identify the correct order and address, use the correct time point, provide an evidenced cause, and know the receiving role. Unresolvable items can be listed as pending verification; but if fully documented cases are still routinely escalated to manual handling, the system has not helped much.

The article provides a detailed “Ontology Project Business Value Acceptance Table” with seven evaluation dimensions:

Task scope: which regions, business types, and observation window; fixed task list including failures, timeouts, and manual takeovers.

Result quality: correctness of object, time point, cause, and receiving role; independent business re-review, sampling accepted results—“no one pushed back” is not proof of correctness.

Manual effort: total time spent searching, cross-checking, correcting, supplementing evidence, and reworking; sum across roles, also record wait time and elapsed time.

Errors and rework: reduction in wrong causes, misrouting, and re-verification; compare returns and re-judgments using the same follow-up observation window.

Follow-up results: whether actions were taken promptly, and whether eligible, still-willing customers were re-installed; track subsequent work orders and verify other influencing factors.

Reuse and maintenance: whether customer service and operations use the same meanings; measure actual usage, change effort, and regression results.

Cost and re-evaluation: construction, operation, review, maintenance spend, and next check date; technical provides records, business confirms results, finance reconciles amounts.

Samples must include normal decidable cases, missing-material cases, and conflicting-state cases, with a hold-out set not used in tuning. Historical work-order cause labels may be wrong; expected results must be validated against original records. Blind review—hiding which solution produced each result while preserving needed business evidence—reduces “new system must be better” bias, though it cannot eliminate personnel and task-difficulty effects on duration. Historical replay must prevent “knowing the answer in advance”: post-hoc cause notes unavailable at the time cannot be fed early. The same analyst testing both tools sequentially may simply remember the answer; use difficulty-matched different tasks or counterbalance order. Cherry-picked complex cases reveal capability gaps but cannot be averaged to project overall business performance. After offline pass, run a controlled pilot recording task difficulty, personnel, and concurrent process changes; a two-month speedup may not be the tool’s credit alone.

Does the Second Integration Really Lower Marginal Cost?

Ontology savings often appear during the next rule change or application integration. Example: both customer service and operations distinguish “resource shortage” from “on-site construction blocked.” When the classification rule changes, previously each team found its own expert and modified code separately; now they confirm the shared definition once, then check their respective mappings and implementations. The expectation is decreasing marginal cost: existing definitions are reused, so the next application avoids some redundant modeling. However, each new integration still adds data mapping, permission adaptation, and version coordination work. Amortizing upfront build cost does not guarantee the next integration is cheaper. The more the new scenario diverges, the less you can promise “the tenth application costs almost nothing.”

To know if effort was truly saved, capture one real change: log expert and engineer hours, what was modified, and whether anything was missed. Without a real change, a controlled drill is acceptable, but do not present drill results as ongoing operational gains. Shared definitions do not auto-propagate to all applications; legacy interfaces, private terminologies, and old versions still need handling, and changes require testing. If ordinary rule services already handle this well, keep using them—no need to swap stacks just to prove the ontology’s worth. “Ten departments will use this later” is an opportunity, not a realized benefit; wait for the second actual integration to see if saved work offsets new adaptation and maintenance.

Saved Hours ≠ Saved Cash

Returning to the example: if quality holds and conditions are comparable, the new approach saves 3 minutes per case. With 1,000 applicable cases per month, that is 50 hours. Where do those 50 hours go? Reduced overtime or reduced variable-contractor spend yields actual cash savings; using them to clear a backlog shows in backlog reduction; simply giving staff breathing room is a valid outcome—record it honestly without forcing a cash conversion. Re-installed orders after cause clarification cannot be wholly credited to the tool; resource availability, construction scheduling, and customer willingness all play roles, and the original process might have recovered some portion.

Total Cost of Ownership (TCO) must cover build through operations, maintenance, and retirement—not just platform license or model inference fees (Microsoft Azure Well-Architected Framework, Cost Optimization principles). At minimum, track three cost segments: upfront expert confirmation, modeling, and data mapping; launch-time interfaces, permissions, rule implementation, and testing; ongoing updates, corrections, and maintenance. Plan for migration or retirement costs. Compare these inputs against consistently scoped, monetizable benefits within the same evaluation period to compute ROI. Financial benefits must trace to amounts and realization evidence; unconverted capacity gains stay separate from cash savings. A common error: evaluating the whole project by comparing total benefits to total costs, but when deciding whether to add ontology capability, compare only the incremental benefits and costs of that addition. Do not take the full project’s 11-minute saving and offset it against only the ontology increment’s cost. Separate cash outlays from internal effort; use the same evaluation period for costs and benefits; rework time already counted in total hours must not be monetized again; shared platform costs need an allocation basis—do not treat them as free for every project.

Acceptance Must Allow “Stop Expanding” as a Valid Outcome

If data and interfaces are fixed and existing services already meet targets stably, further modeling with no measurable improvement can stop—retain the confirmed definitions and tests. If verification is more accurate but downstream handling still waits for people, solve the handoff first; drawing more concepts will not shorten that wait. If shared definitions demonstrably cut duplicate maintenance and runtime holds, then consider extension—each new scenario’s extra cost and headcount still need individual scrutiny. “We’ve spent this much already, might as well invest a bit more” is the sunk cost fallacy (HM Treasury, The Green Book 2026). Past irrecoverable spend stays in the project ledger, but additional budget must be judged on the next increment’s benefits, costs, and risks—not on reluctance to abandon prior investment. For mandatory control requirements, do not drop them because they don’t save money; compare alternative ways to meet the same requirement for reliability and cost-effectiveness.

Summary

Take one real cancellation and trace from verification start to someone acting on the result—this reveals value more clearly than staring at a graph dashboard: which duplicate tasks disappeared, did misrouting and rework drop, and what maintenance cost remains. Graph scale, no matter how large, cannot substitute for vague business metrics. Fix data and interfaces first with simple means; only when shared definitions, complex judgments, or cross-application reuse deliver measurable extra improvement should you weigh the added build and maintenance investment. The next budget request needs that evidence. The next article will discuss readiness conditions: beyond budget, what people, data, and maintenance arrangements are required.