Skip to content
Wrong Candidate List? Don't Rush to Change the Ontology — First Pinpoint the Root Cause

Wrong Candidate List? Don't Rush to Change the Ontology — First Pinpoint the Root Cause

This article presents a systematic methodology for diagnosing incorrect candidate lists in data-driven systems, emphasizing evidence-based root cause analysis over premature ontology changes. It covers distinguishing existing requirements from new use cases, preserving original request context, verifying identity and time validity, tracing errors through data, mapping, rules, queries, and output, and only modifying the ontology when necessary distinctions cannot be expressed.

Data Bricklaying Diary
Data Bricklaying Diary
Data Bricklaying Diary

First Ask: Why Do You Think This Result Is Wrong?

Suppose an equipment assistant produces a candidate list. A scheduler glances at it: "This crane is still under maintenance, how did it get listed?" Someone suggests adding an exclusion rule, another says the status data is inaccurate, yet another wants to try a different model. But the team may have already defined "exclude under maintenance". Adding another rule may not hit the actual error.

The previous article examined missed business content from three perspectives: task, world, and behavior. This article continues by investigating the result: the list is wrong, where exactly is the error, and who should fix it? Below we still use the hypothetical equipment case; the troubleshooting approach is my suggestion, not an actual project fault record.

When someone says the result is wrong, first ask: what should the result have been, and why? Then check whether that was an agreed-upon use case. If the original agreement was to exclude equipment under maintenance, but the result includes it, investigate the error. If the original scope only supported historical record queries, but the user now wants to judge whether a crane can be dispatched tomorrow morning, that is a new use case requiring separate evaluation. If both parties never clarified whether "maintenance complete" equals "restored dispatch eligibility", first confirm the business meaning — do not let the program decide for people.

This step is also needed for open-ended exploration. For example, if research shifts to person-level analysis while the old model only supports brand-level, that may not mean the original model was wrong; it may need extended expressiveness. But if the system already promised to distinguish first-person evaluations from retellings, yet the output still mixes them, then check where the original definition and implementation diverged.

Only after distinguishing these situations do you know whether to hunt for a bug, debate semantics, or add a use case. If it's temporarily unclear, mark it "pending verification" and avoid premature classification.

First Preserve That Query, Don't Rush to Refresh and Re-ask

Next, retrieve what was queried, what was returned, and the feedback basis — all together.

Which equipment, which job, what time was the status checked? What records were fetched, from where? Which version of definitions, mappings, and code did the service use? Are the raw query result and the assistant's final answer consistent? Map these to the same request as much as possible.

Figure 05-01: Preserve the Erroneous Occurrence First
Figure 05-01: Preserve the Erroneous Occurrence First

The inputs, processing basis, and outputs of the same request should correspond; a refreshed new result cannot replace the old error record.

Keeping only a screenshot of "equipment available" may later leave you unable to tell whether the statement came from the filtering service or the assistant overstated "maintenance check passed". Asking again may yield changed status and version; the new result cannot replace the original error record.

The scheduler's claim that the equipment is still under maintenance must also be verified against corresponding evidence — do not change the formal definition on a single verbal feedback. During verification, mark the result as pending, stop using it as a basis for dispatch approval; if tasks were already assigned based on it, record the impact and hand off to someone to handle.

For this example we agree: within the check window, equipment with confirmed identity, trusted source, valid status, and no unresolved conflicts that is under maintenance should be excluded; equipment with unconfirmed identity or source, missing/expired/conflicted status is listed as pending verification. Passing the maintenance status check still does not equal dispatch permission. Real projects need separate evidence for status meaning, source, and validity.

First Verify the Object and Time, Then Trace Which Step Was Not Followed

Is the equipment in the list the same as the one in the maintenance record?

Both the warehouse system and the rental system show local ID "A-01"; that alone does not prove they are the same equipment. First verify the ID scope and identity mapping. If the current model can already represent these scopes but the linkage is wrong, fix the identity mapping first; only if the model truly cannot express the necessary distinction should you assess a model gap.

Time works the same way. Suppose a "not under maintenance" record expires at 8:00 AM; fetching it again at 9:00 AM does not make it valid for another hour. You must verify the time the status applies to, not just the time the interface returned it.

Figure 05-02: A Just-Fetched Record May Already Be Expired
Figure 05-02: A Just-Fetched Record May Already Be Expired

Re-reading an old status does not extend its validity; the applicable time of the status must be verified.

Missing status is another case. It only means there isn't enough record to support a judgment — it cannot be automatically treated as normal, nor can you conclude the equipment is under maintenance. Investigate whether the gap is in the source records, data ingestion, or query omission, then supplement per agreement.

Only after equipment and time match should you check how the rule was applied. Otherwise the program may follow the rule perfectly but process the wrong equipment or yesterday's status.

The Same Erroneous List May Require Fixes in Different Places

The same complaint "list is wrong" may require changes in completely different places:

Situation: Cannot find status — Evidence needed: source records, ingestion results, query conditions for the equipment. Priority: supplement instances per verified evidence, or fix data supply/query; keep pending verification.

Situation: Same ID, unclear identity mapping — Evidence needed: ID owning systems and equipment identity mapping. Priority: identity verification and mapping.

Situation: Conflicting valid statuses at same time point — Evidence needed: both original records, sources, and existing conflict-handling agreements. Priority: fact verification or supplement conflict handling; do not arbitrarily pick one.

Situation: Status expired but service still treats as normal — Evidence needed: validity period, fetch time, actual judgment input. Priority: time-effectiveness handling and implementation.

Situation: Confirmed distinct statuses merged into one in current definition — Evidence needed: business confirmation, existing definition, examples of indistinguishable cases. Priority: evaluate ontology definition or relationship modification.

Situation: Confirmed exclusion condition missing in target rule — Evidence needed: business definition, transformation input, generated result. Priority: expression transformation or knowledge compilation implementation.

Situation: Condition exists and input obtained, but filtering path didn't use it — Evidence needed: applicability conditions, same-request input, actual processing path. Priority: query or filtering implementation.

Situation: Condition generated but service didn't adopt it — Evidence needed: actual running version, call path, corresponding output. Priority: service integration, configuration, or version adoption.

Situation: Service marked pending verification, but assistant answered dispatch allowed — Evidence needed: raw service result and generated output. Priority: output expression and application judgment.

The "knowledge compilation" in the table means converting confirmed business meanings into expressions and rules usable by the target platform. Only projects with this step need to check this link.

This table selects investigation entry points; it does not require tracing every feedback from start to finish. Several causes may coexist; those without evidence remain pending.

Figure 05-03: The Same Error Symptom May Require Changes in Different Places
Figure 05-03: The Same Error Symptom May Require Changes in Different Places

The erroneous list provides a starting point for investigation; the specific cause and fix location still need evidence confirmation.

For example, tracing along the same request confirms the equipment identity is correct, the maintenance status is valid, the existing exclusion rule applies. The service received this status but the filtering step didn't use the exclusion condition. Then fix the query; the ontology stays untouched for now. If you only know the source system has a maintenance record but don't know whether the service received it, you cannot skip the intermediate steps and directly blame the query.

You can trace step by step: does the status in the source record survive mapping? After the service receives it, does it process per applicability conditions? Does the service's raw result change meaning by the time it reaches the assistant? Mark any segment without records as "not yet traced" — don't guess the cause. Finding one error doesn't mean others are cleared.

Similarly, a new file in the release catalog is just a clue. What the service actually adopted must be verified separately. The version check in the accompanying example only compares fictional fields and cannot replace real service verification.

Figure 05-04: Along the Same Request, Trace Where the Status Was Not Used
Figure 05-04: Along the Same Request, Trace Where the Status Was Not Used

In this example, after verifying identity, status evidence, and rule applicability, step-by-step comparison revealed the query did not use the existing condition; the fix was then localized to the query implementation.

What Evidence Justifies Modifying the Ontology

To discuss ontology modification, you must at least clarify:

What question needs answering now, what distinction is required, and why the current definition cannot express it.

For instance, if the business has clearly distinguished "maintenance job complete" from "re-inspection passed, dispatch eligibility restored", but the current model records both as the same state, and queries and data cannot preserve this distinction, then there is reason to propose definition or relationship changes.

If the enterprise hasn't decided how to handle the two phases, hand the business question to the authorized decision-maker — do not let technicians or LLMs decide for them. If the existing model already distinguishes them but a source table merged fields during mapping, fix the mapping.

Figure 05-05: Inability to Express vs. Lost Mapping Are Two Different Things
Figure 05-05: Inability to Express vs. Lost Mapping Are Two Different Things

A needed distinction missing from the definition, and an existing distinction lost in implementation, should be handled separately.

Revisiting the earlier public-opinion example: an account retweets "brand service is terrible", and the system counts it as the author's own negative review. The model indeed lacks expression for attribution and quotation relations — propose a model change. If the relation already exists but was lost during extraction or query, fix the implementation. When research shifts to person-level, first check if the original model can express it; if adding instances suffices, do that first; if extension is needed, state the new problem and adoption rationale, and check impact on existing capabilities.

New data batches, high-frequency new terms, user downvotes — all deserve attention but cannot directly justify definition changes. Bring out the specific gaps and impacts, discuss them, and see if existing concepts can be reused — don't create a new concept just for a wording change.

Some Problems Cannot Be Solved by Modifying the Ontology

Assume the list passes the maintenance status check but the equipment still fails to arrive on time. First verify the current commitment: if the list is only responsible for maintenance status, you cannot directly record the no-show as a filtering error of this item; if the commitment includes dispatch arrangement, then check whether deadline, location evidence, or transfer handover were missed.

This relates to the three perspectives from the previous article. The cause may be that task requirements weren't passed down, reality assumptions didn't hold, or no one took ownership of the arrangement. Don't lower the original arrival requirement or rename "generate list" to "task completed" just to make the result look successful.

Another case is "no answer found". The user may lack permission to view the maintenance record, so the service correctly refuses to return it. Distinguish between no permission, no data, and model inexpressibility — don't open access just to improve hit rates.

The accompanying minimal run example checks preset inputs for missing status, unconfirmed identity, conflicts, expiration, and normal controls. It does not implement automatic diagnosis nor verify real sources; the program returning expected results does not prove the real project's cause has been found.

What Should Be Handed to the Next Step Is an Evidence-Based Handling Proposal

After locating the fix position, continue using the original issue record to supplement verification results — no need to open a new file.

Expected & Basis: This check agreed to exclude valid under-maintenance equipment; list applicable rule and source; belongs to existing requirement verification, not new dispatch use case.

Phenomenon & Scope: A maintenance status filtering included an equipment that should have been excluded; record the request, object, and check time point.

Verified Evidence: Identity, valid status, applicable rule, actual service input and filtering path; attach corresponding record locations.

Judgment & Proposal: Existing definition can express the requirement; propose fixing the query, ontology unchanged for now; list affected paths.

Unknowns & Limits: Whether other requests are affected remains to be verified; before verification, do not use related results as dispatch basis.

Confirmation & Handoff: Business owner verifies rule and scope, data owner verifies records, technical owner handles implementation; assign to specific persons.

Follow-up Re-verification: Original error should be excluded; missing, conflict, and expiration still pending; valid "not under maintenance" controls pass only this item; separately verify actual adoption.

Figure 05-06: The Attribution Deliverable Is a Handling Proposal
Figure 05-06: The Attribution Deliverable Is a Handling Proposal

Verified causes support concrete handling proposals, but a proposal does not mean the fix is effective.

If not yet clear, write "pending verification" with what's missing and who continues. Reproducing the error only shows it's reproducible; issuing a proposal still requires someone to implement and re-verify.

If investigation reveals a new use case, separately record why extension is needed, what materials to supplement, and who confirms. If business meaning isn't settled, find the authorized person. Don't close feedback by writing "modify ontology" for everything, and don't write unverified items as already fixed.

For that under-maintenance crane, the next step should be concrete: "verify which identity mapping", "fix which query condition", or "confirm which status definition". Then business and technical people know what each owns, instead of both receiving "improve the ontology some more".

After the handling proposal is set, how to confirm the change is reasonable, existing capabilities unharmed, and the business actually uses the new version? The next article continues that discussion.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

data qualityroot cause analysisontologyknowledge representationevidence-based diagnosisdebugging methodologyidentity mappingtime validity
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.