Organizing Maintenance Knowledge: Start from a Concrete Work Order, Not Document Piles

Scenario: A Pump Leaks Again After Repair

An equipment enterprise builds a maintenance knowledge assistant. Hundreds of manuals, regulations, and historical work orders are ingested, classified, chunked, and indexed. A frontline maintainer asks: "The pump seal was replaced, the work order closed, why is it leaking again?" To trace the cause, the maintainer needs to know what verification was done after repair and who confirmed restart. The assistant finds the seal replacement steps but cannot answer what post-repair verifications were performed or who authorized restart. The manual tells how to repair; the work order only says "closed" — neither captures the field judgment at the time.

Don't Let 'Complete Materials' Be a Prerequisite

Asking the business for "all knowledge" upfront yields little. Instead, start with a single concrete work order. Narrow the scope to "verification basis for restart after repair." Identify the equipment and repair instance, then ask: does work order closure mean the repair is done or that required verifications are complete? Who confirmed restart, and where is it recorded? This tells you what to look for. A ten-year-old training deck may be irrelevant; a short verification record may fill the gap. Large language models can locate relevant passages and normalize terminology, but they must return the file version and original location so humans can verify. A fluent summary alone does not reveal which statements are trustworthy.

Regulations, Records, and Experience — Don't Merge Them into One 'Knowledge' Type

Suppose the applicable procedure at the time required specified verifications and confirmation by a responsible person before restart. First verify equipment, job type, and effective date; do not judge past actions with today's newer procedure. The historical work order only states "seal replaced, work order closed" with no verification attachment. The master recalls: "We rushed production, later checked on site again." These three sources answer different questions: the procedure states "what should have been done," the work order records "what the system logged," the uncertain recall hints "where to supplement investigation." "Work order closed" does not prove verification was completed; "should have checked again" cannot be written as "re-verified." Conversely, a missing attachment does not prove verification was skipped. Other accessible records must be checked and personnel consulted; if still unclear, record "verification basis not yet found." Preserve provenance for later review. W3C PROV uses entities, activities, and agents to describe data origins for quality and trustworthiness assessment; it explains how content was derived, not that the content is correct.

Don't Just Ask 'What Experience' — Ask Why They Judged That Way

Asking "what to watch for when repairing a pump" yields a generic checklist. Better: with the work order in hand, ask "On what basis did you decide restart was acceptable? What did you hesitate on?" Let the expert reconstruct the process; do not assume missed checks or delayed restart. Example: the master says "Static test showed no leak, but operating conditions change." Follow up: what phenomena were observed? Which were firsthand, which inferred? Which records were consulted? Would the judgment differ under different conditions? Do not force a generic rule now. Record equipment model, repair location, and job conditions; applicability is later confirmed by domain professionals. Also ask for two experiences with different outcomes: one passed confirmation, one did not. Compare differences: conditions, evidence, procedure requirements, or unapproved temporary measures. Past practice does not automatically become a future rule. A previous article discussed the distinction between contributing experience and transferring responsibility; the same applies here: frontline staff provide clues, not sole responsibility for promoting them to formal rules.

After Interviews, Leave Traceable Records

The master says "this situation needs another look"; the organized draft must not turn that into "all seal replacements must do the same test." He flagged a condition; the project must clarify that condition and its scope. First produce a "business judgment basis checklist." The table below illustrates the fields (actual projects must include material locations and responsible persons):

Question to support: Can the verification basis for this repair's restart be traced?

Judgment to verify: Work order closed, but cannot confirm required verification completed.

Regulatory basis: Procedure version, clause location, and effective date applicable at time of repair.

System records: Equipment, repair task, closure record, verification and confirmation records, with occurrence and entry timestamps.

Experience clues: Master flagged operating condition differences; specific conditions need professional verification.

Current gap: Verification basis corresponding to this repair not yet found; further investigation needed.

Scenarios to investigate: If attachments found, check whether they cover this repair and this equipment; these are investigation items, not confirmed facts.

Confirmation arrangement: Procedure owner verifies requirements, record owner verifies facts; unresolved issues assigned to a follow-up owner.

Disposition and use: Unconfirmed content only for supplementary investigation; confirmed definitions used for modeling, facts still linked to original records.

Original words, organized draft, and subsequent revisions must remain cross-referenceable. Changing "sometimes" to "every time" or "suggest" to "must" alters business meaning. Such changes must go back for confirmation, not slip in during polishing. If two people disagree, first check whether they refer to the same equipment, same procedure version, same job type. Different conditions → record separately. Same conditions but persistent disagreement → escalate to authorized decision-maker; do not let the model compromise with a sentence nobody said.

Organized, But Don't Stuff Everything into the Ontology

Objects such as equipment, repair tasks, verification records — their definitions and relationships belong in the business model. Using OPM (Object-Process Methodology) can express state changes across repair, verification, and restart, checking for missing confirmations. Detailed steps remain managed by procedures or workflows; individual verification results stay in business records; unconfirmed experience remains pending verification. Complex cases can serve as retrieval material or test cases for the assistant. Do not break the entire manual into nodes. Clarify which procedure applies to which task, which record corresponds to which verification, and what basis the current judgment lacks — this helps the application gather scattered materials. Unconfirmed experience entering the search index must have usage restricted by state: it can prompt supplementary investigation, but must not be treated as a current requirement. Search results should return material type, confirmation status, applicability scope, and source location. When answering regulatory questions, only use confirmed and applicable bases. Writing "for reference only" in the document but not distinguishing during query and judgment still risks misuse.

Done Organizing — Test with a Few Work Orders

First select work orders with complete bases, then add cases with missing attachments, wrong associations, inapplicable procedure versions, and conflicting personnel accounts. Business personnel confirm expected results from original materials, including which questions truly cannot be judged. Then evaluate whether the assistant can explain its basis and identify gaps. Example output: "Found closure record, but missing verification basis corresponding to this repair; needs record owner to supplement investigation." This is more useful than a generic "consult an expert" and does not assume verification was definitely not done. Also hold out a batch of work orders not used in organization or debugging; provide raw materials but not pre-written answers or retrospective conclusions. Answering two discussed cases well does not prove a new case will work. When a new work order is answered incorrectly, find the root cause: procedure changed, records incomplete, or experience from one equipment applied to another. Do not add a special-case rule for each exception, ending with conflicting rules. Whether this round of organization is sufficient depends on whether cases in the agreed scope can be verified as expected, gaps clearly stated, and confirmation bases traceable. If yes, deliver this part; if key gaps for certain task types remain, keep restrictions and do not rush the assistant to give affirmative answers.

Summary

Returning to the leaking pump: this round need not cover all fault diagnosis. Being able to verify what procedure was followed, what verification and confirmation records exist, and what questions need follow-up is closer to field needs than stuffing dozens of generic manuals. Materials organized, AI generated an ontology draft; what should experts review next? The next article discusses how to review without redoing manual modeling from scratch.

Reference

W3C, PROV-Overview, Abstract and Section 1. Used to illustrate that provenance information can record entities, activities, and agents; the article's knowledge checklist, repair cases, and confirmation processes are designed by the author. https://www.w3.org/TR/prov-overview/