Why Saved Chats Don't Enable Reuse: Preserving Verification Methods Over Conclusions

Separate Facts from Conclusions for Reusable Experience

In a bearing replenishment scenario, a purchaser noted that a specific purchase order would arrive next week and should not count for this week's replenishment. The assistant recalculated correctly. Days later, the same supplier appeared in another query, and the assistant excluded all of that supplier's unfilled orders—turning a single-order, date-specific exception into a blanket supplier rule. The context (which order, which date range, which rule triggered the exclusion) was lost.

Chat logs show what was asked, how the model answered, and how the purchaser corrected it. But they do not distinguish verified facts from guesses, or later-discovered errors. For each correction, at least these must be recorded separately: which purchase order had which expected arrival date, which replenishment period was being calculated, which rule was used to exclude the order, and what check should be performed next time. Relative terms like "this week" must be anchored to the actual calendar date at that time.

Expected arrival dates change; the reusable reminder is not "exclude this supplier" but "when calculating unfilled quantities, do not rely on a saved date—query the current expected arrival date for that order and compare it with the current period." This does not create a new replenishment policy; it exposes that stale data was being reused. The fix is to correct the query and check logic, then retain the case to remind maintainers not to miss such changes.

The article calls this "ontological memory": mapping content to specific purchase orders, tasks, rules, sources, and applicability scopes. It can be stored in existing record stores without turning every sentence into a graph node. The focus is on application-level save, search, and use of experience—not model retraining.

Preserving "why this judgment was made" is more useful than preserving "don't count this supplier."

Retain the original judgment's conditions; reuse the verification method; do not expand into a fixed supplier rule. "This week" must map to the actual date at that time.

One Correction: First Return to the Source for Verification

Assume the assistant used a target of 300 units, available stock of 120, and 120 unfilled purchase units from prior records, suggesting a replenishment of 60. The purchaser then pointed out that the corresponding purchase order's expected date had changed, making the old data unusable.

First verify: which order, which line item, how many unfilled units, whether the source system's date has been updated, then recalculate per the rule. The purchaser's reminder is a lead for re-verification, not a one-sentence overwrite of the original record. If the source system hasn't been updated yet, assign the responsible person to verify; unverified quantities cannot be treated as confirmed supply nor arbitrarily zeroed out.

Place the old input, the purchaser's reminder, the newly found evidence, and the revised result under the same task. Keep the old 60-unit suggestion in history, annotated with its basis and which version of the suggestion replaced it. If the final purchase content changes, follow the agreed re-check and approval process; if a purchase order has already been created, it must still go through the original verification and change process—not simply create a new one after adjusting the suggestion.

W3C's PROV documentation describes provenance using entities, activities, and agents. Applied here, it enables tracing which purchase record was used, how it was processed, and who verified it. PROV helps people trace origins; it does not automatically guarantee the experience is correct.

Treat the reminder as a re-verification lead; recalculate after verifying the source record; retain before/after bases and check approval; existing purchase orders still follow the original verification and change flow—do not just create a new order after fixing the suggestion.

Experience Prepared for Others Must Be Reviewed

Original dialogues can remain as task records, but not every segment needs to become a reusable experience. Prioritize those that genuinely exposed a problem and may recur. The model can draft a candidate, attaching the original text and linked records, then hand it to someone familiar with the business for validation.

In this example, the person responsible for the purchase order verifies the date and quantity; the purchasing rule owner confirms which business scenarios the reminder applies to. If the draft writes "all purchase quantities from this supplier are excluded," it should be rejected. Unconfirmed content serves only as a re-verification lead, not as a formal basis for current calculation or processing.

The following table illustrates the experience record for "check expected arrival time."

Original Problem: Which unfilled purchase quantities within this replenishment period can be counted?

Experience Content: Re-query the current expected arrival date of the order and compare it with the current period; do not reuse the old date.

Source: Specific task, original dialogue, purchase order line items, applicable rule at the time, and correction records.

Applicable Scope: Products and warehouses that use this period-filtering rule, plus the rule's effective time range.

Cases Where It Must Not Be Applied Blindly: New orders that satisfy the period and other inclusion criteria cannot be excluded just because the supplier is the same.

Version & Confirmation: Experience ID and version, status (candidate/confirmed), confirmer, and confirmation timestamp.

How to Use Next Time: Prompt to verify the date as an aid; calculation still uses the current query results and current rules.

When to Re-review: When arrival rules, source fields, or related processes change.

Access & Maintenance: Access conditions for source and summary; persons responsible for revision, deactivation, and re-testing.

Experiences with unclear scope stay pending verification; they do not automatically become formal policy just because they appear multiple times. A one-time approved exception must be linked to that specific approval, not become a default rule.

Checks that are mandatory must be written into the query or calculation code. Storing "must check date" in an experience library and hoping the model retrieves it every time is insufficient. Even if the experience is not retrieved, the check must still execute.

Experience records require completion and confirmation—especially source and scope. Mandatory date checks belong in code, not reliant on model retrieval.

After Deactivation, Can Old Content Still Be Retrieved?

Suppose the company changes the purchase quantity inclusion rule; the old experience no longer applies to new business. Simply clicking "deactivate" in the admin UI is not enough: search indexes may hold old versions, caches may hold summaries, and ongoing sessions may have already read it.

Experiences and summaries must be traceable to their versions. During retrieval and use, the program must check whether the version is still approved and whether the scope matches—not just retrieve by text similarity. Indexes, caches, and related tool documentation must be updated synchronously; until updates complete, the system must be able to block deactivated versions. If an in-progress task already used a deactivated experience, arrange recalculation, re-review, or pause based on impact.

For historical audits, it is acceptable to show "this experience was used at that time" within the approved scope. But for answering "what to do today," it must not be treated as current basis. If a review reveals the old judgment was wrong, add a correction note preserving the before/after relationship—do not silently rewrite history to make it look like it was always correct.

Permissions must also be enforced. Price data or customer information that a user cannot read today must not become accessible via an experience summary. Restricting the original document while still feeding detailed summaries to the model does not secure the information.

Decide which records are kept for audit, which expire and are deleted, and which are only stopped from use in new tasks—based on purpose and organizational policy. Deactivating an experience does not require deleting all history; keeping history does not mean allowing it to participate in current judgments.

Deactivation must restrict subsequent retrieval and use, and re-verify affected tasks. Permission to audit history does not equal current usability; both sources and summaries must enforce access control.

Test Reuse by Changing Conditions

Asking the original question again and getting a correct answer does not reveal whether the model misapplied the scope. More useful tests: change the order, change the date, or deactivate the experience first, then observe the response.

Same rule, different purchase order: Query the current date and status for that order, then form a judgment.

New order from same supplier meets period and other inclusion criteria: Count it normally; do not carry over the previous exclusion conclusion.

Testing authorized for Warehouse 2, which uses a different rule set: Select the locally applicable rule; do not apply Warehouse 1's experience.

Original experience deactivated but cache or old session still has content: Do not use for new judgments; re-review affected tasks per agreement.

Source still unconfirmed: Treat as a re-verification lead, not as established fact.

User lacks permission to read the source: Restrict retrieval and output; do not bypass via summary.

Experience not retrieved at all: Still perform the mandatory date check per current replenishment rules; do not skip required steps.

During testing, record which experience version was retrieved, which queries were invoked, which rule version was used, and the final calculation and explanation. A model saying "referenced history" is opaque; cross-check tool logs and results to ensure no eligible order was wrongly excluded.

Compare outcomes with and without the experience, using identical inputs, rules, and tools—keeping all mandatory checks in both arms. Do not remove existing validations to exaggerate the memory's effect. Include the time spent on review and maintenance; if the experience merely duplicates an existing rule, it may not be worth maintaining separately.

Facts that change (inventory, dates) must still be queried from the source system; experience is better suited for capturing "where mistakes easily happen and how to verify next time."

Two groups use the same rules and tools, both keep mandatory checks, then compare results under changed conditions. Cross-warehouse testing requires authorization; record results after actual runs.

Summary

"Don't count this for this week" must be saved together with that specific order and date. Next time the same supplier appears, still verify the new order's arrival date; reuse the verification method, not the exclusion conclusion.

Whether experience works is judged by whether the assistant can still judge correctly based on current evidence when the order or period changes. Merely repeating the old answer more fluently does not count as reuse.

Warehouse 1 has codified these experiences; what differences will Warehouse 2 encounter when trying to adopt them directly? The next article discusses cross-department reuse and change release.

References

W3C, PROV-Overview, Section 1. Used to illustrate that provenance information can link entities, activities, and agents. The experience storage, review, deactivation, and test checklists in this article are author-designed; PROV does not automatically execute these management processes.

https://www.w3.org/TR/prov-overview/