Memory as Governance: Lifecycle, Feedback & Forgetting for Reliable AI Actions

This article is the fourth in the Knowledge-driven Computing (KDC) series. It addresses long-term operation of AI systems: after a correct action completes, which history should influence future actions, and how to prevent systems from amplifying accidental experiences and erroneous feedback.

01. Single Correctness Does Not Equal Long-Term Reliability

A refund agent handles a first request perfectly: it confirms conditions, waits for explicit user consent, checks policies, and completes the refund. The system stores a summary: "Users prefer direct refunds when eligible." Weeks later, the same user asks about another order: "Can this be refunded now? I'm just checking." The memory retrieval finds the previous refund record, and the agent treats the past action as a stable cross-order preference, skipping explanations and pushing toward confirmation. The history was not recorded incorrectly—the user did choose a refund last time. The error is overgeneralizing a single contextual action into a permanent preference that now drives new actions. Remembering more does not guarantee reliability.

Long-term reliability requires a boundary between two failure modes:

Remember nothing of the past <-> let all past permanently influence the future

02. Conventional AI Memory Asks What to Retrieve; KDC Asks How History Takes Effect

Typical context memory asks: "What history should be retrieved into the current context?" KDC's memory runtime asks: "Which history can influence future judgments and actions under what conditions?" For a history segment to enter the future action chain, at least these governance questions must be answered:

When and in what context was it formed?

Does it come from fact, user expression, model summary, or action result?

Is it only applicable to a single task or reusable across scenarios?

Has it been supported by subsequent feedback, or is it a one-off observation?

Does it conflict with new reality, new knowledge, or new user expressions?

Does the current subject have permission to access and use it?

Should it continue to take effect, have its influence reduced, or be kept only for historical explanation?

Vector similarity helps find relevant history but cannot answer these governance questions. Therefore, KDC defines a Memory Object as a long-term context object carrying time, experience, context, feedback, and evolution state, and a Memory Runtime responsible for formation, retrieval, reinforcement, conflict, decay, forgetting, migration, and archival. This remains a theoretical proposal; memory classification, forgetting strategies, and organizational migration lack unified standards, but the direction is clear: memory's core is not saving more history but governing history's influence on the future.

03. Not All History Should Become Memory Objects

Logs, chat records, execution results, and user feedback can be memory sources, but long-term storage does not automatically grant the right to influence the future. Logs answer "what happened"; memory objects must answer "what significance does this history have for future understanding and action." Example: refund_order returns success → execution record

Refund arrives after three days → action result

"This payment channel usually takes longer on holidays" → experience formed after multiple supporting results

"This user always prefers refunds" → likely erroneous memory overgeneralized from a single behavior

Memory formation should have a threshold. A single accidental event can first become a candidate memory, retaining source and applicable context, awaiting later feedback. It should not become a stable portrait or organizational experience just because a model summarized it fluently. Similarly, model outputs cannot be directly written as long-term memory; the system must distinguish between a situational judgment, a candidate preference, and a stable tendency verified by multiple behaviors.

04. Four Memory Types Closer to Action Governance

To ground memory discussion in engineering, KDC proposes four candidate memory types for capability governance:

4.1 Action-Result Memory

Records a capability invocation, real-world result, and error attribution. Example: refund capability rejected by policy because order entered fulfillment; refund API succeeded but reconciliation showed delayed arrival; external notification failed due to permission misconfiguration. This memory directly affects future capability selection and risk judgment, often with strong timeliness—external systems, policies, and capability versions can quickly invalidate it.

4.2 Preference or Portrait Memory

Records preferences and tendencies of users, organizations, or agents within a certain scope. Example: user prioritizes camera when buying phones; a team prefers small-scale canary releases; an approver usually asks for cost breakdowns. Preferences are not permanent labels; they need time, scenario, and evidence boundaries, and must allow user correction, withdrawal, and inspection. A single behavior typically only forms a candidate preference and should not override explicit expressions.

4.3 Decision Precedent Memory

Records historical decisions, their rationale, and subsequent outcomes under similar goals. Example: high-risk refunds historically escalated to manual review; phased delivery mitigated certain project risks; a failure type recovered via degradation instead of retry. Precedents can support future judgment but cannot replace current evidence. Multi-scenario validated precedents may migrate to candidate knowledge, but migration requires re-verification of applicability boundaries.

4.4 Episodic Memory

Records the context of a specific interaction or task, e.g., user focused on battery life in last conversation, or a project meeting temporarily shelved a proposal. This is closest to common dialogue memory and vector retrieval. It helps restore context but should not be mistaken for stable preferences or generalizable experience.

These four types are not a mature unified taxonomy; they are research slices for differentiating governance approaches. The first three directly affect capability invocation, risk, and human confirmation; episodic memory mainly serves context continuity.

05. Memory Must Have a Lifecycle and Must Allow Forgetting

If a memory system only supports write and read, it will eventually fill with outdated reality. KDC memory runtime must consider at least these state transitions:

5.1 Formation

Identify contexts worth long-term retention from knowledge objects, reasoning objects, capability feedback, human decisions, and business events. Formation does not equal stable effect; it can start as a candidate memory.

5.2 Reinforcement

When multiple independent feedbacks continuously support the same experience, increase its credibility or expand its applicability range. Reinforcement is not simply increasing hit counts—repeated use of the same erroneous source may only amplify the error.

5.3 Decay

As time passes, reality changes, or long-term support is absent, reduce the memory's influence on future reasoning. Payment channel processing times, promotional preferences, and project statuses often need faster decay; stable decision principles may decay slower.

5.4 Conflict

When new behavior, new knowledge, or explicit user expression contradicts old memory, explicitly mark the conflict instead of silently choosing one. In the opening case, "I'm just checking" is a current constraint against "user prefers direct refunds" and should block the old memory from automatically driving action.

5.5 Forgetting

Stop a memory from participating in current judgment and action governance. Forgetting does not equal physical deletion; it can manifest as downweighting, expiration, context restriction, requiring re-validation, or migration to a historical area.

5.6 Migration

Move local experience to team or organizational scope, or convert repeatedly stable action results into candidate knowledge. Migration is not copying; it must re-examine evidence and applicability boundaries.

5.7 Archival

Retain audit and historical explanation value but no longer let the memory influence current actions. Historical experience related to old refund policies can help explain the past but should not guide new orders.

In long-running AI systems, forgetting is not a defect but a governance capability. The real danger is not that the system forgets an irrelevant history, but that it cannot stop expired, unauthorized, or erroneous experiences from influencing the future.

06. Feedback Is Not Just a Reward Signal, Nor Automatic Truth

After action completion, the system may receive multiple feedback types:

Tool or API execution results

External system state changes

User confirmation, correction, rejection, or complaint

Human approval results

Rollback, compensation, and exception handling

Subsequent business outcomes

Audit or retrospective conclusions

These feedbacks vary in importance and may contradict each other. An API returning success means the request was processed, not that the user received the refund. User saying "already received" is closer to real outcome but may still involve misunderstanding or correlation errors. Reconciliation system confirming fund completion provides another evidence class.

Therefore, feedback entering the system needs source, timestamp, object association, credibility, and applicability scope. Feedback cannot bypass validation to directly upgrade knowledge or reinforce memory.

A minimal feedback processing chain:

Execution result or reality observation
 -> associate with original goal, reasoning, and capability
 -> judge feedback source and credibility
 -> error attribution or result verification
 -> update reasoning object
 -> form candidate memory or candidate knowledge
 -> corresponding runtime decides reinforcement, downgrade, or archival

Feedback may change different system parts. If refund non-arrival is due to payment channel failure, update action-result memory and capability quality summary. If system cited outdated policy, let relevant knowledge degrade or expire. If agent misinterpreted consultation as authorization, correct goal interpretation and reasoning strategy. If governance policy should have intercepted but didn't, adjust capability policy. Without error attribution, all feedback is crudely attributed to "model quality," leaving the system unable to know whether to fix knowledge, memory, reasoning, capability implementation, or governance policy.

07. Continuous Consistency: Enabling Systems to Detect and Correct Deviations

KDC's long-term goal is not to make AI automatically smarter with use, but to let application software continuously observe deviations between domain reality and digital systems, and retain engineering paths to correct them. This direction is called Continuous Consistency —not absolute synchronization at every moment, but the system's ability to:

Know which realities it covers and which it misses

Detect inconsistencies between digital assertions and reality observations

Identify knowledge that has expired or drifted

Re-associate action results with original judgments

Update knowledge, memory, reasoning, or governance after deviations appear

Current KDC research proposes five candidate measurement directions:

Reality Coverage – How much of the important reality elements within system responsibility are covered?

Reality Mapping – Are digital assertions still consistent with reality observations?

Knowledge Freshness – How long since the currently used knowledge was last validated?

Knowledge Drift – In what ways is knowledge content or validity changing?

Consistency Verification – Can important actions be traced from goals and rationale to reality feedback?

These directions are currently Derived + Open Research. Candidate operational definitions and minimal signals exist, but how to enumerate reality domains, determine ground truth, set timeliness thresholds, calculate drift, and enforce audit chain integrity still require practical validation. Especially, they should not be prematurely combined into a seemingly precise "system consistency score"—a single score may mask local high-risk gaps. Early practice should start from collectible signals: how long active knowledge has gone unvalidated, how many high-risk actions lack reality feedback, how many failures trace to expired knowledge, how many memories remain in use after conflicts.

08. Not Just Compliance Logging, But Production Capability

Audit and compliance are the baseline of feedback loops, not their full value. When a system can associate goals, knowledge, memory, reasoning, capabilities, policies, and reality feedback, teams can more concretely answer:

Which knowledge categories most often cause erroneous actions

Which user preferences have expired

Which capabilities are frequently selected in wrong contexts

Which governance policies are too strict, too loose, or trigger excessive human intervention

Which historical experiences are worth migrating, which should only be archived

Whether the system is repeating the same type of failure

These capabilities directly affect production value. Teams can automate high-value processes with more confidence, and downgrade or hand off to humans when risk rises. Organizational experience no longer lives only in postmortems and expert memory but can be reused with preserved boundaries. However, restraint remains necessary. Feedback loops provide correctable conditions, not guarantees of correct improvement. Erroneous feedback, erroneous attribution, and erroneous reinforcement can equally make the system drift further. The system must allow questioning its own memory and retain boundaries for human correction and stopping automatic evolution.

09. Relationship with Logs, Traces, and Traditional Observability

Logging, APM, distributed tracing, audit logs, and LLMOps metrics provide necessary signals for long-term operation: events, performance, call paths, errors, and model usage. KDC does not replace these systems; runtime observability reuses their data.

The difference: traditional observability excels at answering:

Which service failed?

How long did the request take?

Which tool was called?

How many tokens did the model consume?

KDC additionally needs to answer:

Which knowledge version was referenced by this action?

Which memory segment influenced the judgment?

What conclusions and risks did the reasoning object produce?

Why did the policy allow or deny?

Did reality feedback verify the original judgment?

To which object, runtime, or owner should the error be attributed?

Thus, KDC runtime observability is not a new log product but requires existing signals to form audit chains through stable object identities and causal relationships. Observability systems provide evidence; they should not overstep to directly modify knowledge or memory. Object state changes remain the responsibility of respective runtimes and governance processes.

10. Practice: Establish a Feedback Contract for an Important Action

Select an important AI action already in or entering production. Don't design a unified metrics platform first; define how this action will be verified by reality. Start with six questions:

Which real-world outcome verifies action success?

Who generates feedback, and within what time window?

How does feedback associate with the original goal, reasoning object, and capability invocation?

When multiple feedbacks conflict, who has authority to confirm or correct?

Which results should form memories, and when should they decay, be forgotten, or archived?

Which stable experiences can upgrade to candidate knowledge, and which must remain individual cases?

Then create a minimal Feedback Contract. Example for a refund scenario:

Business Goal: User explicitly confirms and completes order refund

Execution Result: Refund request accepted by payment channel

Real Success Condition: Funds arrive, or reconciliation system confirms refund completion

Feedback Sources: Payment channel status, reconciliation results, user confirmation

Association Keys: Order, original payment, refund request, reasoning object, capability invocation

Feedback Time Limit: Within 72 hours after acceptance; timeout enters exception handling

Credibility & Conflict: Reconciliation result takes priority; user feedback triggers verification; API success ≠ funds arrived

Error Attribution: Knowledge, goal interpretation, reasoning, governance, capability execution, external channel

Memory Formation: Save channel processing duration, failure reasons, and manual intervention results

Decay & Forgetting: Decay by channel version and policy cycle; stop influencing new actions after rule changes

Knowledge Upgrade: Multi-verified stable processing patterns enter Candidate Knowledge

Minimal Observability: Feedback loop closure rate, arrival timeout, expired knowledge references, error attribution completeness rate

The core of a Feedback Contract is clarifying the distinction between "execution completed" and "real goal achieved," and specifying how feedback changes the future. The first version doesn't need a composite score. As long as the system can stably associate an action's goals, rationale, execution, and results, and knows whether feedback should enter knowledge, memory, reasoning, or governance, the minimal loop is established.

11. Reliable Systems Don't Remember Everything—They Know What Should Still Influence Now

The long-term value of an AI system depends not on how much historical text it accumulates, but on whether it can correctly govern history's influence. Memory needs formation thresholds, context and permissions, reinforcement and decay, as well as conflict, forgetting, and archival. Feedback needs source and credibility, association with original action, and error attribution. Continuous consistency needs observability and verifiability, but cannot be packaged into a unified formula unsupported by practice.

Therefore:

Saving history ≠ forming memory
Retrieving the past ≠ the past still applies
Receiving feedback ≠ obtaining truth
Accumulating more ≠ continuously improving

KDC does not study how AI systems automatically get better with use. It aims to provide a different capability: when reality changes, knowledge expires, memory fails, or actions fail, the system can discover deviations, explain deviations, and know what to correct. By now, Reality, Representation, Knowledge, Reasoning, Skill, Capability, Memory, and Feedback have each appeared. The final question—how these objects compose a complete, incrementally adoptable KDC software engineering model through runtimes and governance mechanisms—will be the topic of the fifth article.

12. Theoretical Boundaries and Open Problems

The memory objects and memory runtime in this article derive from KDC's current theoretical stage, overall maturity Hypothesis + Derived. Memory formation, context binding, reinforcement, and archival have relatively clear derivation bases. Unified forgetting strategy, Memory Taxonomy, organizational migration, and memory credibility calculation remain open problems. The five continuous consistency measurement directions are currently Derived + Open Research, not yet mature standards.