When Your Internal AI Is Fed Bad Data, How to Fix It?

The author describes a workplace incident: a new employee presented an AI‑generated SOP that referenced a 23‑year‑old policy, leading to an incorrect fine standard. Investigation revealed that the company had dumped five years of historical files into the AI knowledge base to make the model “smarter,” without any version control or data isolation.

Why More Data Became Toxic

The author initially fell into a “data hoarding” mindset, assuming that feeding all past documents, chat logs, and old SOPs into a Retrieval‑Augmented Generation (RAG) system would make the AI omniscient. The reality is simple: AI mirrors its input—garbage in, garbage out. Without version locks, random sampling, or contamination isolation, the knowledge base becomes a ticking time bomb.

Shift to Subtraction: A Three‑Step Protocol

To turn the knowledge base from a trash dump into a reliable source, the author adopted a subtraction approach, establishing a whitelist, locking effective dates, and creating isolation zones.

Step 1 – Corpus Cleaning Checklist

Target audience: knowledge‑base administrators / content operators.

Input locations: corporate WeChat, Feishu knowledge‑base backend, local Excel files.

Action: run a full scan on the 1st of each month, tag results, and archive expired items.

Checklist items:

Mark all policies and process documents with effective/expiry dates.

Move historical versions to an “_ARCHIVE” folder and block AI retrieval.

Delete internal complaints, unresolved drafts, and purely emotional records.

Replace core SOPs with the current version (V + sign‑off).

Step 2 – Knowledge‑Base Routing and Isolation Rules Target audience: automation platform / AI application configuration backend. Input locations: vector‑database permission page, platform routing configuration. Action: layer data by sensitivity, set retrieval weights, force external requests to the low‑sensitivity zone. Three zones:

Green (public) : open policies, standard scripts, product manuals; fully open to AI; real‑time sync with business.

Yellow (internal) : process SOPs, case libraries, historical contract templates; accessible to internal staff after approval; reviewed monthly and version‑locked.

Red (isolated) : salary data, unreleased strategies, personal evaluations; AI access prohibited; physical isolation or manual review only.

Step 3 – Automated Contamination Scan Prompts Target audience: AI large‑model operators. Input location: chat dialog / batch‑run script. Action: feed the document list to the AI, export a red‑flag report, and manually verify deletions. Prompt examples (in red text) ask the AI to: Mark documents older than one year without an “effective now” label. Identify contradictory content or inconsistent data points. Detect pure meeting minutes, drafts, or chat logs with no conclusions. Output: a “Discard/Archive Recommendation Table” with tags for keep, isolate, or delete.

Key Purposes and Pitfalls

The overall goal is to control source‑level quality, reduce erroneous references, and dramatically shorten onboarding time for new staff. Absolute no‑go zones include directly linking the original shared drive without processing and performing full automatic deletions without human review.

Common rookie mistakes: treating a one‑time clean‑up as sufficient, or configuring rules without testing. The author advises using an external test account or red‑team simulation to verify that the AI cannot cross into prohibited zones.

Reflective Question

When a knowledge base turns into a garbage dump, is your value in “feeding more” or in “filtering precisely”? The 2026 data‑governance mantra is not about storing everything, but about daring to delete what’s harmful.