Why Knowledge Bases Alone Won’t Make AI Agents Effective: The Need for Actionable Experience

Large models now possess impressive language understanding and factual knowledge, yet when deployed in real‑world scenarios they often cannot complete specific tasks. The core contradiction has shifted from "whether knowledge exists" to "whether knowledge can be turned into action". This demands a new form of knowledge engineering that focuses on actionable experience.

1. Skill as Compressed Action Experience

Skill is not a written manual; it is a highly compressed form of action experience that combines procedural knowledge, contextual judgment, pattern recognition, automation, and reflective regulation. Knowing a rule is far from being able to execute it correctly in a complex environment.

2. No Single Correct Skill Representation

Skills lack a unique standard answer because they must respect the cognitive endowment of each model. The same task encoded in Claude 4.6 yields a transferable method template, while Qwen 3.5 produces a concrete operation procedure anchored to specific URLs and module categories. Both are valid, but the former favors abstraction and cross‑scene reuse, whereas the latter preserves detailed steps for stability.

3. Agents as Carriers of Human Skills

In the self‑evolving AI era, agents become new carriers of human expertise. Skills are acquired through expert‑supervised exploration: the agent acts in a real or simulated environment, receives corrective feedback, demonstrates the task, and iteratively refines its experience. This process captures tacit knowledge that cannot be fully expressed in static documents.

4. Knowledge Remains Essential, but Content Trumps Form

Traditional knowledge engineering (knowledge graphs, schema, rule bases) still matters for consistency and reasoning, yet the rise of powerful LLMs widens the knowledge entry point. Formats such as natural‑language instructions, tables, code snippets, and Wiki pages become equally valuable if they can drive execution and close the feedback loop.

5. Practice as Knowledge Production

An illustrative case is a Baidu map measurement task: the agent knows it must zoom the map, but initially fails because it does not place the cursor on the target object. Repeated practice reveals the missing steps—positioning the cursor, confirming the target remains centered, then measuring—turning an implicit action into a stable skill.

6. Embedding Process Evidence in Knowledge

Each execution should record four signals: the action taken, environmental change, success/failure outcome, and failure diagnosis. This evidence enables the system to compare strategies, identify stable causal patterns, and build reusable procedures while avoiding the mistaken belief that a correct result implies a correct process.

7. Layered Knowledge Architecture

A practical architecture consists of three layers: (1) low‑level factual databases and graphs for consistency, (2) a middle layer of experiential artifacts (natural language, tables, code, operation procedures), and (3) a high‑level dynamic layer where agents retrieve, compose, and execute knowledge on demand.

8. New Evaluation Criteria

Success is no longer measured by the sheer number of stored facts but by the agent’s ability to act reliably in complex environments, learn new skills under expert guidance, convert one‑off successes into stable capabilities, and automatically revise outdated experience. Traceability of process evidence becomes a prerequisite for trustworthy knowledge.

In summary, self‑evolving AI transforms knowledge engineering from a static description of the world into a dynamic system that captures, refines, and applies actionable experience, focusing on blind‑spot remediation rather than exhaustive documentation.