How Agent KB Enables Cross‑Framework Knowledge Sharing for Smarter AI Agents

Background and Core Challenges

Rapid advances in AI agents have led to fragmented ecosystems where frameworks such as smolagents, OpenHands, and OWL accumulate valuable problem‑solving experience in isolation, preventing knowledge reuse.

Three major design flaws are identified:

Task‑specific experience isolation – agents cannot transfer knowledge across tasks and must start from scratch on new domains.

Single‑granularity retrieval – a uniform retrieval mechanism cannot distinguish between high‑level planning and fine‑grained execution adjustments.

Static experience replay – raw experience is stored without abstraction, limiting adaptability to new contexts.

Agent KB Architecture

Agent KB introduces a Reason‑Retrieve‑Refine workflow combined with a teacher‑student dual‑agent collaboration.

Experience Representation

Each experience is encoded as a structured tuple consisting of:

Problem pattern (task type, input schema, constraints)

Desired output (optional)

Workflow (sequence of reasoning and execution steps)

Context features (domain, difficulty level)

Links to related experiences

Student Agent (Workflow‑Level Planning)

When a new query arrives, the student agent performs problem reasoning, retrieves similar experiences based on query embeddings, and adapts the retrieved workflow to the current context through parameter substitution, step expansion/contraction, and domain translation.

Teacher Agent (Log‑Based Refinement)

After the student agent executes the initial plan, it forwards the query and execution logs to the teacher agent. The teacher analyzes errors, extracts key execution patterns, retrieves refined experiences from Agent KB, and generates improvement suggestions that are fed back to the student for plan optimization.

Disagreement Gate

A Disagreement Gate evaluates the compatibility between retrieved knowledge and the current reasoning path, integrating external knowledge only when it is likely to enhance reasoning.

Experimental Validation

Comprehensive evaluations were conducted on benchmarks such as GAIA, Humanity’s Last Exam, GPQA, and SWE‑bench.

GAIA: GPT‑4.1 with Agent KB improved pass@1 from 55.15% to 61.21% (+6.06 pp); Claude‑3.7 on Level‑3 tasks rose from 50.00% to 57.69%.

smolagents achieved an 18.7 pp boost on pass@3 (55.2% → 73.9%).

SWE‑bench: o3‑mini gained 8.67 pp (23% → 31.67%); OpenHands improved by 4.0 pp (24.3% → 28.3%).

Cross‑model consistency was observed across GPT‑4o, GPT‑4.1, o3‑mini, Claude‑3.7, Qwen‑3 32B, and DeepSeek‑R1, confirming the generality of Agent KB.

Ablation studies revealed:

Hybrid retrieval (workflow + log) outperforms pure text‑similarity or pure semantic similarity methods.

Automatically generated experiences perform on par with manually curated ones (75.15% vs 76.97%).

Knowledge Construction Pipeline

Multi‑source data aggregation from datasets such as BrowseComp, MultiHopRAG, and SWE‑bench.

Human‑guided error analysis to abstract common failure patterns (e.g., tool selection errors, reasoning misalignments).

Template‑based generation of structured experiences.

Hierarchical knowledge graph construction where nodes are experiences and edges encode semantic and structural relations (abstraction, composition, alternatives).

Typical Applications

Case 1 – PDB Distance Calculation (Bioinformatics) : Using Agent KB to filter irrelevant ATOM/HETATM records and validate N‑CA bond lengths corrected a previously erroneous O‑H distance of 0.961 Å to the correct 1.456 Å.

Case 2 – API Documentation Generation (Software Engineering) : Agent KB suggested precise string‑handling checks to eliminate spurious backslashes introduced during automatic documentation, improving output correctness.

Insights and Future Directions

Agent KB marks a shift from isolated to collective intelligence for AI agents, democratizing access to high‑quality problem‑solving strategies.

Future work includes developing causal reasoning frameworks, implementing continual learning mechanisms that auto‑update policies from deployment feedback, and establishing theoretical foundations for cross‑agent knowledge transfer.

{
  "question": "...",
  "agent_plan": "...",
  "agent_experience": "..."
}

Agent KB: Leveraging Cross‑Domain Experience for Agentic Problem Solving https://arxiv.org/pdf/2507.06229 https://github.com/OPPO-PersonalAI/Agent-KB