Which Layer Should Your Self‑Learning Agent Evolve? A Three‑Layer Breakdown

Three Learning Layers: Model, Harness, Context

Many hype "self‑evolving agents," but most products don’t need the version that changes the model itself. CopilotKit CEO Atai Barkai highlights that the most valuable learning signal is the one most teams ignore – real user feedback.

The cleanest decomposition, attributed to Harrison Chase, separates self‑learning into three layers:

The Model : the trained weights.

The Harness : code surrounding the model – the run loop, tools, prompts.

The Context : the growing memory and skills outside the model.

In Claude Code, the model is Claude, the harness is Claude Code itself, and the context consists of your CLAUDE.md and various skills. Each layer can improve independently without touching the others.

Layer 1 – Model

Improving the model itself is the approach most people imagine, yet it is rarely run in production. Three routes are described:

Modify training code and keep useful changes – Karpathy’s AutoResearch runs a tiny training loop, scores each five‑minute run, and after about 100 rounds found an ~11% speedup. The model’s weights never change.

Generate training data and inject it into the weights – MIT’s SEAL creates its own data, retrains for 30‑45 s per round, but suffers catastrophic forgetting.

Use a free scorer to evolve code – DeepMind’s AlphaEvolve proposes a code change, scores it, and keeps winners. It accelerated an attention kernel by 32.5% and discovered a matrix‑multiplication shortcut that had not been broken since 1969. This route only works where a computer can automatically score results, such as code or math, not in a refund‑approval scenario.

Because scoring requires a free, trustworthy metric, model‑level self‑learning is usually confined to labs.

Layer 2 – Harness

The harness is everything outside the model: the loop, tools, prompts, and pre‑checks. Four strategies are outlined:

Manual loop engineering – a basic loop calls tools until the model signals completion. Advanced loops add validation scoring, scheduled triggers, and self‑modifying code. Sydney Runkle’s "The art of loop engineering" visualises four such layers.

LangChain Deep Agents – run tasks, record trajectories, let an encoding agent rewrite prompts and tools. On Terminal‑Bench 2.0 the score rose from 52.8 to 66.5, moving from the top 30 to the top 5.

Self‑Harness – the loop runs without human oversight, keeping only changes that improve test results. When applied to three frozen models, scores rose from 40.5→61.9, 23.8→38.1, and 42.9→57.1, showing the harness was the bottleneck.

Microsoft’s Agent Framework – an installable stack providing file memory, disk‑based skills, plan‑and‑execute mode, and a sandbox shell.

All harness approaches require a computable scoring function; otherwise they cannot operate.

Layer 3 – Context

Context stores memory and skills as pure text, editable by the system. It comprises three memory types:

Semantic : factual statements (e.g., "the refund limit is $2000").

Episodic : past experiences (e.g., "last week the refund was rejected").

Procedural : how to handle a situation (e.g., "loyal customer, repeated issue, approve").

Most teams only use semantic memory, but a self‑improving agent needs episodic and procedural memory. Three concrete approaches are described:

Letta / OpenClaw – freeze weights and rewrite pure‑text memory in the background each night.

Hermes (Nous Research) – maintain a skillbook that logs failures, frequencies, and successful actions; it can improve without GPU.

Anthropic / Manus – store a successful run as SKILL.md (≈100 tokens) that is loaded on demand.

These methods keep learning in immutable text, which can be migrated across models or rolled back.

The Overlooked Signal: Real User Feedback

All previous methods learn only from the agent’s own execution. A far richer signal comes from the humans who work alongside the agent – customer‑service reps, salespeople, managers. Their decisions cannot be forged.

Two ways to capture this signal are:

Watch‑over‑the‑shoulder – record screen, keystrokes, clicks. Meta deployed this in 2026 on employee laptops, giving a complete view of human actions but not the agent side.

Learn from interaction – the agent sees the text the human types but misses the underlying actions.

Only the interface where human and agent co‑work provides both halves. CopilotKit’s AG‑UI (Agent‑User Interaction Protocol) streams every event – tool calls, state changes, approvals – into a unified log. Using CopilotKit Intelligence, companies can ingest this log, as demonstrated in a refund‑agent example where a manager’s manual approval becomes procedural memory for future cases.

One‑Sentence Takeaway

The real question is not whether an agent should get stronger, but which layer should be strengthened ; the best layer is the context that learns from genuine user decisions, because that signal cannot be fabricated.