How Externalization Drives the Evolution of LLM Agents – Insights from a 54‑Page SJTU Review

Overview

The paper "Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering" (arXiv:2604.08224, Apr 2026) presents a systematic analysis of why modern LLM agents improve primarily through external cognitive infrastructure rather than larger base models.

Core Claim

Agent performance increasingly depends on four pillars—persistent memory, reusable skill documents, standardized tool interfaces, and a Harness layer—collectively termed externalization . The authors identify three structural mismatches that explain the limited impact of merely upgrading the base model:

Continuity mismatch : limited context windows prevent stable cross‑session state.

Consistency mismatch : multi‑step workflows are re‑derived each time, leading to divergent execution paths.

Coordination mismatch : ad‑hoc tool and agent interactions rely on fragile temporary agreements.

These mismatches are resolved by moving cognitive burdens from the model to external artifacts, a concept rooted in Don Norman’s “cognitive artifacts” theory.

Evolution of Capability Layers

The authors trace a three‑stage shift:

Weight layer (2022‑2023) : capabilities reside in model parameters; updates are monolithic and hard to audit.

Context layer (2023‑2024) : prompt engineering, chain‑of‑thought, and retrieval‑augmented generation keep the model frozen while externalizing short‑term state.

Harness layer (2024‑present) : reliability hinges on external memory, tool registration, protocols, sandboxing, and orchestration. Systems such as OpenHands, SWE‑agent, and Deep Research exemplify this pattern.

Memory Externalization

Memory is organized into four levels: work context, episodic experience, semantic knowledge, and personalized memory. Architectures evolve from monolithic prompt stuffing to hierarchical, adaptive retrieval systems that replace “recall” with “search”.

Skill Externalization

Skills are packaged as explicit artifacts consisting of an operation program, decision heuristics, and normative constraints. Four generation pathways are described: manual authoring, trajectory distillation, autonomous discovery (e.g., Voyager), and compositional construction. The pipeline registers, progressively reveals, and composes skills for runtime binding.

Protocol Externalization

Protocols formalize interaction contracts, externalizing call syntax, lifecycle semantics, permission boundaries, and discovery metadata. The paper categorizes three protocol families: Agent‑Tool (e.g., MCP JSON‑RPC), Agent‑Agent (e.g., A2A), and Agent‑User (e.g., AG‑UI).

Harness Engineering

The Harness layer provides a unified cognitive environment, handling agent loops, sandbox isolation, human approval gates, observability, policy enforcement, and context‑budget management. Six design dimensions are enumerated, forming a self‑reinforcing cycle where memory informs skills, skills generate execution traces that enrich memory, and protocols bind the whole system.

Illustrative Scenario

In a software‑engineering agent tasked with adding a feature, running tests, and opening a PR, externalization replaces a fragile monolithic prompt with persistent project memory, reusable skill documents encoding workflow conventions, standardized tool protocols, and a Harness that orchestrates steps, validates outputs, and recovers from failures.

Future Directions

The authors outline six research frontiers: expanding externalization boundaries (planning, verification, orchestration), embodied agents adopting the same pattern, self‑evolving Harnesses, security & governance of memory poisoning and skill injection, shared infrastructure across multi‑agent ecosystems, and new evaluation metrics for infrastructure contributions (transferability, maintainability, context efficiency).

Take‑away

Better agents are not merely stronger reasoners; they are better organized cognitive systems whose reliability stems from systematically externalizing memory, skills, protocols, and harness infrastructure.