How Externalizing Memory, Skills, and Protocols Powers Next‑Gen LLM Agents

Unified Review of Externalization in LLM Agents

Recent papers from Shanghai Jiao Tong University, Sun Yat‑sen University, and Carnegie Mellon provide a comprehensive overview of how reliable agent capabilities stem not only from model weights but also from externalizing cognitive burdens into structured infrastructure.

1. From Weights to Context to Harness: Three Capability Migrations

1.1 Weights Era

Early LLM deployments relied entirely on model parameters. Pre‑training compresses statistical regularities, world knowledge, and reasoning habits into weights. Scaling laws show a predictable relationship between parameter count and performance.

Limitations: Knowledge updates require retraining, auditing is hard because knowledge is distributed across billions of parameters, and personalization is lacking as a single weight set serves millions of users.

1.2 Context Era (Prompt Engineering)

Capability shifted from internal model knowledge to external input design. Few‑shot examples, chain‑of‑thought reasoning, and retrieval‑augmented generation (RAG) demonstrate that model behavior can be dramatically altered without changing weights.

Key transition: The difficult "recall" problem (retrieving knowledge from parameters) becomes a simpler "recognition" problem (using provided context).

1.3 Harness Era

As context windows saturate and prompt templates become cumbersome, engineering focus moves to the environment in which the model runs. The Harness layer integrates three externalization dimensions—memory, skills, and protocols—providing orchestration logic, constraints, observability, and feedback loops.

2. Externalized State: Memory Systems

Memory externalization addresses the temporal continuity burden of agents. Native LLMs are stateless generators; each call starts with a fresh context, requiring continuity to be rebuilt in prompts.

Four architectural patterns are identified:

Monolithic Context: All history stored directly in the prompt (simple but capacity‑limited).

Context + Retrieval Store: Recent state in the prompt, long‑term trajectory in external storage (RAG pattern).

Hierarchical Memory & Orchestration: Explicit extract‑consolidate‑forget operations (e.g., MemGPT, Memory OS).

Adaptive Memory Systems: Dynamic modules and feedback‑driven retrieval strategies (e.g., MemEvolve, MemRL).

Cognitive‑Tool View: Memory turns "unbounded recall" into "bounded, curated retrieval," reshaping the task structure at each decision point.

3. Externalized Expertise: Skills

Skills externalize procedural burden. While a model may "know" how to perform a task, reliable execution requires repeatable workflows, default values, and constraints.

3.1 Three Skill Components

Operational Procedure: Task skeleton (step decomposition, phases, dependencies, stop conditions).

Decision Heuristics: Practical rules for branching points (what to try first, when to abort).

Normative Constraints: Acceptability boundaries (testing requirements, scope limits, access control).

3.2 Evolution of Skills

Stage 1 – Atomic Execution Primitives: Stable calls to single tools (e.g., Toolformer).

Stage 2 – Large‑Scale Primitive Selection: Retrieval and selection among many tools (e.g., Gorilla, ToolLLM).

Stage 3 – Skills as Packaged Expertise: Bundling task‑type procedures into reusable units.

Key Mechanisms:

Progressive Disclosure: Incremental exposure of skill documentation (name → summary → full guide).

Execution Binding: Skills must be bound to executable actions via protocol interfaces (tools, APIs, files, sub‑agents).

Composability: Skills can be combined serially, in parallel, conditionally, or recursively.

4. Externalized Interaction: Protocols

Protocols externalize coordination burden. Without explicit contracts, a bare model must improvise message formats, parameter structures, lifecycle semantics, and recovery behavior.

4.1 Protocol Content Dimensions

Invocation Grammar: Parameter names, types, order, and return schema.

Lifecycle Semantics: Multi‑step interaction coordination rules (state machines, event flows).

Permission & Trust Boundaries: Authorization rules, data flow, audit requirements.

Discovery Metadata: Capability registries, capability cards, schema endpoints.

The Harness layer manages protocols through three functional interfaces: Interact (API/tool communication), Perceive (environment, context, memory, feedback), and Collaborate (agent‑agent or agent‑human coordination).

5. Unified Externalization: Harness Engineering

Harness is not a fourth externalization dimension; it is the runtime environment in which the model operates, providing perception, decision, and action capabilities.

Six analysis dimensions of Harness:

Memory (state persistence)

Skills (reusable routines)

Protocols (deterministic interfaces)

Permission (sandbox, file isolation)

Control (recursion limits, cost caps)

Observability (structured logs, execution traces)

From a distributed cognition perspective, Harness shapes what enters the perception field, what persists across sessions, which operations are callable, which actions require approval, and which intermediate states can be revised.

6. Cross‑Module Coupling

The three externalized modules (memory, skills, protocols) are tightly coupled, forming six key interaction flows that enable coherent agent behavior.

Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering
https://arxiv.org/pdf/2604.08224