The New AI Stack: Models, Harnesses, Loops, and Self‑Evolving Agents

The Myth of Smarter Models

While the AI community fixates on foundation models such as GPT‑5.x or Claude Sonnet 5, the author asserts that the belief "smarter models equal better applications" is the biggest lie in the industry. The real differentiator is the engineering system that wraps the model, called the harness .

What Is a Harness?

The harness is a control layer surrounding the model, analogous to an operating system for a CPU. It decides how the model plans, when it calls tools, what it remembers, how it stores intermediate artifacts, how it self‑evaluates output quality, and when it should backtrack and retry.

Pattern 1: The Loop

Every production‑grade AI system uses a loop that repeats the cycle Plan → Execute → Observe → Improve → Repeat. A simplified Claude Code loop looks like:

Read the task

Plan the approach

Write and run code

See what failed

Fix it

Run again

Repeat until all tests pass

Even though the underlying model does not become smarter after each iteration, the system as a whole becomes more intelligent because each loop injects new context (error traces, test results, execution traces) into the next iteration.

Pattern 2: File‑System as Memory

Putting all data into the model’s context window quickly exhausts its capacity. Instead, long‑running tasks should persist logs, diffs, and intermediate results to files. The author demonstrates a bad practice versus a good one:

# Bad practice: cram everything into the context
context = previous_output + tool_result + error_log + history
# At step 47 the context overflows and crashes

# Good practice: use the file system
agent.write("experiments/run_3/error_log.txt", error_trace)
agent.write("experiments/run_3/results.json", metrics)
relevant = agent.read("experiments/run_3/results.json")

This change enables agents to recover from crashes, reason over their own history, keep the context window clean even after hundreds of steps, and share state between sub‑agents via files.

Pattern 3: Sub‑Agents

A single agent cannot handle every sub‑task. The parent agent splits a large job into independent sub‑tasks, spawns asynchronous sub‑agents, monitors them, and merges their results. Example workflow for a competitive‑analysis report:

Parent Agent receives task: "Write a complete competitive‑analysis report"
Spawns 4 sub‑agents:
→ Sub‑Agent 1: research pricing and core features of competitor A
→ Sub‑Agent 2: research pricing and core features of competitor B
→ Sub‑Agent 3: fetch latest industry news about both competitors
→ Sub‑Agent 4: scrape real user reviews from Reddit and app stores
Parent Agent waits, then merges the four outputs into the final report.
Total time equals the slowest sub‑agent, not the sum of all.

The key design rule is that sub‑agent outputs must be written to files, not left in transient context, ensuring auditability and crash‑resilience.

Context Engineering (CE)

Because the model’s weights are immutable at runtime, the only lever is the context it sees. Effective CE provides highly structured, dynamically evolving context that supplies the model with precisely the information it needs at each step. The author describes the ACE (Agentic Context Engineering) architecture with three components:

Generator : executes tasks using a structured playbook of (id, insight) pairs.

Reflector : analyses successes and failures, distilling them into concise insights.

Curator : updates the playbook by adding, deleting, or de‑duplicating insights.

Example playbook entry:

{"id":"001","insight":"Before retrying, write the current error trace to disk."}
{"id":"002","insight":"When using web search, add 'site:' to improve result quality."}
{"id":"003","insight":"Run local tests before committing; catches 80% of regressions."}

Self‑Harness (Self‑Improving Harness)

The harness can improve itself without human intervention through a three‑step closed loop:

Mine weaknesses : run many tasks, collect failure traces, cluster them, and identify root causes (e.g., timeouts on large files, loss of sub‑agent results after parent crash, insufficient error detail, context bloat after 30 rounds).

Propose fixes : a large model suggests precise code patches, such as inserting timeout handlers, persisting sub‑agent outputs after each atomic step, standardising error formats, and triggering context compression every 25 rounds.

Validate and merge : each patch is tested on an isolated benchmark suite; successful patches are merged into the production branch, failed ones are discarded.

Applying Self‑Harness to Claude 3.5 Sonnet raised its SWE‑bench Verified score from 20 % to 50 % without changing the underlying model.

Evolutionary Harness Search (AlphaEvolve)

AlphaEvolve extends Self‑Harness by maintaining a population of candidate harness implementations. The algorithm:

Initialise a pool of diverse harness code candidates.

Benchmark each candidate.

Select the top performers as parents.

Prompt a large model to generate diffs and evolution suggestions for each parent.

Produce child candidates from those suggestions.

Benchmark children and keep those that improve.

Repeat the evolutionary cycle.

Only code marked with explicit evolve blocks may be modified, protecting critical system components:

# EVOLVE-BLOCK-START
def plan_next_step(context, tools):
    # This block may be evolved by the algorithm
    prompt = f"Current context: {context}
Available tools: {tools}
Suggest next action:"
    return llm.generate(prompt)
# EVOLVE-BLOCK-END

def run_tool(tool_name, args):
    return tool_registry[tool_name](**args)

This sandbox ensures the optimizer cannot corrupt security‑critical code.

Darwin‑Gödel Machine (DGM)

DGM is the ultimate self‑reconstructing agent: it repeatedly audits its own logs, proposes harness code improvements, and evolves through generations. The process mirrors natural selection and yields dramatic benchmark gains (SWE‑bench 20 %→50 %, Polyglot 14.2 %→30.7 %).

Five Common Failure Pitfalls and Mitigations

Context collapse : after ~20 steps, rely on files instead of the growing context.

Implementation drift : generate an immutable spec file at task start and enforce compliance each loop.

Over‑optimism : keep a held‑out test set that the agent never sees and use it for final validation.

Reward hacking : isolate the evaluator from the loop and require human‑in‑the‑loop checks at critical control points.

Diversity collapse : add a novelty score (e.g., cosine similarity of candidate embeddings) to penalise overly similar offspring.

Practical Four‑Week Plan for Developers

Week 1: Implement the basic loop Plan → Execute → Evaluate → Retry instead of single‑prompt calls.

Week 2: Add persistent memory by writing intermediate results to files and reading them back.

Week 3: Introduce sub‑agents to parallelise independent subtasks and merge their file‑based outputs.

Week 4: Build a structured CE playbook of insights from past runs and have the system read it before each execution.

The Four‑Layer AI Stack

Layer 1 – Model : the passive foundation model (the CPU).

Layer 2 – Harness : the OS that provides toolkits, persistent memory, loop control, sub‑agent orchestration, and context engineering.

Layer 3 – Optimizer : automatically mines trace logs, proposes precise code fixes, validates on isolated benchmarks, and merges improvements.

Layer 4 – Evaluator : an external auditor that supplies final benchmark scores, human reviews, and a blind test set inaccessible to the optimizer.

Skipping any layer reduces a product to a fragile chatbot or forces endless manual patching.

Implications for Today’s AI Developers

The rapid progress of companies like Anthropic and OpenAI is driven not by sudden model breakthroughs but by ever‑stronger harness frameworks that enable self‑looping, persistent memory, task parallelisation, and self‑correction. The true moat lies in the system that wraps the model, and that system is now learning to evolve itself.