From Tool Loops to Agent Runtimes: How AI Agent Architecture Is Evolving

From Tool Loops to Agent Runtimes

From 2023 to 2024 the typical way to build an AI that can call tools was to wrap a large model in a while loop: the model generates a function call, the code executes it, and the result is fed back to the model. This ReAct loop (think‑call‑return) works well for demos but quickly shows shortcomings when deployed in production.

Problems include loss of state on container restart, credential leakage when generated code runs on the host, and the inability to pause long‑running jobs for human approval. The issue is not model intelligence but the loop’s lack of production‑grade capabilities such as persistence, isolation, recovery, and manual takeover.

By August 2026 the major SDKs—OpenAI Agents SDK, LangGraph, Microsoft Agent Framework, AWS AgentCore, and Claude Agent SDK—have converged on a common set of features: persistent state with checkpoints, sandboxed execution, failure recovery, tool discovery, and human‑approval primitives. The agent is now a full‑task runtime rather than a simple model‑tool loop.

01 Architecture Evolution: From a Linear Loop to a Runtime

Early agents followed a straight line: model thinking → tool call → result return. All state lived in the single request’s context window; when the process died, everything reset.

Modern runtimes are layered. The planning layer decomposes a task into dependent steps. The state layer checkpoints each step to durable storage. The sandbox layer isolates generated code from host credentials. The recovery layer resumes from the last successful checkpoint after a node failure. The approval layer pauses before irreversible actions until a human authorizes.

In other words, an agent has become a "digital lifeform" with built‑in scheduling, memory, isolation, and braking systems.

02 What the Five Platforms Added

OpenAI Agents SDK (v0.14, Apr 2026) separates the control plane from the compute plane. Harness holds credentials and orchestrates; model‑generated code runs in a pluggable sandbox (Docker, local Unix, Cloudflare, E2B, Modal, etc.). Long‑running jobs survive via checkpoint, snapshot, and rehydration. Memory is split into session memory (SQLite/Postgres/Redis back‑ends) and workspace memory (distilled MEMORY.md).

LangGraph introduces a Checkpointer and interrupt. A StateGraph writes the whole graph state to a database after each node, using thread_id as a pointer. An interrupt(payload) freezes the current state and raises an approval request; a Command(resume=value) continues from the same line, even after pod changes or hours of delay. Version v0.4 (Apr 2026) embeds interrupt objects directly in the invoke return value, enabling bulk branch recovery.

Microsoft Agent Framework 1.0 (GA Apr 2026) merges AutoGen multi‑agent collaboration with Semantic Kernel’s enterprise pipelines, using Durable Functions / Durable Task Scheduler for stateful orchestration. Its Durable Extension restores sessions across stateless workers and avoids token consumption while waiting for human or external events. Native Entra ID provides identity governance; Azure Monitor offers end‑to‑end observability.

AWS AgentCore follows a "managed runtime" model. Each user session runs in an isolated microVM for up to 8 hours; after 15 minutes of idleness the VM pauses but retains state. Policies expressed in Cedar enforce per‑tool call interception at the gateway, making the model unaware of the block. Runtime Instances (released 6 Aug 2026) extend a single session to 14 days and support GPU instances.

Claude Agent SDK differentiates with hooks and permission primitives. It offers 30 lifecycle hooks; PreToolUse can inspect, modify, or block a tool call before execution. canUseTool enables programmatic authorization per call. Sub‑agents declare their toolset, model, and max rounds via AgentDefinition. Sandbox mode confines Bash to a network‑ and filesystem‑restricted environment.

03 Long‑Running Task Systems: Agents Going “Overnight”

Historically an agent’s unit of work lasted seconds. All five platforms now aim to keep agents alive across hours, overnight periods, and human‑approval windows. The common mechanisms are:

Asynchronous background execution so the main thread does not block.

Persistent session state that survives hours or days.

Checkpoint‑driven resume using the session ID as a durable cursor.

AWS documentation states a real agent “runs for minutes, then sleeps for a day”. The managed runtime keeps the session dormant without incurring cost and wakes it instantly on demand. Claude’s resume and forkSession let the agent branch on error lines, retry, and preserve original context.

Implementation details per vendor:

AWS microVM pauses after 15 minutes of idleness, retains state, and scales to zero for cost‑free sleep.

Microsoft Foundry also pauses after 15 minutes and offers a "persistent workflow mode" for multi‑hour external waits (e.g., a 6‑hour API call).

OpenAI uses checkpoint, snapshot, and rehydration to revive expired containers from the last checkpoint instead of restarting from scratch.

04 Key Design Principles

Separate state from orchestration. LangGraph’s checkpointer and OpenAI’s control/computation split decouple the "decide next step" logic from the "remember where we are" storage. This allows the orchestration layer to swap models or policies while the state layer continues persisting, and recovery can replay state without re‑executing the whole chain.

Sandbox isolation. Generated code must never share the same process as API keys. Whether using Docker, microVM, Firecracker, or gVisor, isolation prevents lateral movement from a successful prompt injection to an internal network breach.

Human‑takeover as a first‑class primitive. Approval should not be buried in business‑logic if‑else blocks. Instead, a runtime‑level interrupt, permission‑request hook, or Cedar policy can cleanly pause execution before irreversible actions.

05 A Verdict: Models Should Not Directly Manage State

Embedding state management inside the model was the most hidden cause of early agent failures. Model context windows overflow, processes restart, and concurrent pods split execution, making "the model remembers where it is" unreliable.

This explains why all five platforms introduce an independent runtime layer that isolates orchestration, state, sandbox, and approval. The model sits only at the top decision layer, while persistence, isolation, and recovery live elsewhere, allowing agents to move from demo prototypes to production‑grade systems.

All five platforms converge on a clear consensus: persistent storage owns state, the runtime owns orchestration, and the model only makes decisions given the current state. The model is the brain, but memory, hands, and brakes belong to the surrounding system.