Building Containerized Sandboxes for Multi‑Agent AI: Architecture, Key Technologies, and Real‑World Practices

Why a Sandbox Is Needed for Multi‑Agent AI

As AI agents evolve from simple Q&A bots to complex task‑oriented systems, the underlying infrastructure must provide strong isolation, rapid provisioning, state persistence, and cost efficiency. The article outlines a sandbox solution built on container technology to meet these demands.

Agent Sandbox Construction with Containers

Using a code‑debugging scenario, an agent first invokes a large model for planning, then executes generated code inside an isolated environment. The workflow may involve tool calls (MCP) to retrieve history, reviews, knowledge bases, and memory, finally aggregating results and updating the memory module. Because generated code can be erroneous or malicious, capabilities such as Code Interpreter, Browser Use, and Compute Use must run inside a sandbox.

Core Challenges

Data security: isolate sessions to prevent malicious code from accessing the host or other sessions.

Massive, bursty scaling: a single task can spawn dozens or hundreds of sandboxes, causing rapid resource fluctuations.

State persistence and cost control: agents often run long‑lived, multi‑round tasks, but sandboxes spend much time idle, leading to waste if kept running.

Sandbox Lifecycle and Checkpointing

Sandboxes are created from templates that include image, specifications, orchestration config, and optional checkpoints. They transition through states such as Pending , Running , Paused , and Completed . While running, a sandbox can be paused to release compute resources and later resumed. Checkpoints can capture the filesystem, memory, or GPU memory, enabling reuse for long‑running tasks or reinforcement‑learning exploration.

Key Isolation Technologies

Isolation is achieved through a combination of mechanisms:

CPU/Memory isolation reinforced by KVM, Kata, Firecracker, or gVisor.

Network isolation that limits east‑west traffic between agents and restricts north‑south egress to required endpoints.

Storage isolation via independent mount points for shared storage.

RBAC‑based authentication for each sandbox.

Observability and audit trails covering creation, execution, pause, resume, and destruction.

Resource Management Strategies

Two paths address the “fast start, short run, long wait, unpredictable load” problem:

Serverless compute (ACS Pod, AWS Fargate, Azure AKS Pod, GKE Agent Sandbox) reduces user‑level resource management complexity.

Self‑built runtimes on Kubernetes node pools using runD, Kata, Firecracker, or gVisor provide custom security and elasticity.

Hardware reference: AMD EPYC CPUs up to 192 cores/384 threads, offering a 7:1 integration ratio and up to 67% total‑cost‑of‑ownership reduction for dense sandbox deployment.

State Persistence Techniques

Filesystem state is saved via rootfs, temporary, and persistent volumes. Memory state can be checkpointed with CRIU (process‑level) or full VM memory snapshots. GPU memory can be captured using NVIDIA cuda‑checkpoint. Firecracker typically uses persistent storage plus VM memory snapshots, while gVisor can combine persistent storage with CRIU.

Fast Wake‑Up Practices in Alibaba Cloud ACS Sandbox

The practice consists of five steps: template‑based pooling of homogeneous configurations, serverless large‑scale resource provision, reuse of ACS security sandbox for compute/network/storage isolation, block‑storage snapshot pre‑warming and replication for rapid rootfs recovery, and lifecycle management by a Sandbox Operator that hides pool complexity. Pre‑warmed objects include MicroVMs, compute specs, NICs, storage devices, rootfs, and container image caches.

Sleep‑Wake Paths

Two approaches are compared:

CRIU‑based process checkpoint: lightweight, small data size, low hardware coupling, but limited for multi‑process, multi‑container dependencies.

Full VM snapshot: preserves guest OS, network stack, and other context, offering higher stability at the cost of greater hardware coupling and larger data size.

Both can be combined with lazy‑load techniques to achieve sub‑second restoration.

OpenKruise Agents Project

OpenKruise Agents is an open‑source sub‑project of the CNCF‑incubated OpenKruise community, targeting AI Agent sandboxes. It provides create, sleep, wake, resource pooling, checkpoint, fork, and commit capabilities. Integration points include the E2B SDK, REST, or gRPC for agents, and the Kubernetes API for pod‑based sandbox execution.

Integration Options

Developers can use the Python SDK compatible with E2B to invoke sandbox operations without Kubernetes knowledge. Platform teams can employ declarative resources such as Sandbox CR and SandboxSet to manage lifecycle, state sync, pause scheduling, and resource reclamation.

Architecture Overview

The system consists of three components: Agent Apiserver (standard API and routing), Sandbox Manager (lifecycle and pooling), and Checkpoint Manager (snapshot, clone, and image commit).

Sleep‑Wake Workflow

An agent sends a Sleep request via E2B; the Apiserver triggers a Kubernetes Job that pauses the sandbox and saves filesystem and memory state to cloud disks or snapshots. When wake is needed, another Job restores the state and resumes the sandbox. Users only configure template selection and whether to persist memory or filesystem; the underlying interactions with CRIU, runtimes, and cloud storage are encapsulated.

Pooling and Autoscaling

Sandboxes are classified as In‑Using, Available, or Paused. Upon request, the Apiserver quickly allocates an instance from the Available pool, and Envoy‑based proxy routes traffic, achieving second‑level provisioning. An Autoscaler adjusts pool size based on water‑level or time, while a Refresher cleans and reuses used instances and updates images. Pre‑warm instances can run on minimal specs and be scaled up just before execution.

Commit and Checkpoint

Commit captures the overlayfs diff, packages it as a standard OCI image, and pushes it to a registry, enabling reproducible environments. Checkpoint builds on Commit by adding CRIU‑captured process registers, memory pages, and network connections, forming a “image + memory dump” snapshot that can be forked for reinforcement‑learning or Monte‑Carlo Tree Search branches.

Ecosystem Layers

Agent frameworks (LangGraph, AgentScope, Kagent, Dapr Agent) for multi‑agent coordination and reasoning.

Sandbox infrastructure (OpenKruise Agents, AgentCube, SIG Agent‑Sandbox) for resource provisioning and state persistence.

Agent runtimes (AIO Sandbox, redroid, AndroidWorld) for tool integration and OS emulation.

Container/VM technologies (Kata, gVisor, MicroVM) for secure, fast execution.

Customer Case Studies

MiniMax (MaxClaw, MaxHermes) : Control‑plane/execute‑plane separation using ACK for orchestration and ACS Agent Sandbox on MicroVMs. Reported metrics include 20‑40 ms instance provisioning, 15 000 sandboxes per minute, strong isolation, and state recovery after instance drift or restart.

Kimi : Supports tens of thousands of concurrent C‑side agents and RL training. Hierarchical scheduling uses ACK node pools for baseline load and overflows to ACS Agent Sandbox when queue > 500 or wait > 30 s. Results show 90 % reduction in virtualization overhead, > 60 % faster sandbox start‑up, stable scheduling of 100 k pods, and seconds‑level cloning of RL branches.

Conclusion

The key to multi‑agent infrastructure is not merely launching more containers but combining strong isolation, ultra‑fast delivery, continuous state, and cost control into a full lifecycle. Containers, MicroVMs, serverless, pre‑warm pools, pause‑resume, commit, and checkpoint each solve specific stages, while OpenKruise Agents abstracts these capabilities into developer‑friendly APIs and ops‑friendly interfaces, enabling scalable, production‑grade AI agent execution.