How ACK One Fleet Transforms Agent Sandbox from Single-Cluster to Multi-Cluster

To enable AI agents that can not only chat but also perform tasks, each Agent requires a secure, isolated, on‑demand runtime environment called an Agent Sandbox. When deployed on a single Kubernetes cluster, the sandbox faces two major problems: a capacity bottleneck because the sandbox consumes compute, network, and storage resources, and a concentrated fault risk since the whole service depends on a single control plane.

If customers manage multiple clusters themselves, additional challenges arise: they lack a global water‑level view of remaining capacity and load, forcing manual or polling‑based sandbox placement that leads to uneven utilization; static allocation rules cannot keep up with dynamic resource changes, causing requests to be sent to near‑saturated clusters and resulting in startup failures; there is no automatic health detection or failover across clusters, so requests may hit unhealthy clusters; and utilization remains low because peak‑shaving across clusters is impossible, requiring manual capacity adjustments that are error‑prone and hard to scale.

ACK One Fleet addresses these issues by aiming to increase total sandbox scale, provide high availability and failover, improve resource utilization, and accelerate sandbox startup, all while requiring minimal changes from users.

The Agent Sandbox itself offers MicroVM‑level isolation, memory‑level sleep and wake, checkpoint cloning, and can elastically scale to up to 15 000 sandboxes per minute. It is fully compatible with the native Kubernetes ecosystem and integrates seamlessly with the E2B SDK, AgentScope, and other popular AI agent frameworks.

ACK One Fleet is Alibaba Cloud’s distributed cloud container platform that provides a unified management plane for multi‑cluster, multi‑region Kubernetes scenarios. It can ingest self‑built clusters, clusters from other clouds, and ACK clusters across different regions, allowing applications, policies, and traffic to be governed from a single control point.

For the Agent Sandbox scenario, Fleet adds dedicated scheduling enhancements that upgrade the capabilities of a single‑cluster Kubernetes setup to coordinated multi‑cluster operations, while keeping the user experience unchanged.

Architecture and core capabilities: Two pathways are provided—an E2B link and a Kubernetes CR (Custom Resource) link.

E2B SDK link (global control‑plane / data‑plane separation):

User deploys a SandboxSet to multiple clusters via Fleet.

User invokes the E2B SDK client, which reaches Fleet’s multi‑cluster gateway.

The Global Sandbox Scheduler performs global scheduling to select a target cluster and records the routing in a Global Route Table.

The request is routed through the Multi‑cluster Gateway to the chosen cluster’s Sandbox Manager ALB Ingress.

The selected sub‑cluster’s Ingress, Sandbox Gateway, Manager, and Controller provide the actual sandbox capacity and run the Agent Sandbox.

The design separates control‑plane traffic (lifecycle management such as create, sleep, wake) from data‑plane traffic, which is handled directly by each sub‑cluster. This avoids network bottlenecks and single‑point failures for data‑intensive workloads like model downloads or high‑frequency API calls.

Global scheduling capabilities include:

Single‑cluster capacity control: Prevents any cluster from exceeding its sandbox capacity, reducing overload risk.

Water‑level balanced scheduling: Distributes sandboxes based on current water‑level metrics across clusters for balanced resource usage.

Multi‑cluster fault‑tolerant failover: Automatically or manually redirects sandbox creation to healthy clusters when a failure occurs, ensuring continuous service.

Pre‑warm pool awareness and affinity scheduling: Prioritizes clusters with sufficient pre‑warmed sandboxes to achieve rapid startup, beneficial for RL training and other cold‑start‑sensitive scenarios.

Cluster‑level priority scheduling: Across regions, prefers the primary region until its single‑cluster limit is reached, then schedules to backup regions.

In summary, ACK One Fleet’s multi‑cluster Agent Sandbox upgrades the capabilities of a single‑cluster Kubernetes deployment to a coordinated multi‑cluster solution. It exposes a unified entry point, manages multiple ACK clusters across regions or accounts, and retains the sandbox’s native MicroVM isolation, memory‑level sleep/wake, and checkpoint cloning. By leveraging both the E2B SDK and Kubernetes CRD pathways, it removes the capacity and fault‑domain ceiling of single‑cluster deployments.

Business users gain four key benefits: massive scale through distributed control‑plane pressure, multi‑region high availability via cross‑cluster failover, higher resource utilization through capacity control and water‑level balancing, and faster sandbox startup thanks to pre‑warm pool awareness and affinity scheduling—especially valuable for reinforcement‑learning training workloads. All of this requires only minimal adaptation from existing workloads.

Future work will continue to enhance multi‑cluster scheduling, adapt to more real‑world scenarios, and enable truly cross‑cluster automatic flow of production‑grade AI compute.