Evolution of Agent Infrastructure: Engineering Insights from Tencent Cloud Agent Runtime

01 Industry Perception Correction

Kubernetes can schedule containers but cannot natively schedule the continuity of agents. The author highlights several mismatches: lifecycle mismatch (agents need session‑bound lifecycles, not pod lifecycles), PVC cannot store all agent state, streaming connections bind execution instances, scaling signals are distorted because agent workloads are bursty, and security isolation requirements are higher.

A typical pitfall is deploying agents with Deployment or StatefulSet, PVC, and sticky sessions: it works in demo, but in production it leads to idle resource consumption, unstable recovery, scaling that does not help old sessions, and complex fault‑migration.

Conclusion: Kubernetes remains the entry point, but an Agent‑native runtime layer is required on top of it.

02 Pain Points Attribution

The team identified eight high‑frequency fatal issues when moving agents to production:

State & Session Recovery : Demo relies on context windows, but production must handle disconnections, restarts, timeouts, human confirmations, and cross‑day continuation.

Long‑Task & Lifecycle Management : Agents run for minutes with human waits; long‑lived connections cause cost and stability problems.

Tool Execution & Security Isolation : Agents run code, access internal systems, and need sandbox, outbound control, credential injection, file isolation, and audit.

Enterprise System Integration & Credential Governance : Agents must access CRM, ticketing, finance, knowledge bases; permissions, credential storage, and audit become bottlenecks.

Streaming Output, Reconnection & Routing : Channel, stream, and execution must be decoupled; connections may drop while tasks must continue.

Observability & Troubleshooting : Production failures must be explained; failures can stem from model, prompt, tool, network, permission, or state.

Cost & Capacity Model : Costs include sandbox, storage, long connections, cold start, pause pool, logs, vector retrieval, tool calls, and human fallback; idle time also consumes resources.

Delivery & Version Management : Production requires versioning, canary, rollback, evaluation, config management, IaC, and environment reproducibility.

Solution: make Session, Memory, Workspace, event logs, and checkpoint/snapshot core capabilities rather than ad‑hoc business code.

03 Scheduling & Resource Mismatch

Kubernetes schedules pods (stateless services) while agents require scheduling of continuously evolving sessions. This leads to concrete problems:

Requests can be scheduled, but the task’s execution environment cannot be.

Scaling adds new pods for new tasks but does not relieve pressure on long‑running sessions.

Scaling down may cut off a pod waiting for user confirmation, effectively aborting the task.

Idle pods are not releasable because agent state must be preserved; pause, freeze, wake, continue are needed.

Fault recovery is not just a container restart; without a dedicated state layer the restored pod lacks the previous agent context.

Therefore, K8s can schedule containers, but an Agent‑native runtime must handle session continuity.

04 State Persistence & Recovery Challenges

Using PVC only protects disk files, not the full execution context. Problems observed:

State loss beyond disk : In‑memory state, unflushed events, tool results, browser sessions disappear on pod restart.

Uncontrolled recovery time : Restoring requires pulling images, mounting PVC, loading history, re‑logging into systems, which feels like a full restart.

Snapshot consistency : A PVC snapshot cannot capture a consistent combination of session events, files, tool side‑effects, and external state.

Session affinity vs fault migration : Strong binding to a pod makes migration after failure difficult.

Scaling & upgrade difficulty : Teams avoid scaling or frequent upgrades to prevent state loss, leading to long‑term resource hoarding.

Verification of correct recovery : Pod health does not guarantee agent logical health.

A layered state system is proposed: session event log, long‑term memory, workspace (files & artifacts), checkpoint/snapshot (environment), and a scheduler that knows where to resume.

05 RL Training Scenario

MiniMax RL training launches thousands of environments (5‑10 × 10⁴) with pulse‑style sampling. Lightweight VMs were chosen over containers for three reasons:

Strong isolation boundary : Each VM has an independent kernel, containing any malicious behavior.

State & snapshot as scheduling capability : Whole‑machine snapshots enable fast copy, restore, and audit, turning an environment into a reproducible experiment unit.

Scheduling goal shift : From “start a container” to “make all environments ready”. The slowest batch of environments becomes the GPU bottleneck; a 1‑second VM start delay scales to millions of seconds of GPU wait time.

Benefits observed: improved scheduling throughput, clearer network isolation, and better resource isolation, ultimately reducing end‑to‑end training time.

06 CodeBuddy Production Practice

To achieve extreme cold‑start speed without compromising security, the team made several trade‑offs:

Pre‑warm the image and snapshot during build, so creation time only launches a prepared sandbox.

Scheduler considers data locality (image, snapshot, storage) in addition to CPU/memory.

Resource pre‑allocation is layered: hot cache for frequent templates, local cache for secondary, on‑demand pull for cold, and paused instances sink to low‑cost storage.

Pause‑resume uses multiple paths (image cold start, snapshot start, local pause, remote pause, disk mode).

Security governance stays outside the sandbox: credential injection, outbound control, audit, and rate‑limiting are handled by a gateway.

Result: cold‑start latency is achieved while keeping the security boundary intact.

07 Security Isolation Boundary

Lightweight VMs raise isolation from a shared kernel to a virtualization boundary, but risks remain:

Semantic credential risk : If an agent obtains a valid GitHub token, it can delete repositories; mitigation is short‑lived minimal‑permission credentials injected by the gateway and human‑in‑the‑loop confirmation for high‑risk actions.

Outbound data leakage : Even with VM isolation, agents may exfiltrate data; mitigation is default deny‑all outbound with whitelist, DLP, rate limiting, and audit.

Resource exhaustion & abuse : Multi‑layer quotas (sandbox, user/team, bandwidth, disk) and automatic circuit‑breakers prevent abuse.

Supply‑chain & image risk : Image admission, signature verification, vulnerability scanning, and whitelist of base images.

Snapshot & log sensitive data : Encrypt snapshots, tenant isolation, TTL, field redaction, and avoid persisting credentials.

Overall, VMs solve hard isolation, while gateway and governance layers close the production security loop.

08 Governance Capabilities

Agent Runtime provides built‑in governance:

Permission control : Models sandbox identity, user identity, roles, temporary credentials, API keys, OIDC; credentials are injected via gateway rather than placed directly in the sandbox.

Outbound access control : Sandbox is closed by default; all outbound traffic passes through a Zero Proxy/Gateway with domain/path/protocol whitelist.

Behavior audit : Logs from sandbox, tool calls, network accesses, and resource metrics are collected to trace which user, which agent, when, accessed what, and what result occurred.

Cost & quota governance : Manage concurrency, runtime, CPU/memory, storage per instance, team, or quota group; per‑second billing and free pause reduce idle cost.

High‑risk operation constraints : Combine with Human‑in‑the‑Loop; reads are auto‑approved, high‑risk writes, financial ops, or permission changes require user confirmation or policy approval.

These capabilities differentiate enterprise‑grade agents from simple demos.

09 Open‑Source Strategy

Cube Sandbox, the core isolation component, has been in production internally for two years and is now open‑sourced under Apache 2.0. The project aims to accelerate the industry’s agent infrastructure by supporting three scenarios:

Agent tool use : 60 ms startup, massive concurrency, high density for massive tool invocations.

Agent harness : Event‑level snapshot/rollback, credential management, traffic‑triggered pause‑resume, and seamless deployment on existing K8s.

Agent‑friendly services : Services become agent‑ready when run inside Cube Sandbox.

The team invites the community to co‑build the next generation of agent infrastructure.

10 Industry Landscape & Future Outlook

Framework layers will keep moving toward runtime, but production agents still rely on specialized infrastructure for isolation, state recovery, scheduling, outbound governance, and cost accounting. The author stresses that the deeper the production integration, the more agents must be treated as stateful, permissioned, code‑executing workloads.

Agent Runtime is Tencent Cloud’s unified infrastructure platform for Agentic RL, Agentic Agent, and enterprise‑grade agents, offering access, execution, governance, and intelligence layers. Its core execution unit is the open‑source Cube Sandbox , a high‑performance, sub‑100 ms startup, hardware‑level isolated sandbox released in April 2026.