LiteLLM Agent Platform: K8s Sandbox Stops Agents Accessing Real API Keys

The LiteLLM team released an open‑source LiteLLM Agent Platform that tackles a frequently overlooked security risk for coding agents: accidental exposure of API keys.

Core Security Mechanisms

Session‑level sandbox isolation : each agent runs inside a brand‑new Kubernetes pod that is destroyed as soon as the session ends, leaving no persistent storage and no cross‑session contamination.

Outbound credential substitution : the agent environment only contains stub tokens (e.g., GITHUB_TOKEN=stub_github_a8f1). When the agent makes a TLS request, a credential vault replaces the stub with the real secret, so the agent never handles the actual key.

Key Features and Compatibility

Supports multiple agent frameworks—Claude Code, Codex, Hermes—without requiring changes to existing toolchains.

Deployable in various environments: local, AWS EKS, GCP GKE, Render.

Terminal‑direct sandbox access via the lap CLI, which attaches a local terminal to the remote sandbox’s TUI over WebSocket.

Architecture Diagram

Quick Start – CLI Experience (No Self‑Hosting Required)

# Install lap CLI
git clone https://github.com/BerriAI/litellm-agent-platform.git
cd litellm-agent-platform/cli && npm install
ln -sf "$PWD/bin/lap.mjs" ~/.local/bin/lap

# Log in to the platform
lap login

# Launch a Claude Code sandbox
lap claude-code-cli

Running the command creates a fresh K8s pod; the local terminal attaches to its TTY. Press Ctrl-D to disconnect; the session is retained for 24 hours by default.

Self‑Hosted Deployment

Local development : use a Kind cluster and run bin/kind-up.sh + docker compose up to start the platform, then access localhost:3000 to create agents.

Production deployment : AWS EKS is recommended for the sandbox cluster, with Render hosting the web and worker components. Deployment scripts and configuration live in the repository’s deploy/ directory.

Repository

https://github.com/BerriAI/litellm-agent-platform