Deploying Enterprise Agents with a Unified Harness, Skills, and Virtual Filesystem

Enterprise agents transition from demo to production faces three fundamental problems: what resources a task may access, how scattered domain expertise is incorporated, and where the state of multi‑step workflows is persisted. Without a unified answer, adding more models or tools only creates a powerful but ungovernable chatbot.

Why the Runtime Matters More Than the Model

A typical agent prototype consists of a model, a system prompt, a set of tools, and a loop that feeds tool results back to the model. This is sufficient for simple "fetch data then generate a report" scenarios but fails to answer production concerns such as checkpoint recovery, context explosion, cross‑domain data mixing, code execution location, prompt maintenance, and root‑cause attribution.

The solution is an Agent harness – a full runtime that orchestrates model‑tool interactions, manages session state and checkpoints, enforces tool exposure and human approval, loads skills, provides a file and code execution environment, and emits trace data for debugging and evaluation.

Two Unsustainable Deployment Patterns

"One scenario, one agent" – fast initial delivery but leads to duplicated logic, divergent retry policies, permissions, audit rules, and unclear ownership of prompts, tools, or models.

"Ship a coding agent to everyone" – powerful file, terminal, and code execution capabilities are exposed, yet knowledge workers need controlled business objects, read‑only data, and shareable reports, not unrestricted shell access.

Stripe’s experience with its No‑Code Agent Builder (over 4,000 agents) showed duplicated prompts, inconsistent quality, and maintenance pain, prompting a shift to a shared runtime with domain‑owned skills (Stripe Knowledge AI Platform – Kai).

Overall Architecture: Stable Core + Pluggable Domain Capabilities

The platform is split into four layers:

Entry layer – web chat, IM, data platform, ticket system, or browser extensions all call a surface‑agnostic API.

Control plane – manages rapidly changing assets: skills, agent configuration, default tools, evaluation suites, versioning, and quality signals.

Harness – provides unified identity, session scope, permissions, and infrastructure adapters.

Runtime – handles model calls, middleware, streaming events, checkpoints, and recovery.

The principle is that generic agent problems are solved once in the harness, while enterprise‑specific logic stays in the skill layer.

Minimal Responsibilities of a Production‑grade Harness

Identity and session management

Tool registration per skill

Permission checks and audit logging

Checkpoint creation and restoration

Trace generation for evaluation

Sandbox isolation for code execution

Skill: Turning Domain Experience into Governable Software Assets

A skill defines *what* can be done, *when*, *which tools* to use, and *what standards* to follow. A complete skill includes:

Metadata for discovery and routing

Execution steps, decision criteria, and failure branches

Declarations of tool, data, and runtime dependencies

Reusable scripts, reference material, and artifact templates

Positive/negative examples and evaluation suites

Owner, version, risk level, and change history

Example skill registry entry (YAML‑style) is shown in the article, and CI checks enforce naming, duplicate detection, missing owners, and risk‑policy mismatches.

Virtual Filesystem (VFS): Managing Long‑Running Context and Deliverables

Agents need to retain raw evidence, downloaded documents, query results, intermediate scripts, cleaned data, charts, draft reports, and final artifacts across many turns. Storing all of this in the message history leads to three issues: token cost explosion, difficulty locating the latest version, and inability for users to take ownership of outputs.

VFS provides familiar file operations ( ls, read, write, edit, grep) without requiring a real POSIX disk – paths can map to in‑memory state, object storage, databases, or remote workspaces. The layout is:

/sessions/<session-id>/
  scope.json          # snapshot of project, tenant, environment, permissions
  evidence/           # read‑only raw evidence files
  working/            # mutable intermediate data, scripts, drafts
  artifacts/          # user‑consumable reports, charts, documents
  checkpoints/        # persisted execution state
  manifest.json       # provenance, hash, owner, approval, delivery status
/skills/               # versioned skill definitions
/memories/            # cross‑session preferences and conventions
/shared/              # team‑shared assets after explicit publishing

Evidence files are immutable to prevent the model from “correcting” raw data. Working files can change frequently, while artifacts become official only after a manifest confirms validation, deployment, and acceptance.

Security Model: From User Permissions to Session‑Scoped Capabilities

Traditional enterprise apps authorize based on user identity. Agents add a delegation layer: a user authorizes an agent to perform a task, but the task should not inherit all user permissions automatically. Effective capability is the intersection of user authorization, agent configuration, selected skill policy, session scope, and tool‑side enforcement.

Four gates enforce this:

Tool visibility – only tools required by the selected skill are registered.

Parameter policy – the tool gateway validates tenant, project, environment, object, and operation type.

Execution isolation – shells, scripts, and document parsers run in per‑session sandboxes with network and resource limits.

Human approval – publishing, deletion, production changes, and cross‑scope reads require interruptible approval steps.

Permissions defined in Deep Agents (e.g., allowed-tools) apply only to built‑in file tools; custom tools and MCP services must enforce their own checks.

End‑to‑End Request Flow

A request such as "analyze last week’s rollbacks and publish a weekly report" follows a observable, interruptible chain:

Session gateway fixes the scope (project, tenant, environment) – the model cannot infer it.

Skill and tool candidates are narrowed in two stages: first a high‑recall filter (few hundred → dozens), then an LLM‑based precise selector.

Tool results become evidence files; the model only receives paths, summaries, line counts, and hashes.

Write operations are blocked until explicit approval.

The final response references the manifest.json rather than a free‑form model description.

Trace data is used to detect recurring failures (e.g., permission denials, tool timeouts, approval blocks) and automatically generate regression test cases and patch suggestions.

Phased Adoption Roadmap

Baseline assets and risk model – inventory existing agents, skills, and tools; answer who owns what, which entry points use them, data read/write scope, and success criteria. Produce a registry, lint checks, and evaluation baseline.

Integrate the unified harness – pick a high‑frequency, read‑heavy, low‑write workflow (e.g., weekly report) as a pilot. Route all entry points through a session API, register tools behind a gateway, create a session scope, and migrate results to VFS.

Two‑stage skill routing – initially measure pure LLM routing quality (top‑1/top‑k accuracy, token cost, latency, irrelevant tool exposure, high‑risk skill mis‑fires). When the catalog grows, add vector or classifier pre‑filters and compare metrics before committing to extra complexity.

Trace‑to‑skill improvement loop – use production traces to surface missing permissions, flaky tools, context gaps, or skill conflicts. Generate candidate patches, run isolated regressions, review with owners, and publish updates.

Metrics for Real‑World Success

Beyond call volume and user satisfaction, monitor four metric groups:

Task completion rate with clear denominator (e.g., 80 / 100 tasks that entered execution, excluding clarified or cancelled requests).

Top‑1 and top‑k skill selection quality.

Token cost per request (including skill/tool tokens).

Latency to first effective tool call and rate of high‑risk skill mis‑fires.

Maintain a control baseline to distinguish correlation from causation.

Common Pitfalls

Treating the harness as a monolithic agent – skills remain domain‑owned and load on demand.

Assuming allowed-tools alone provides full authorization – combine with MCP policies and sandbox constraints.

Using VFS as an infinite knowledge store without lifecycle management – enforce TTL, publishing workflow, and ownership.

Introducing sub‑agents prematurely – only split when deterministic scripts cannot be expressed in a single agent.

Testing only happy paths – include permission denials, tool timeouts, approval interruptions, sandbox exhaustion, skill conflicts, and stale evidence scenarios.

Conclusion

Enterprise agents gain lasting value not from the latest LLM but from reusable, owned skills, verified tool boundaries, recoverable execution state, traceable evidence, and a feedback loop that turns production traces into continuous improvement. The Stripe Kai case demonstrates that the key is separating the generic harness, domain‑specific skills, and a virtual filesystem, allowing the platform to evolve independently of model upgrades.

References

Stripe: "Meet Stripe's Knowledge AI Platform" – https://stripe.dev/blog/meet-stripes-knowledge-ai-platform

LangChain: "How Stripe Built Kai, its Company‑Wide AI Agent, on Deep Agents" – https://www.langchain.com/blog/how-stripe-built-their-knowledge-ai-platform

Deep Agents GitHub – https://github.com/langchain-ai/deepagents

Deep Agents Overview – https://docs.langchain.com/oss/python/deepagents/overview

Deep Agents Skills – https://docs.langchain.com/oss/python/deepagents/skills

Deep Agents Backends – https://docs.langchain.com/oss/python/deepagents/backends

Deep Agents Permissions – https://docs.langchain.com/oss/python/deepagents/permissions

Deep Agents Subagents – https://docs.langchain.com/oss/python/deepagents/subagents