Why Only 30% of an AI Agent Is Deployed – The Critical 70% That Determines Success

Part 1: From Demo to Production – Six Gaps

Most agents work well in demos but fail in production. The speaker defines an Agent as a goal‑driven execution system, not a chat model, with the formula Agent = 目标 × 上下文 × 规划 × 工具 × 反馈 . The six gaps are:

Gap 1 – Traffic spikes and throttling: Adding machines is too costly for GPU workloads; temporary API‑key load‑balancing is not a real solution.

Gap 2 – Effect volatility (“card draw”): Large models are probabilistic; random failures can even produce dangerous commands like rm -rf /.

Gap 3 – Black‑box reasoning: Unlike traditional services, agents hide the full reasoning chain, making debugging hard.

Gap 4 – Compute queuing: AI tasks can take seconds to hours, breaking traditional QPS‑based autoscaling.

Gap 5 – Knowledge‑security risk: Agents can call tools and access data without engineered safeguards, leading to privilege‑escalation and data leakage.

Gap 6 – High maintenance cost: Tool growth, model upgrades, drift, and memory loss cause exponential maintenance complexity.

These gaps stem from the lack of intelligent Agent infrastructure.

Part 2: Runtime Layer – Serverless Elasticity & Session Isolation

Traditional web services scale by adding CPUs, but AI requests last 3‑15 seconds and need GPUs, making that approach infeasible. AgentRun uses Function Compute (FC) to provide millisecond‑level auto‑scaling and automatic shrinkage, solving traffic spikes without idle compute.

Dynamic Memory Recall: Instead of full‑session replay (high token cost) or sliding‑window compression (information loss), AgentRun stores the entire session history and recalls only relevant context when token usage exceeds a threshold, guaranteeing information completeness with minimal compression.

Intelligent Sandbox + “拓Agent”: A dedicated sandbox Agent handles tool execution, reducing the main Agent’s token consumption. In browser scenarios this cuts token usage by ~92%.

Message Side – Million‑scale LiteTopic: RocketMQ LiteTopic provides per‑session queues, enabling independent consumption, fine‑grained rate limiting, and transforming asynchronous Agent collaboration into a scalable, low‑latency backbone. Single‑node throughput supports millions of queues with 10‑20 ms write latency.

Part 3: Collaboration Layer – Hierarchical Multi‑Agent Governance

Agents are organized like a company:

Manager Agent: Global planning, task decomposition, progress monitoring – does not execute directly.

Team Leader Agent: Coordinates tasks within a team.

Worker Agent: Executes tool calls and model inference.

This hierarchy prevents a single “Super Agent” from being overloaded, keeping context boundaries clear.

HiClaw adds asset governance: unified key management, skill security scanning (prompt poisoning, vulnerability checks), and zero‑code MCP Server conversion via Nacos‑registered services, ensuring that models never see raw credentials.

Part 4: Observation & Evolution – Data‑Driven Flywheel

Effective observation starts with LoongSuite, an open‑source collector covering Python, Java, Go, and Node.js, achieving 84% field coverage versus 31‑51% for competing products.

Effect Evaluation – Agent Paradigm vs LLM Paradigm: Traditional LLM evaluation scores only final output (65‑80% consistency) and cannot verify intermediate steps. The Agent paradigm uses an evaluator Agent that can retrieve evidence and assess planning, tool use, and memory, reaching 85‑95% consistency and reducing evaluation cost to 1/30 of manual checks.

Continuous Optimization: Two complementary paths – (1) expert‑plus‑AI tuning for rapid baseline gains, and (2) cross‑session self‑evolution that extracts best practices into a cloud knowledge base, cutting token consumption by up to 90% in tests.

Part 5: Vertical Validation – Intelligent Operations

STAROps demonstrates the hardest use‑case: operations. It introduces UModel , a graph‑based dynamic operations data model that resolves alerts within five hops, providing trustworthy context for AI reasoning.

Brain‑Hand Separation: Core reasoning runs in a “brain” Agent, while a sandbox executes actions, keeping the brain isolated from production.

Progressive Context Disclosure (VFS): Skills and tools are stored as files; the model only loads the four base paths on demand, avoiding context overload.

Long‑Running Tasks & Security: Dual‑Agent design (Planner + Executor) and hierarchical memory compression enable multi‑hour tasks. Architecture‑level token isolation prevents models from decoding Base64‑encoded keys.

Part 6: Engineering Takeaways

Launching an Agent is only 30%; the remaining 70% is a data‑flywheel of observation, evaluation, and optimization.

Correct, neither excessive nor insufficient, context is the single biggest factor for inference quality.

Security must be enforced at the architecture level, not just via prompts.

Sandboxing is a factual requirement for safe, scalable Agent execution.

Asynchronous, event‑driven designs are essential for scaling to millions of agents.

Intelligent governance transforms a stack of isolated capabilities into a cohesive, self‑evolving system.

The complete AgentOps solution therefore spans serverless runtime, lightweight messaging, hierarchical multi‑agent governance, asset security, full‑stack observability, and a data‑flywheel loop, turning a 30% demo into a 100% production‑ready AI agent.