Enterprise Agent Implementation: AI DESK's Architecture, GraphRAG & Field Lessons
The article details NetEase AI DESK's three-layer architecture (Agent Studio, Runtime, OS), knowledge engineering with RAG/GraphRAG/Wiki, AOP for execution, in-chat UI cards, human-AI collaboration, and field lessons on skill granularity, interface adaptation, and regression testing for enterprise Agent deployment.
Pain Points in Enterprise Agent Adoption
Enterprises face a gap between models answering generic questions and completing complex business tasks. Insurance example: a user asking "Has my policy expired?" triggers identity verification, policy query, compliance checks, and refund initiation — pure dialogue cannot deliver the business result. Models lack private context (org structure, CRM, order systems, security rules). Last year's intent-recognition-plus-fixed-workflow Agents are already outdated. Translating business boundaries, exceptions, and permissions into model-executable language remains a bottleneck (high FDE — Field Deployment Engineer — barrier).
Strategic Choices
Private deployment within enterprise authorization, connecting core systems, data, and knowledge.
Productized subscription with continuous upgrades so technology and business changes flow into the product service.
Three-Layer Architecture
Agent Studio (Orchestration Layer)
Defines task capabilities via Agent Profile: configures knowledge, memory, Skill, tools, and business procedures.
Agent Runtime (Execution Layer)
Self-developed unified kernel organizes task context, invokes business capabilities, advances tasks, and verifies results.
Agent OS (Unified Control Layer)
Analogous to an OS: resource scheduling, state management, permission control, audit logging.
Five Standard Closed Loops for Business Processes
Example: policy processing decomposed into:
Identity verification — integrate CRM to auto-identify asker and assets.
Compliance check — multi-dimensional validation against product terms and claim rules.
Reasoning & decision — LLM infers next action path, asks clarifying questions.
Execution — perform business operations via secure interfaces.
Audit feedback — record every reasoning step and execution log for 100% traceability.
Knowledge Engineering: Three Organization Methods
Real insurance query: mainland female client buying in Hong Kong, husband also policy role — involves product, identity, age, relationship conditions. Single semantic recall misses requirements. Three methods used together:
RAG chunks — locate relevant paragraphs, clauses, regulations.
GraphRAG knowledge graph — express entities and relationships, support tracing associated conditions.
Wiki topic pages — organize related knowledge into directories, attributes, common phrasings.
Search Agent loop : decompose verification items, select retrieval sources per question, read original text, check each item for evidence, supplement retrieval if gaps. Summaries help discover information; original text supports final conclusions. Retrieval sources and loop counts configured per task, not always three-way concurrent.
Execution Division: Skill vs AOP
Early approach: one Skill per sub-scenario. Skills grew heavy with exceptions and multi-interface calls. Pre-defined large flows hard to maintain (clarifications, supplements, ad-hoc branches). Solution: AOP (Agent Operating Procedure) .
Skill handles intent understanding, planning, task progression.
AOP organizes rule-based business steps.
Enterprises define independent processes as procedures. Execution can wait for user input, hand off to human, or resume from saved state. Code nodes manage details, reducing need to stuff entire flow into model context. Permissions, exceptions, result verification still implemented per task.
In-Chat Business Cards (A2UI)
Pure text interaction hurts UX. AI DESK encapsulates business components as structured cards ("AI mini-programs") via A2UI protocol. Demo: policy maturity pre-registration — user selects policy, confirms currency/payment method, adds bank account and proofs, completes identity verification, submits application. Conversation handles communication; cards handle filling/confirmation; business system tracks subsequent approval.
Human-AI Collaboration
Decision accountability remains clear. Modes: human-led with AI suggestions; Agent-led with human assist at high-risk/exception points; human takeover for complex issues. Refund example: Agent collects materials and verifies info; human confirms approval/adjustment per rules. Handoff includes summarized context to avoid re-explanation.
Legacy System Integration
Keep existing CRM, ticketing, call systems. Add AI for: online chat assistance, ticket drafting, ASR-transcribed call assistance, sales visit suggestions using CRM data. AI analyzes/drafts; human confirms/executes per rules; legacy systems retain responsibilities. Requires solving interface, permission, and result-writeback conditions.
Field Lessons (FDE Experience)
1. Skill Bloat
Single Skill accumulating all responsibilities becomes heavy. Split: Skill for understanding/planning; AOP for execution. Split based on business, not "more Skills = better".
2. Interface Adaptation Beyond Connectivity
Legacy interfaces return irrelevant fields. Tool integration needs pre/post processing: trim parameters, crop responses to task-relevant info. This common requirement becomes reusable product capability.
3. Regression Testing Across Shared Skills/Tools
Multiple Agents share Skills/tools. Tuning scenario A may break scenario B. Establish evaluation baselines covering normal, exception, edge cases. Run regression after each change. Post-launch: user feedback and human corrections feed analysis, locate issues in knowledge/Skill/procedure, fix, re-evaluate, confirm no regression, then proceed. One service's experience must be verified before becoming reusable capability for the next task. General improvements return to AI DESK product; implementation methods, configs, verification experience go into DeskOps delivery platform to reduce reliance on scarce FDE talent.
Live Q&A Highlights
Q1: Skill Granularity Trade-off
Too fine → intent mismatch; too coarse → poor execution. Core reason for AOP: Skill focuses on intent understanding and macro planning; tool calls, multi-interface chaining, exception handling sink into AOP code nodes. Keeps flexibility, removes LLM context burden.
Q2: Skill Script vs AOP
Key differences: maintainability, cross-scenario reuse, interruptible jumps . Deterministic 20-node flow in script must run end-to-end; AOP supports on-demand jump in/out/suspend. Common logic (surrender, identity verification) shared across scenarios as independent AOP assets, avoiding duplication in every Skill.
Q3: Three-Modal Knowledge Retrieval Latency vs Accuracy
Scene-tiered strategy:
Cognitive match : complex consulting — users tolerate "deep thinking" latency.
Configurable policy : DeskOps configures per scenario (max loops, concurrency, Graph vs RAG priority).
Latency-first trade-off : millisecond-response scenes reduce loops or simplify to single recall, balance accuracy/speed per business ROI.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
