Tmall's AI Digital Employee: Full-Chain Delivery Architecture Handling 100+ Monthly Requirements

Tmall Technology has built a unified AI end-to-end delivery architecture called "Digital Employee" (数字员工). It resides in DingTalk group chats as a stable collaboration entry point, sharing requirement context with product managers, frontend, backend, and QA engineers to enable relay delivery from PRD to release. The system employs a main-sub agent structure combining deterministic workflows for stable stages (review, test, release) with dynamic planning for variable tasks, runs in isolated sandboxes per requirement, and enforces strict identity delegation for security. As of July, it has been rolled out across all Tmall business lines with 100+ monthly active deliveries.

Collaboration Form: Embedded in Teams, Not Assigned to Individuals

Unlike typical AI coding tools that improve individual efficiency but leave organizational delivery bottlenecks (requirement clarification, cross-role alignment, knowledge silos), Digital Employee sits in the collaboration scene — the requirement group — where all roles share a single context. The system uses deterministic processes for well-defined stages (review gates, deployment) and dynamic planning for variable work (analysis, coding). A main agent judges intent and dispatches sub-agents (requirement analysis, coding, testing) or skills/tools; once a stable stage is entered, explicit workflows and gates take over.

The article contrasts two product shapes: (left) each role commands its own agent squad — digital PM, multi-agent coding squad, digital role squads — where context remains locked in separate windows; (right) one digital employee in the requirement group — intake, alignment, coding all happen in front of everyone, context stays in one place. Tmall chose the latter: a single stable identity facing the team, though behind the scenes it invokes different sub-agents per task. Private chats route to personal sandboxes; group chats route to group sandboxes, isolated from each other. Autonomous delivery rate measures whether a requirement advances continuously within defined processes and gates without human correction, rejection, or restart; planned human nodes (design judgment, authorization, review confirmation, final acceptance) do not count as interruptions.

Full-Chain Delivery Capability

3.1 Within One Requirement: Cross-Role Relay

A requirement involves four roles — product, frontend, backend, QA — each with direct input/output:

Product : receives one-sentence requirement, discussions, review comments; returns structured PRD and clarification list. Rewrites PRD by business domain when criticized for missing logic.

Frontend : receives interaction confirmation and joint-debug info; returns page changes, build/deploy, diff preview.

Backend : receives interface contracts and design review conclusions; returns technical design, service changes, pre-release deployment.

QA : receives acceptance criteria and defect feedback; returns test cases and self-test results. Quality gates (static check → build → auto test) plus Harness browser verification run throughout; test cases carry full context from requirement clarification.

Coding execution reuses existing tools (Claude Code, Codex CLI, Tongyi Lingma); Digital Employee manages context feeding and progress collection. A real example: the "Ask Bottom Price" feature iteration was handled continuously over two days by one digital employee, with humans appearing mainly at confirmation and acceptance nodes.

3.2 Within One Scenario Group: Out-of-the-Box, Domain-Customized

Scenarios split into non-process (ad-hoc Q&A, doc reading, data query) and standard process (design, coding, deploy, iteration create to release). Digital Employee uses a flexible main-sub structure: main agent handles non-process collaboration; sub-agents handle process delivery . A default runnable path is provided: main agent pre-loads generic capabilities (requirement intake, DingTalk doc/sheet read-write, DB ops, table understanding, iteration/release creation); default coding sub-agent covers analysis, design, coding, test, release with explicit entry/exit/review points. New domains run without customization first, then extend via Skills — delivery phases, outputs, review gates defined by business teams. In one API-assembly scenario, the 9-stage flow compressed requirement-to-design to minutes. Configuration lives in a domain admin page where scheduling instructions drive main agent dispatch, sub-agent lists are generated per domain, and associated Skills are projected to executors at dispatch time. Capabilities configure per domain: Skills, Hooks, System Prompt, and linked knowledge bases let the same digital employee acquire domain-specific knowledge and execution ability.

3.3 Across One Business: Kanban and Proactive Drive

After a requirement enters the group, Digital Employee maintains a kanban: auto-records members, milestones, blockers, artifacts, and updates status from group discussion and delivery progress. Group members see a header card; cross-requirement view uses a web dashboard (four-column status flow, one card per requirement, synced with the ticket system). The kanban makes wait states explicit. Digital Employee syncs progress to group and dashboard, @-mentions responsible persons at confirmation/acceptance, escalates on timeout, and posts a daily summary showing current status, next actions, and pending items — with trend changes, blocker owners, and dwell times. Members can reply "expand item 2" to drill down. Kanban, escalation, and daily summaries solve visibility and follow-up; proactive problem discovery and collaboration initiation are slated for a future "Autonomy" article.

System Teardown: Message, Sandbox, Agent, Identity

Core call chain:

DingTalk message → Message Gateway → route by identity to dedicated sandbox → Main agent judges intent → dispatches sub-agent or answers directly → DingTalk AI Card real-time feedback

. Architecture diagram shows DingTalk and Web entry points converging on one gateway and sandbox; business data unified via data service; model calls exit via unified gateway.

4.1 DingTalk Access and Message Routing: From Group to Sandbox

Uses DingTalk official Stream mode: gateway holds long-lived connections, events pushed down, no public callback needed. Monitors "last event received time" to detect silent connection failures. Three event types: private chat, card callback, group message. Scenario group = requirement group; after creation, digital employee sends first message, then all group messages flow to gateway — @-mentioned go to response, others to passive listening. Stable @-detection solved by maintaining a bot-identifier set: confirmed IDs match directly; uncovered cases fall back to text rules and learn at runtime. Routing rule: one routing identity maps to one sandbox . Private chat routes by sender identity; scenario group derives group identity from conversation ID — one group, one box. Group and personal sandboxes stay isolated. Multiple instances receiving same event perform atomic claim before forwarding.

Gateway is deliberately "thin": no LLM calls, no business logic. Three deterministic duties: receive (immediate ack, atomic claim avoids duplicate processing), retain (images/files downloaded and stored within credential validity), route accurately (identify routing identity, select sandbox, assemble into runtime format). Business scheduling must go through Digital Employee, not gateway, because: (1) user triggers requirement analysis in group — gateway keyword launch would lack unified state/feedback owner; (2) user clicks "start coding" after analysis — gateway parsing card callback wouldn't know analysis result; (3) user sends unrelated message during coding — gateway cannot decide forward vs ignore. Digital Employee centralizes these decisions. Responses use "create card first, then overwrite": create placeholder card to get track ID, then continuously update same card — avoids long silence or multiple cards on cold start. Each message generates a trace ID covering box selection, delivery, callback, card push; dashboard drills to single-message level. Metrics (as of July): hot-path first response ~1s avg, P90 ≤3s; cold-start first response P90 ~6s; first-response success rate >99%. Only conversation-driven initiation supported: user sends "analyze this requirement" or "optimize utils.ts" in DingTalk group or private chat; digital employee matches repo from session/project list, asks for confirmation if ambiguous, then dispatches analysis or coding agent. During execution, agent streams events; digital employee throttles AI Card updates and pushes real-time events for detail-page subscription.

4.2 One Requirement, One Sandbox: Container, Init, Memory

Message routing ends at sandbox: one requirement → one scenario group → one isolated sandbox; private chat uses personal sandbox. Different collaboration scenes isolated — code, deps, debug state of one requirement never affect another. Image contains only base runtime; runtime code pulled from release service at startup, so releases don't rebuild images; online sandboxes upgrade in batches: daemon pulls new version, switches symlink, restarts runtime when idle. On create, gateway injects identity, credentials, domain config; multiple repos cloned in parallel per requirement, tech stack auto-detected. Cold start optimized from two sides: (1) pre-warmed pool of ready sandboxes — first message triggers one-time claim, binds before accepting business messages; (2) state that must survive placed outside container to avoid loss on rebuild. Pipeline from create → inject → pull runtime → prepare repos → ready → first greeting in group. Runtime: each message forward refreshes sandbox TTL to prevent platform reclamation; on forward failure, query actual container state then rebuild/wake/wait. State design: must-remember persists; rebuildable does not persist .

Must keep : Conversation, memory, task state — stored in independent persistent volume, remounted on container rebuild.

Rebuildable : Repo code, temporary execution context — not in persistent volume, regenerated on use. /reset only resets current runtime state, not persistent data, leaving recovery room. Session design supports cross-device: Web workbench reuses instance-associated session — pulls snapshot to replay history, subscribes event stream for live progress, messages merge into same sandbox as DingTalk. A requirement discussed halfway in DingTalk continues seamlessly on Web.

4.3 Agent Layer Openness: Controllable Scheduling, Pluggable Execution

After message enters sandbox, a one-main-many-sub agent structure processes it. Main agent = digital employee core, resident thread in custom agent framework; sub-agents (requirement analysis, coding, test) dispatched on demand. Design principle: genericity belongs to execution layer; controllability belongs to scheduling layer . Why not use a general coding agent as the collaboration hub? Coding agents excel at well-scoped repo tasks; digital employee hub must also manage message routing, structured interaction, multi-session state, cross-role collaboration. Comparison: custom main agent natively controls DingTalk AI Card, schedules coding agents with explicit context/start/recycle, drives structured cards from business state, manages multi-session state explicitly (by group/user/requirement), and routes deterministic messages via code without LLM — whereas a coding-agent-as-hub would need adaptation layers for each.

Main agent aims for controllable behavior: external actions (send message, update card, dispatch task) via explicit tools. But not a rule engine — contextual judgment still needed for messages like "done yet?" or "ignore that, check prod error". Sub-agent list generated dynamically per domain, loads new config at turn boundaries without thread rebuild. Sub-agent layer stays pluggable: same logical agent can use custom executor or Claude Code etc., chosen by domain. Three engineering constraints: no mid-session engine switch; streaming events from different backends unified; all model requests must pass unified gateway (refuse start if gateway unconfigured). Main agent passes domain config to sub-agent — called capability projection — in three steps: (1) load at dispatch : Skills, MCP, Hooks converted to executor native format; (2) query during execution : sub-agent bridges back to group to ask when info missing; (3) collect at completion : execution events return to main thread, main agent updates state and feeds back result. Registration diagram shows dispatch/query/result all follow registered protocol; default sub-agents initialized and customizable; custom agents register by implementing run/resume — carrier pluggable, spec unified. Real main-sub conversation screenshot shows main agent marking user utterance as "task background only", then issuing the actual instruction for the sub-agent.

Message routing split came from pilot feedback: "deployment polling occupies session, my new messages queue". Fixed by adding a deterministic pre-route (no LLM): if sub-agent awaits reply, forward directly; else enter reasoning path where main agent decides answer/forward/new-task. Example: analysis agent asks "which login methods for login page?", user replies "phone OTP and DingTalk QR" — destination already known, no need for another model call. In agent system cost structure, identifying steps that don't need LLM calls is as important as improving model call efficiency. User sees two layers: group card shows stage/status/current executor; detail page shows continuous execution flow. Task enters sandbox after repo/branch location, then execution mode by complexity:

Simple : Lightweight execution, direct verification after completion.

Medium : Single agent executes and self-checks.

Complex : Still single coding agent phased execution, adds design confirmation, self-check, release gates; continuous state feedback, human corrects or restarts on exception.

Controllable ≠ fixed script. Main agent can adjust execution within authorized capabilities: e.g., if sub-agent stalls, replan task, continue with own tools; but subsequent verification, permissions, release gates stay fixed. Reasoning handles variation; engineering constraints guard boundaries. Layer open to business: without platform code changes, domain admin can extend main agent and three default sub-agents (analysis, coding, test) with System Prompt, Skills, MCP, Hooks. Teams with existing agents implement run/resume protocol to register as sub-agent. Group still sees one digital employee; backend executor swappable. Platform retains default analysis and delivery agents so new domains have runnable baseline. Executor changes don't alter identity boundaries — data permissions follow current-turn user.

4.4 Identity and Permissions: Who Consumes Uses Whose Rights, Who Produces Owns Result

In one group, different users ask digital employee to read docs, push code, deploy, query controlled resources. Single identity would either lack permission or over-grant. Principle: who consumes data uses whose permission; who produces result gets attributed .

Read/write DingTalk docs/sheets : Current-turn sender — only accesses data this person originally can access.

Build, deploy, platform login : Digital Employee service identity — no borrowing personal accounts; platform actions independently auditable.

Commit code : Current-turn sender as author — output attributed to actual commander; personal credentials never persisted.

Query middleware, monitoring, data platform : Identity delegated by group owner via delegation service — resource side still judges by owner's original permissions; no extra privilege.

Key point: model cannot specify arbitrary person . Identity-bound calls only accept controlled roles: "current-turn sender", "digital employee service identity", "confirmed group owner". Role-to-real-identity mapping injected by trusted environment; even if model generates "as user X query", control layer rejects. Controlled resources require explicit authorization: after group designates tech lead, only that person's login confirmation activates delegation in authorization service; request logs delegating user, executing agent, source sandbox. Authorization system only proves "who delegated to whom", never expands original resource permissions. Credentials encrypted, renewed on demand; doc/code/platform identities not auto-synced or amplified by delegation. Delegation chain sequence diagram: group initiates → owner login confirms → auth material injected into sandbox via control plane → resource side still judges by owner's original permissions. Constraint: within one sandbox lifecycle, "user + agent" identity combination stays constant . Changing owner destroys old sandbox, cold-starts new one, forbids restore from old snapshot so previous identity's cache/credentials don't leak. Implemented as a state machine: proposal → activate → separate → re-authorize → revoke — binding of user, agent, sandbox must be consistent to be ready; authorization revocable, renewable, auditable.

Effectiveness Measurement and Landing Data

5.1 Measurement: Calibration and Attribution Method

Observability split into Stable Experience (service quality, user retention) and Delivery Effectiveness (AI adoption depth, output efficiency, input cost). Same data, different views: platform sees global; domain sees own. Effectiveness focuses on two signals: Autonomous Delivery Rate and Human Intervention Count , with matrix and ledger built around them. Traditional metrics (code acceptance rate, usage coverage) already high and cannot distinguish single-prompt generation, dev-loop closure, or full requirement delivery. Pure duration stats distort: dev tasks run in parallel, waits/blocks often outside dev chain. Two rulers used: Requirement Complexity C1-C4 (7 dimensions: core mid-platform impact, product form, cross-team scope, etc., take highest) and AI Delivery Depth L1-L4 (prompt-driven → dev closure → test closure → full-chain delivery). Outside C×L grid, a parallel reference signal: Human Intervention Count — after first task launch, human continues to supplement, clarify, chase, or correct. Split into Collaboration Layer (main agent group discussion, clarification, review) and Delivery Layer (supplement/correction after entering coding sub-agent). Used for observation and attribution only, not a "lower is better" target — necessary questions, judgments, reviews are part of delivery; gaming the count by delaying feedback or merging messages makes numbers pretty but not delivery smoother. Different from autonomous delivery rate: planned judgments/confirmations recorded as participation rounds but don't mark delivery "non-autonomous"; only anomaly correction, rejection, or process restart affect autonomous judgment. Cross-view diagram shows C1-C4 rubric (take-highest), L1-L4 definitions, and matrix cells with "requirement count + avg intervention count" for improvement targeting.

5.2 Operational Dashboard: Scale, Depth, Intervention Location

As of July, all Tmall business lines onboarded: 100+ requirements in pool, 20+ domains; last 30 days new requirement groups and active sessions both >100. Every stage (communication, design, review, dev, test, release) has in-flight requirements. This shows real production run, not scale substituting effectiveness. First 10+ pilot requirements manually reviewed — sample too small for overall efficiency proof but locates human participation: most records in Collaboration Layer (group discussion, clarification, review), typically 10-30 rounds per requirement, including planned collaboration, not process failures. Delivery Layer records supplements/corrections after entering coding sub-agent: most requirements a few to 20 rounds; completed coding requirements average ~7 rounds; early-stable domains down to ~1 round. Full-chain participation ~30 rounds, ~20% rework. Optimization focus shifted from "can it generate code" to "reduce repeated confirmation and info supplementation in collaboration". Domain-level attribution dashboard shows delivery-layer vs full-chain dual scales, interventions split by collaboration/delivery. Drill-down to per-requirement intervention breakdown (collab/delivery). Further attribution: each intervention tagged (chase progress, resolve blocker, confirm decision, raise requirement, fix rework, supplement context) with risk signals (context missing, understanding bias). Generates three improvement suggestion types (pre-authorization, prompt, skill); after human confirmation, sunk as domain decision assets. Intervention count thus becomes input for next optimization cycle, not just result metric.

Conclusion and Outlook: New Execution Unit for AI-Native R&D Organization

Viewing Digital Employee as just a group coding bot makes the message routing, long-term state, isolated sandboxes, identity permissions, and main-sub agents seem over-engineered. Together they define a new organizational execution unit: stable identity, continuous context, own work environment and permission boundaries; can receive requirements, invoke tools, produce artifacts, maintain state, and pull humans back at judgment points. This is Tmall's understanding of AI-Native: not adding AI features at every existing process node, but redefining responsibilities among humans, AI, and systems. Previously humans ferried context, pushed status, bridged across systems; going forward digital employee continuously holds context, owns deterministic execution and process drive, while tickets, docs, pipelines auto-flow in background. Humans retain judgment, authorization, accountability, but no longer carry every information move and status poll. When every requirement has such a long-running execution unit, R&D collaboration shifts from serial handoff to async parallel: dozens or more requirements advance simultaneously; humans don't track each agent's work step by step, but engage at high-value moments — design choice, risk confirmation, exception handling. The order-of-magnitude change may come not from one coding task being faster, but from the same human attention supporting far more parallel deliveries.

Today's system is early on this path. Easiest to run end-to-end are requirements where interfaces and base capabilities exist, mainly assembly; complex domain refactors still need human intervention at requirement judgment, design choice, risk confirmation. Next phase isn't simply adding more agents, but completing two foundations: (1) Business mindset — distill rules, definitions, precedents from brains, chats, history into consumable organizational assets; (2) Proactive collaboration — let digital employee discover issues, initiate collaboration, drive closure within authorized and restrained boundaries. These are the next two articles in the series. Digital Employee won't auto-fix unclear PRDs, tacit coordination, or uncodified business knowledge. Instead, those problems surface earlier in delivery: insufficient input blocks progress; unclear ownership blocks routing; missing gates block go/no-go decisions. Building Digital Employee is the process of making explicit every part of the R&D system that previously relied on humans to catch. Diagram illustrates shift: from "humans run around systems" to "systems run around humans" — tickets auto-flow, status auto-write-back, collaboration returns to conversation scene. Not removing existing systems, but making systems operate around the real collaboration scene. AI-Native endpoint may not be everyone using more AI tools, but the organization gaining a class of digital colleagues that can be authorized, collaborated with, audited, and continuously learn the business. When they truly become nodes in the delivery network, R&D efficiency ceiling no longer depends on how fast one person codes, but on how many things the organization can push in parallel while keeping scarce human judgment at the moments that matter most.