Why Some AI Agents Reach Production While Others Stall at Demo: The Harness Layer Difference

The article summarizes 12 presentations from DACon 2026 Beijing, all converging on a single insight: the difference between AI agents that reach production and those stuck in demos is not the underlying model but the Harness layer —the engineering stack between the model and real business that enforces constraints, injects knowledge, and guarantees deterministic execution.

Section 1: Defining Harness — What Constraints Belong in the Framework

Runhe Software (Zhan Lingli) frames three adaptation paths for vertical domains: Harness Engineering (framework-side control), RAG/Prompt (knowledge injection), and post-training (model-side solidification). Using a GUI test execution agent as a running case, she identifies 10 framework intervention points (context engineering, tool-calling management, fallback mechanisms) and a decision matrix: classify the problem as knowledge-gap, capability-gap, or stability-gap, then verify feasibility with Harness before committing to costly post-training. Concrete challenge: Harness complexity runaway leads to unmaintainable patch stacks; solution: modularize the 10 intervention points into configurable components. Post-training data scarcity is addressed via human-AI collaboration + trajectory replay to mine positive/negative samples from real execution traces, with rule-based auto-labeling plus human spot-checks. Evaluation uses end-to-end task success rate as the north-star metric, supplemented by single-step accuracy, retry count, and latency.

JD Technology (Meng Chuishi) argues the fundamental difference between a general-purpose agent and a business agent is the constraint layer , not model capability. He abstracts Harness into three constraints: Role (system-prompt charter fixing identity and refusal boundaries), Knowledge (business graph, metric library, case library, copy library served as retrieval-augmented services), and Action (business primitives + deterministic checks on resources, permissions, ports, ratios, timestamps — enforced by code). A guiding principle: side-effect gradient — the closer an operation is to send/publish/write, the more it shifts from generative to verification mode, with code as the final gate. Deployed in chatMA (marketing strategy agent) and JoyAnalysis (data analysis agent), turning business goals into verifiable, replayable, lineage-tracked artifacts. Biggest cost: making tacit knowledge explicit (role charters, business graphs, metric libraries) via one-on-one extraction from senior staff.

Lingyue (Guo Zhihao) tackles the Skill vs. Agent boundary on a data platform. A real SQL case: the query parsed, executed, and returned results, yet the business conclusion was wrong. Root cause: executable ≠ business-correct . He introduces an autonomy decision table that classifies every request into four lanes — deterministic workflow, constrained agent, reject, or human confirmation — based on goal clarity, verifiability, risk, and reversibility. Architecture uses a "four layers + one cross-cutting plane" : knowledge/context layer (versioned business definitions), skill layer (reusable SQL generation/validation methods), workflow/agent decision layer (workflow handles codable state branches; agent only handles local judgments needing runtime feedback), tool execution layer (metadata query, sandboxed data exploration, SQL validation), and governance plane (identity, parameter policies, budgets, approvals, termination, audit, observability). Key tenet: natural-language constraints cannot serve as safety boundaries ; governance must be enforced by external systems.

Yiche (Wang Linhong) traces evolution from ChatBI → DataAgent → DataWork, building a three-layer foundation: trusted data, semantics/ontology/knowledge, and MCP+Harness. Highlights enterprise-level Harness system construction as a dedicated chapter. Challenges: keeping agent-ready knowledge fresh long-term and achieving system auto-evolution.

Section 2: Harness in the Production Pipeline — End-to-End Orchestration

Dewu (Lin Feng) identifies context fragmentation across stages (requirements clarification → AI coding → AI testing → notebook analysis) as the root cause of inefficiency, not model weakness. They chose Harness-controlled orchestration over single-point copilots, fixed workflows, or free multi-agent collaboration. Architecture: layered foundation — agent understanding & scheduling, skills codifying expert processes (steps, checks, failure branches, human gates), MCP/CLI for read/write execution, web/notebook entry points, plus session memory and tool convergence. Human-in-the-loop gates: IDE review (human approves diff before commit), STOP points (human confirms table selection). Tool boundary: metadata exploration via MCP, execution converged to CLI to prevent tool sprawl. Four agents share one Harness: clarification agent (intent, smart table discovery with STOP, quality gates), AI coding agent (confirm model → create table → create task → dependency validation → store dev draft only; human reviews diff in IDE before commit counts as AI commit), AI testing agent (AI code review + AI data validation; smoke/baseline/DQC + SQL pre-compile quality gates; offline/Spark diagnosis follows evidence→hypothesis→human verification chain), notebook analysis agent (AI writing assist + conclusion reuse). Engineering hard points: multi-end context consistency (tools and memory funneled through Harness; human pins tables, spaces, reviews diffs to pin state), context budget (filter first, spill large details to disk), project space must be human-selected, never inferred from schema names.

ZTE (Lu Fangfang) reports ≥6× requirement development efficiency , ≥70% code adoption rate , 60%+ fault diagnosis adoption, 50%+ diagnostic adoption, 30% overall productivity lift. Background: 70% of demands consume 90% of effort in traditional serial, siloed, cross-functional relay. Target: shift from multi-role relay to single-node full-stack closed loop via AI agents. Seven-stage progressive workflow : requirements analysis → solution design → coding → compile/debug → test verification → code review → summary/submit. Techniques: progressive prompt disclosure, layered resource loading, context optimization. Automation: compile self-healing (error log analysis → localization → patch generation), test self-healing (FT case failure diagnosis, logic defect identification, adaptive adjustment), AI-assisted code review with human focus. Operations: "lights-out factory" self-diagnosis pipeline (fault collection → domain identification → code lock → intelligent diagnosis → result write-back) and self-healing pipeline (online fix → auto test → code submit), targeting 30% self-diagnosis, 10% self-healing. Scale: 230+ component penetration cases, 30 participants; skill resource pool operation (high-frequency scenario immersion, version management, continuous evolution); integrated platform dashboard for visibility and traceability.

Peking University DCAI (Qiang Meiyi) presents DataFlow (7.2K GitHub stars) and DataFlow-Harness for NL2Pipeline . Problem: LLM training/fine-tuning and RAG knowledge base construction need massive data cleaning, quality assessment, structural conversion, semantic enrichment, and pipeline reuse — traditional scripts don't scale; single-point NL2SQL can't cover complex data prep flows. DataFlow uses pipeline + operator orchestration for reusable, large-scale AI-ready data production. DataFlow-Harness closes the NL2Pipeline gap: user describes data prep goal in natural language → WebUI or CodingAgent + MCP/Skill calls translate to executable, debuggable, reusable data processing pipelines. Industrial deployments with SenseTime, Tencent, PKU, State Grid, Coupang, Kuaishou.

JD Data (legend) builds an AI-Native Data OS oriented to deliverables (physical tables, logical tables/metrics, dashboards, A/B experiments). Shift from human-facing to AI-driven full process; humans focus on key decision nodes; AI bears no accountability. Tech: cloud-native, MCP+Skill for standardized atomic ops, sandbox for security, multi-model adaptation. Roles: DA (consumer), DE (producer), DG (architecture quality guardian) — boundaries defined by system, not human friction. Knowledge base: layered, categorized, with positive/negative signal feedback forming a "smarter with use" flywheel. Legacy compatibility: pragmatic GUI+API dual-track, not rip-and-replace. Called out three hard problems: data change Harness design, multi-loop loop mechanisms, organizational intelligence flywheel design . Real production pains shared: file write chaos, context loss of control, exception messages mismatching root causes.

Section 3: When Harness Must Guarantee "No Mistakes" — High-Stakes Domains

JD Health (Hu Haoyuan) : Medical AI moves from Q&A assist to multi-step task execution where every output must be deterministically correct . Four dilemmas of dropping general LLMs in: uncontrollable probabilistic retrieval, reasoning models poor at execution, generic frameworks can't meet domain determinism, missing closed-loop quality assurance. Core thesis: when AI must be deterministically right, redesign not the model but the entire engineering stack from model → knowledge → engineering → delivery . Four-layer reconstruction: Model layer — upgrade reasoning model to execution-grade medical code LLM; code as task execution carrier, making output verifiable and traceable. Knowledge layer — drop probabilistic RAG; build deterministic Agentic Search + structured knowledge base LLMwiki. Harness layer — custom directed Harness (MedWork) for task orchestration and critical medical logic validation. Delivery layer — evaluation, execution logs, error recovery, human review forming delivery closed loop. Three-stage evaluation: model scores, task success, actual adoption. Methodology transfers to finance, government, manufacturing; bedrock is data semantics, knowledge structure, context organization.

Kurui International (Wang Xu) : MiraDay recruitment agent. Counter-intuitive finding: delivery quality bottleneck is not model capability but fulfillment rate . Capability ceilings converge fast; fulfillment rate is the real battleground. Built MiraBench for deployment-time alignment and treated "stop" as the single largest loss source in fulfillment , handling it specially in Harness. Result: MiraBench composite score 40.1 → 75.2 . Challenges: open business scenes lack stable ground truth; judges drift. Solution: combine deterministic assertions, business constraints, model judgments, real user behavior, continuously calibrated with human gold standard. Harness boundary tension: too few constraints → no delivery guarantee; too many → damages autonomy. Fix: layer hard constraints, evidence verification, budget control, and stop judgment separately instead of stuffing all rules into prompts.

Global Luxury Brand + Xinzhixin (Tan Zong, Yuan Panfeng) : Dual perspective — business pain (multi-cloud, multi-brand data silos, compliance walls, fragmented AI apps → "agent online but no data usable") and technical solution. Architecture: multi-cloud AI-native platform + enterprise semantic layer. Four core capabilities: skills ecosystem, context engineering, sandbox security, closed-loop iteration. Enterprise-specific: multi-version compatibility/fault tolerance, business knowledge base remediation governance. Metrics: end-to-end data delivery efficiency +70% , agent task diagnosis accuracy +50% , code generation accuracy 30% → 90% via semantic knowledge base. Tools: Apache Ossie ontology spec, CodeWiki automated semantic extraction.

Yunqi Technology (Guan Tao) : Opening thesis — model + Harness capability sets the floor; enterprise knowledge system sets the ceiling of business value . Why do companies with data platforms, RAG, and knowledge graphs still get wrong/unclear/unactionable answers? Missing a knowledge system that expresses business semantics, relationships, rules, context, and connects to real actions. Ontology is key but has no unified standard. Compares five mainstream lines: Palantir, Snowflake, Databricks, Apache Ossie, Agent Memory — difference lies in how knowledge is represented, executed, produced, and validated . Reference architecture for continuous knowledge engineering: start from high-value structured business domain, establish deterministic semantic core via Semantic View, supplement long-tail knowledge via context and memory, connect ecosystem via open specs, close loop with AI ETL, observability, and knowledge engineering. Guan Tao background: MSRA (first-gen distributed KV, Bing storage), Microsoft US (Cosmos/Scope 200k nodes, Azure Data Lake), Alibaba Cloud (MaxCompute, DataWorks 100k+ scale), founded Yunqi 2021; USENIX ATC best paper award 2021.

Synthesis: Consistent Answers Across 12 Talks

Runhe: decision matrix by gap type; Harness first, post-train later.

JD Tech: three constraints (role, knowledge, action) + side-effect gradient principle.

Lingyue: hardest boundary — natural language constraints ≠ safety boundaries; enforce via external identity, policies, budgets, approvals, termination, audit.

Kurui: made "stop" a first-class Harness citizen; fulfillment rate 40.1 → 75.2.

ZTE: Harness Engineering → 6× requirement dev efficiency.

Dewu: most restrained — don't chase full automation; chase full-chain orchestratability, measurability, rollback.

JD Health: when AI must be deterministically right, redesign the whole engineering stack from model to knowledge to engineering to delivery .

Conference: DACon 2026 Beijing, Oct 23-24, Beijing Hilton Yilin Hotel. 15 sub-forums, 60+ real cases, 2-day immersive learning, closed-door sessions and evening roundtables.