From One-Off AI Agent Success to Scalable, Governable Delivery with Better Harness

In Better Harness, an analysis system has been built to understand agent execution processes and their engineering environments. The open-source version already supports reading and analyzing local data from over ten coding agents. However, as developers run multiple agents in parallel, maintain multiple projects, and switch devices, the core challenge is not merely aggregating scattered data but linking execution data distributed across different agents, projects, and machines into a complete task chain from requirement to verification, and turning that capability into something that helps individuals, enables team reuse, and supports organizational-scale delivery.

Why: Agent Challenges Are Shifting from "Single-Point Success" to "Scale Operations"

When adopting a new agent, the immediate question is whether it can complete the task at hand. Yet one success does not equal a stable capability, so building Agent Harness requires continuously enriched test sets for evaluation. The same task given to another developer, another agent, another project, or another machine may yield completely different results. Any difference in context, tools, permissions, project conventions, Skills, MCPs, models, or acceptance criteria can change the outcome.

Therefore, for agent capabilities to truly enter teams and organizations, they must pass through three phases:

Single-Point Success : Focuses on what context, knowledge, and tool capabilities are needed for a single task to produce reliable artifacts.

Team Replication : Focuses on which practices from a success can be codified into Skills, templates, MCP services, and quality rules.

Scale Operations : Focuses on how these capabilities can be securely provided to more teams via Skill marketplaces, MCP gateways, evaluation systems, and model gateways.

This path corresponds to three escalating questions: can a single task capability be done well; can similar tasks be stably reused; can mature capabilities run safely and economically at scale. Answering them requires more than counting sessions, tokens, or tool calls — it demands a collection and feedback mechanism centered on task results, connecting Skill and MCP execution processes, acceptance outcomes, and resource consumption, then using that to optimize Harness assets, improve team practices, close individual skill gaps, and ultimately lower the overall cost of a qualified task.

Focus-Driven: Different Levels Need Different Harness

As builders move from individual to team to enterprise, the problems Harness solves change: individuals need to make one agent collaboration work; teams need to stabilize reuse of effective methods; enterprises need to run mature capabilities at scale with safety and cost control. Harness capabilities should therefore be designed in layers.

Individual: Making One Agent Collaboration Work

Today everyone can be a builder, directing multiple agents daily. For individuals, the most direct question is: how to get the agent to do the job right the first time? This requires:

Clarify the task before the agent acts : Define the problem, scope of changes, non-goals, deliverables, acceptance method, and risk points — the content of a spec document. If the agent cannot accurately understand the completion criteria, it does not enter the execution phase.

Prepare context and tools per task : Provide only the project documentation, architecture knowledge, domain knowledge, and skills truly needed for the current task; open only necessary directories, commands, tools, and network permissions.

Collaborate around real artifacts : Annotate, modify, and accept directly on code diffs, web pages, or documents, while requiring the agent to provide verification results (screenshots, tests, etc.).

The individual perspective focuses on whether the complete process from task definition through execution to final acceptance can be faithfully reproduced.

Team: Turning Personal Experience into Stable Engineering Practices

For teams, the focus shifts to: how to turn personal experience into stable engineering practices?

Make the environment operable by agents : Establish unified installation, startup, test, and debug methods so agents can quickly enter verification, reproduce issues, and recover from failures — for example, defining build tasks via standard tools like npm or Gradle.

Make rules automatically checkable : Configure corresponding check mechanisms and acceptance methods for architecture, security, testing, and visual requirements, so rules actually enter the execution flow — for instance, using Playwright CLI or Agent Browser to access environments for visual checks.

Continuously precipitate task experience : Link requirements, sessions, file changes, tests, commits, and review evidence;沉淀 repeatedly validated practices into project conventions, Skills, scripts, hooks, task templates, or quality gates.

The team perspective cares not about a single lucky success but whether unified environments, automated checks, and Harness assets can keep a class of tasks consistently stable.

Organization: Running Agent Capabilities at Scale with Acceptable Cost

When Harness spans multiple teams, projects, and devices, the organization must solve not just whether an agent completes a task, but how to continuously govern and optimize delivery capabilities based on real evidence across tasks and teams:

Build cross-agent task evidence chains : Associate requirements, task definitions, agent sessions, context and tool usage, code changes, verification results, human acceptance, and subsequent defects, so a delivery can be reconstructed, traced, and reviewed.

Govern reusable Harness capabilities : Combined with trajectory analysis, version proven-effective Skills, MCPs, hooks, rules, and evaluation sets, and clearly define their applicable tasks, quality requirements, and maintenance ownership, preventing "works once" experience from spreading without boundaries.

Decide investment and promotion based on results : Continuously compare different Harness solutions by task type across acceptance pass rate, human intervention, failure retries, rework, resource consumption, and subsequent defects, to judge which capabilities should be promoted, optimized, narrowed, or retired.

The organizational perspective focuses not on "whether the agent looks smart" nor on single-call cost alone, but on whether scattered execution activities can be transformed into verifiable delivery evidence, and whether the coverage of effective capabilities can be continuously expanded with controlled quality, risk, and cost.

Better Harness's Initial Attempt: From Execution Data to Task Feedback

The individual collaboration, team reuse, and enterprise operations discussed above all rely on one premise: can we know how a task was completed, what results it produced, and which capabilities actually contributed? A complete task does not only happen in the agent's chat window; it passes through four interconnected Harness layers:

Downward, organizational and team goals are translated into task boundaries, acceptance criteria, context, tools, and rules, which the agent uses to plan, execute, verify, and recover. Upward, real artifacts undergo automatic verification or human acceptance to form task evidence, and repeatedly validated methods are precipitated into Skills, MCPs, hooks, rules, or task templates, providing a basis for team reuse and organizational decisions.

This is what the Better Harness Dashboard aims to do: turn scattered agent execution records into observable task feedback, reducing friction between humans and agents, so organizations waste fewer tokens/credits and can allocate them to higher-value tasks.

Currently, the Dashboard can view Skills, MCPs, models, tokens, and tool calls across projects, helping individuals and teams identify whether capabilities are being used, at which stages, and whether there are abnormal consumption or execution friction.

But "being called" does not equal "being effective." The next step is to connect these activities with task intent, acceptance criteria, real artifacts, and human acceptance, to gradually answer:

Did the task truly pass acceptance? Do the corresponding edit-file tool calls result in commits that appear in the corresponding release version?

Were Skills and MCPs used at the correct stages? How many calls failed or errored during Skill and MCP execution? These should be reasonably counted and fed back to publishers for improvement, reducing extra token consumption.

Which consumption did not translate into effective progress? Are there long-running tasks with huge consumption but no value?

Which experiences are worth precipitating as team Harness assets? For example, at the current stage a 400k context window achieves the best cost-effectiveness balance.

Therefore, the Dashboard is not the end goal but an observation entry point for task feedback. The most important thing now is to connect execution activities with real results, establishing a foundation for continuously improving Harness and lowering the overall cost of qualified tasks. Later, these experiences and lessons can be combined to share more best practices with more teams.

Welcome to Experience Better Harness

Better Harness is still evolving rapidly. You can run the Harness UI locally to see how it reads agents, Skills, MCPs, and task data from your project:

git clone https://github.com/QoderAI/better-harness.git
cd better-harness
npm install
npm run harness-ui:dev

Visit http://127.0.0.1:3410 to view the Dashboard. Then, explore some interesting topics together:

Connect sessions, task histories, code changes, and delivery results to build cross-agent task evidence chains.

Identify reusable work patterns from multiple tasks and precipitate them as traceable Harness Issues.

Verify whether changes to Skills, MCPs, and rules are effective through versioned components, evaluation sets, and controlled experiments.

Refine cost and routing evaluation, long-term task benchmarks, and real validation across different agents and operating systems.