Designing Harness Engineering for Enterprise Vertical Agents: From First Principles to Architecture

Prompt Engineering

Prompt engineering addresses the expression issue by shaping how a task is described. Core techniques include:

Role setting : define the model’s identity.

Few‑shot examples : provide concrete instances (as in OpenAI’s 2020 “Language Models are Few‑Shot Learners”).

Chain‑of‑Thought : decompose, reason, then answer (Google 2022).

Format and boundary constraints : prescribe output shape and refusal rules.

Prompt engineering works because LLMs are highly sensitive to context, but it reaches a ceiling when the task requires factual knowledge rather than clear expression.

Context Engineering

When prompts are insufficient, the system must supply the right information. Context engineering combines:

RAG (Retrieval‑Augmented Generation) : fetch external knowledge and inject it into the prompt.

Progressive disclosure : expose only the most relevant metadata, instructions, or resources at each step (Anthropic Agent Skills).

Window management : LLMs have hard token limits (e.g., GPT‑3 = 2048 tokens, GPT‑3.5 = 4096 tokens, modern models up to 128K‑1M tokens). Exceeding the window truncates older content, causing “context rot”.

Effective context engineering balances the amount of information with the model’s attention budget.

Harness Engineering

LangChain defines an agent as Agent = Model + Harness. Anthropic describes Harness as the loop, tools, context management, and guardrails surrounding the model. The hierarchy is Prompt ⊂ Context ⊂ Harness. Harness ensures that an LLM not only answers but also executes safely and reliably.

Five‑Dimensional Diagnostic Tool

To decide where to focus Harness effort, a five‑dimensional coordinate system is proposed:

Verifiability : Can the result be objectively judged at low cost?

Reversibility : Can a wrong action be rolled back?

Environment determinism : Is the execution environment stable and reproducible?

Autonomy : Does the task span multiple steps or long‑running sessions?

Compliance : Are there regulatory, audit, or accountability requirements?

Each dimension maps to a specific engineering focus (e.g., low verifiability → manual review; low reversibility → pre‑execution constraints).

Six‑Layer Capability Model

Starting from the first‑principle view that an LLM is a stateless text function, six incremental layers plug the model’s gaps:

L0 – Bare model

Only understands prompts and generates text.

L1 – Tool loop

Enables the model to act. Example: kubectl get pods → CrashLoopBackOff → kubectl logs. Implementations: ReAct (2022) and OpenAI Function Calling (2023).

L2 – Context management

Feeds external information and prevents context rot. Techniques: RAG, progressive disclosure, window compression, external storage of intermediate artifacts.

L3 – Task state & memory

Persists progress across turns or sessions. Implementations: Claude Code uses a git‑commit checkpoint plus a JSON progress file; DeepSeek dsh stores an append‑only event log for crash recovery.

L4 – Independent verification

Separates execution from judgment. Verification can be:

Deterministic (tests, health checks, reconciliation).

Reasoning‑based (a second LLM acting as a skeptical judge).

Human review for high‑risk conclusions.

L5 – Safety guardrails & failure recovery

Pre‑execution controls (permission tiers, two‑phase commit, immutable environment binding) and post‑failure mechanisms (limited self‑healing, crash replay, cleanup). All external inputs are treated as zero‑trust.

L6 – Multi‑agent orchestration

Splits a large task into sub‑agents (context isolation) or collaborates specialized agents (planner, coder, reviewer). Orchestration is added only when a single context cannot handle the workload.

Mapping Dimensions to Layers

Verifiability → primary focus on L4; stricter requirements shift effort to manual review.

Reversibility → primary focus on L5; stricter requirements add pre‑execution constraints and two‑phase commit.

Environment determinism → primary focus on L1 and L5; lower determinism strengthens exception handling and environment‑bound validation.

Autonomy → primary focus on L3 and L6; higher autonomy requires state persistence and orchestration.

Compliance → primary focus on L5; higher compliance adds audit, permission segregation, and data isolation.

Case Study 1 – Production Ops On‑Call Agent

Background

An enterprise runs multiple clusters across data centers. Operators handle daily queries and alarm triage via IM and web. The goal is an on‑call agent that can answer questions, inspect status, locate faults, and perform controlled changes.

Five‑Dimensional Diagnosis

Verifiability: medium – some conclusions can be auto‑checked, many need human judgment.

Reversibility: extremely low – actions like restart or config change are irreversible.

Environment determinism: low – topology changes constantly.

Autonomy: limited – long‑running inspection is allowed, but critical changes require a human in the loop.

Compliance: extremely high – every action must be auditable and accountable.

Result: Harness focus shifts to pre‑execution guardrails (L5) and moderate verification (L4), with heavy context (L2) and state persistence (L3).

Capability Allocation

L1 Tool loop – medium: read‑only inspection tools are primary; write tools are locked.

L2 Context management – heavy: RAG pulls SOPs, real‑time metrics, and logs.

L3 State & memory – moderate‑heavy: persists multi‑hour troubleshooting paths, with automatic forgetting when infrastructure changes.

L4 Independent verification – medium: objective health checks plus a skeptical LLM reviewer and optional human sign‑off.

L5 Safety guardrails – heaviest: permission tiers, two‑phase commit, immutable environment binding, full‑chain audit.

L6 Multi‑agent orchestration – cautious: sub‑agents for network, storage, DB each operate read‑only and return summarized judgments.

Architecture

Entry adapter : normalizes IM and web messages.

Retrieval layer : (a) knowledge‑base RAG for SOPs, (b) read‑only APIs for cluster, monitoring, logs.

Context orchestrator : merges retrieval results, task goal, permissions, and bound environment; handles window compression.

Main loop & tool set : read‑only tools always enabled; write tools gated behind the authorization gate .

Authorization gate : two‑phase commit, human approval, environment binding check.

Validator : objective health‑check + independent LLM critic; high‑risk outcomes also go to human review.

Memory & sink : persists progress and long‑term experience with expiration policies.

Audit log : immutable record of every tool call, decision, and approval.

Case Study 2 – Intelligent Q&A Bot

Background

A user‑facing chatbot answers product or sales questions via web or app. It never performs actions on external systems.

Five‑Dimensional Diagnosis

Verifiability: low‑medium – answer quality is hard to judge instantly.

Reversibility: medium – wrong statements cannot be undone.

Environment determinism: medium‑high – mainly reads from a stable knowledge base.

Autonomy: low – typically single‑turn or few‑turn dialogs.

Compliance: medium‑high – must stay on approved messaging and avoid false promises.

Result: The critical layer is L2 (knowledge grounding) and L5 (output guardrails) with opposite direction – guardrails prevent “talking nonsense”.

Capability Allocation

L1 Tool loop – light: only read‑only lookups (knowledge, order status).

L2 Context management – heaviest: RAG must retrieve exact documents; answer quality depends on retrieval precision.

L3 State & memory – light: short dialogs need no long‑term memory.

L4 Verification – medium: compliance checks before release, post‑deployment satisfaction metrics.

L5 Output guardrails – heavy (but opposite focus): filter for prohibited content, enforce approved phrasing, hand off to human when out‑of‑scope.

L6 Orchestration – light: a single context suffices.

Architecture

Entry adapter normalizes user messages.

Retrieval layer performs RAG against the knowledge base.

Context orchestrator assembles retrieved passages with the user query.

Main loop uses read‑only lookup tools.

Output guardrail validates every response against compliance rules and routes uncertain queries to a human operator.

Three‑Ring Theory

The methodology is abstracted into three concentric rings:

Inner ring – Tool loop : enables the LLM to act.

Middle ring – Control loop : pre‑control, execution, post‑control ensure correctness and safety.

Outer ring – Value loop : sinks knowledge back into the organization (e.g., incident post‑mortems) so the agent adds lasting value.

Only when all three rings are properly engineered does an agent become reliable and business‑critical.

Conclusion

The gap between successful coding agents and struggling vertical agents lies not in model size but in the surrounding Harness. By diagnosing a task with the five‑dimensional coordinate, allocating effort across the six‑layer capability model, and respecting the three‑ring architecture, teams can build agents that are both safe and valuable in any domain.