Turning Implicit Code Rules into AI‑Readable Specs and Automated Checks with Harness Pilot

Problem: AI agents lack visibility into implicit project rules

AI agents working on a codebase cannot perceive layered constraints, naming conventions, or architectural guidelines that are only documented in wikis or exist as tacit knowledge, leading to repeated violations, endless fix cycles, and noisy logs.

Solution: Harness Pilot – rule documentation and automated pre‑validation

Harness Pilot externalises and versions project rules inside the Git repository and runs automated scripts before code is written, replacing the reliance on agents’ “self‑consciousness” with enforced pre‑checks. This guarantees that AI agents collaborate reliably within defined architectural and quality constraints.

Key Advantages

Rules are stored alongside code, not in separate wikis.

Versioned rules evolve together with the codebase.

Agents obtain the full context simply by accessing the repository.

Pre‑validation prevents violations before they occur, reducing repair cost.

Four Design Principles

Git repository as the single source of truth (SSOT) – all rules live in the repo.

.harness/ directory is self‑contained – generated files stay under .harness/ to avoid polluting the project root.

Boundary‑focused, implementation‑agnostic rules – only layer‑dependency and quality rules are enforced; design patterns remain free.

Pre‑validation over post‑hoc checks – structural operations are verified before code is written, dramatically lowering fix effort.

Architecture Overview

The plugin lives under plugins/harness-pilot/ and creates a .harness/ directory in the target project. The directory contains versioned manifests, capability snapshots, documentation, templates, scripts, hooks, and handoff artifacts.

plugins/harness-pilot/
├── .claude-plugin/          ← plugin metadata (plugin.json)
├── skills/
│   ├── harness-analyze/    ← analysis skill (SKILL.md + 4 tools)
│   └── harness-apply/      ← generation skill (SKILL.md + 4 tools + config/)
├── agents/                 ← 5 Agent definitions (.md files)
├── templates/             ← layered template system
│   ├── base/
│   ├── languages/
│   ├── frameworks/
│   ├── rules/
│   └── capabilities/
├── lib/                   ← shared utility modules
├── scripts/                ← template engine, JIT test generator
├── hooks/                  ← session‑start hook
└── tests/                  ← unit tests

When harness‑apply runs, it generates a structured .harness/ tree inside the project:

my-project/
└── .harness/
    ├── manifest.json            ← state tracking for re‑entrant apply
    ├── capabilities.json        ← enabled capabilities snapshot
    ├── docs/
    │   ├── ARCHITECTURE.md      ← project architecture and layer rules
    │   ├── DEVELOPMENT.md       ← development commands and test methods
    │   └── PRODUCT_SENSE.md     ← business context
    ├── scripts/
    │   ├── lint-deps.*          ← import‑direction checks
    │   ├── lint-quality.*       ← code‑quality checks
    │   ├── lint-imports.*       ← import limits & circular‑dependency detection
    │   ├── lint-semantic.*      ← semantic business‑logic validation
    │   └── validate.*           ← unified validation entry point
    ├── rules/
    │   ├── common/
    │   │   ├── safety.md        ← AI safety constraints
    │   │   └── git-workflow.md ← Git workflow rules
    │   └── {language}/
    │       └── development.md ← language‑specific development standards
    ├── memory/                 ← three kinds of memory (scenario, program, failure)
    ├── trace/                  ← failure traces for later analysis
    ├── hooks/                  ← Git post‑commit hooks
    ├── tasks/                  ← task status and checkpoints
    └── handoffs/               ← cross‑session handoff artifacts

Skills and Agents

Two core skills are provided: harness-analyze – analyses project health, documentation coverage, architecture compliance, quality‑rule adherence, test coverage, and validation status. harness-apply – generates the .harness/ infrastructure, creates rule files, and sets up linting and validation scripts.

Five built‑in agents (code‑reviewer, architect, implementer, reviewer, etc.) are automatically scheduled based on the project’s needs. The agents read AGENTS.md placed at the project root to discover key files and first‑principles.

# {PROJECT_NAME} - Agent Navigation Map
## First Principles
# 四条核心原则（从需求出发，而非模板）
## Project Overview
# 语言、框架信息
## Key Files
# ├─ AGENTS.md（本文件）
# ├─ .harness/docs/ARCHITECTURE.md（架构规则）
# ├─ .harness/docs/DEVELOPMENT.md（开发命令）
# ├─ .harness/docs/PRODUCT_SENSE.md（业务上下文）
# ├─ .harness/manifest.json（配置快照）
# └─ .harness/capabilities.json（已启用能力）

Workflow

The recommended pipeline is:

analyze → apply → develop → validate → ship

After apply, a “Ralph Wiggum Loop” automatically runs up to three cycles of lint‑test‑fix, ensuring continuous quality enforcement.

Ralph Wiggum Loop and Handoff Mechanism

The loop executes harness-apply repeatedly (max 3 rounds) until no new violations appear. When the LLM context window approaches its limit (≈ 100 k tokens) or the loop exhausts without solving the problem, a handoff persists the current state to files ( agent-state.json, context.json, resume.json) so the next session can resume without losing information.

Sample Metrics from a Real Project

A dry‑run of harness-analyze on the claude-code-go repository produced the following health report:

╔══════════════════════════════════════════════════╗
║          Harness Health Report                  ║
║ Project: claude-code-go                         ║
║ Score: 91/100 (A - Excellent)                  ║
╠══════════════════════════════════════════════════╣
║ Category          Score   Status                ║
║ ─────────────────────────────────────────────── ║
║ Documentation      100    Excellent             ║
║ Architecture       100    Excellent             ║
║ Quality Rules      100    Excellent             ║
║ Test Coverage       54     Fair                  ║
║ Validation          100    Excellent             ║
╠══════════════════════════════════════════════════╣
║ Details                                        ║
║ ─────────────────────────────────────────────── ║
║ [✓] .harness/docs/ARCHITECTURE.md               ║
║ [✓] .harness/docs/DEVELOPMENT.md                ║
║ [✓] .harness/docs/PRODUCT_SENSE.md               ║
║ [✓] .harness/scripts/lint-deps.go                ║
║ [✓] .harness/scripts/lint-quality.go            ║
║ [✓] .harness/scripts/validate.go                ║
║ Import Analysis: 83+ imports, 0 layer violations║
║ Test Coverage Highlights: sandbox 95.1% | …      ║
║ Packages with 0% coverage: cmd/claude, internal/…║
╠══════════════════════════════════════════════════╣
║ Recommendations                                 ║
║ 1. Update ARCHITECTURE.md to reflect recent changes║
║ 2. Add tests for 0% coverage packages            ║
║ 3. Commit pending harness changes                ║
╚══════════════════════════════════════════════════╝

Industry Context

Major AI labs are converging on similar “harness” concepts:

OpenAI’s Harness Engineering keeps Codex reliable on complex tasks.

Anthropic’s Managed Agents separate “hand” and “brain” for scalable deployment.

DeepMind’s AutoHarness shows that a small model plus a good harness can outperform larger models.

LangChain’s Agent Harness Anatomy demonstrates a 14% boost in Terminal Bench scores.

Recommendations and Usage Steps

Run /harness-pilot:harness-analyze to obtain a health report.

Run /harness-pilot:harness-apply to generate the .harness/ structure.

Adjust lint-deps scripts to encode your layer‑dependency rules.

Build a full validation pipeline: build → lint → test → validate.

Enable the Ralph Wiggum Loop and handoff mechanisms for continuous quality enforcement.

By moving from “teaching” AI agents to “automatically verifying” their actions, teams can rely on code, linters, and tests rather than on the LLM’s intuition, dramatically reducing bug density and repair cycles.