Why Claude Code’s Harness Only Lasts Six Months – Boris Cherny’s 10 Tips for AI Practitioners

Ablation study for Claude Code

In the July 28 YC interview, Boris Cherny recommends deleting the entire Claude Code system prompt, tool definitions, and harness code, then re‑adding them line by line while observing the impact on model behavior. This is an explicit ablation experiment: keep all other conditions constant, remove or replace a single module, and compare performance, stability, efficiency, or cost.

Anthropic’s Claude 5 release trimmed the Claude Code system prompt by more than 80 %, leaving only safety, permission, and static‑analysis components. Cherny advises against guessing which instructions the model needs; instead, iteratively delete, test, and identify the points where the model repeatedly stalls.

Product Overhang

"Product Overhang" describes the gap between rapidly, discontinuously advancing model capabilities and the slower, incremental pace of product integration. As a result, a model’s latent abilities often exceed the functional boundaries of existing products.

Example: at the end of 2024, Sonnet 3.5 could generate an entire source‑file in one request, while contemporaneous coding assistants such as Copilot or early Cursor versions were limited to line‑or‑completion‑level suggestions.

Unhobbling (解缚)

Unhobbling means removing artificial constraints so the model can attempt harder tasks and operate autonomously for longer periods.

Anthropic’s internal experiment connected Opus 5 to the OpenCV library. Without any weight changes, the model began producing portraits and landscape drawings—behaviors it had never been explicitly trained to perform. This illustrates “model elicitation”: designing prompts, context, tools, or product forms that surface latent model abilities.

Three practical ways to unhobble a model

Assign tasks harder than the model expects. Define clear goals, boundaries, and exit conditions, then let the model attempt the task without over‑specifying the solution.

Run abundant experiments. Give the model freedom to explore creative, non‑commercial activities, observing how it behaves when not constrained by a narrow success metric.

Make the model self‑validate its work. Shift focus from prompt engineering to building mechanisms that let the model check its own output during execution, enabling longer independent runs.

Cherny notes that many teams still treat AI as a pure prompt‑engineering problem, neglecting self‑validation, which prevents models from running independently for extended periods.

Worked example: Electron → Swift rewrite

Boris’s prompt to Claude:
"Rewrite the Electron application as a native Swift app. Run the Electron version in a macOS VM, capture a screenshot, then compare each pixel with the Swift version. Do not stop until the comparison is complete."

The model has been running this task for over two weeks, continuously posting progress screenshots to an internal Slack channel, demonstrating both task persistence and the need for self‑validation mechanisms.

Key takeaways

Iterative deletion and testing provide concrete evidence of which prompts or harness components are essential.

Model capabilities can be elicited without weight updates by redesigning prompts, tools, or product interfaces.

Assigning deliberately difficult tasks and requiring the model to verify its own results are effective strategies for extending autonomous operation.