LLM-Harness Coupling: How Model Differences Shape Code Agent Architecture

Proliferation of Code Agents

The ecosystem now includes many code agents developed by major LLM providers, each with its own repository:

OpenCode – https://github.com/anomalyco/opencode Codex – https://github.com/openai/codex Grok Build – https://github.com/xai-org/grok-build Goose – https://github.com/aaif-goose/goose Cline – https://github.com/cline/cline Aider – https://github.com/Aider-AI/aider OpenHands – https://github.com/OpenHands/openhands MiMo-Code – https://github.com/XiaomiMiMo/MiMo-Code Qwen Code – https://github.com/QwenLM/qwen-code deepseek – https://github.com/deepseek-ai/deepseek-harness kimi code – https://github.com/MoonshotAI/kimi-code PI – https://github.com/earendil-works/pi Claude (closed-source) – https://claude.ai/new Zcode (closed-source) – https://zcode.z.ai/cn antigravity – https://github.com/google-antigravity/antigravity-cli This abundance creates choice paralysis and switching costs for developers, raising the question: how tightly coupled are Harness frameworks to the underlying LLMs? Do different models require different system contexts, tool descriptions, or prompt formats?

Commercial Motivations vs. Technical Abstraction

Vendors build proprietary agents for commercial reasons: creating a closed-loop "data flywheel" that collects high-quality interaction chains (prompt → code → error → fix) to train stronger future models, and increasing developer ecosystem lock-in through skill marketplaces. Technically, the article asks whether every model truly needs a dedicated agent, or whether a Harness can abstract away model-specific differences. Many open-source agents already have a Provider directory for integrating different model APIs, but the article probes what other special handling is required.

Four Key Dimensions of Model-Specific Differences

1. Training Data and Alignment Styles

Models differ in how they follow instructions. For example, OpenAI (GPT-4o/GPT-4) exhibits extremely high instruction adherence, prefers structured and logically rigorous prompts, and responds excellently to role-playing and step-by-step Chain-of-Thought prompting.

2. Attention Mechanisms and Context Handling

Models vary in their ability to capture key information in long contexts. Some suffer from the "Lost in the Middle" phenomenon, where instructions placed in the middle of a system prompt are easily ignored. Consequently, critical rules must be positioned at the very beginning or the very end of the context.

3. Sensitivity and Safety Guardrails

Certain models enforce strict safety constraints. If a system prompt touches on ambiguous political, violent, ironic, or sensitive topics, the model may refuse to respond unless guided with complex "desensitization" or "scoped" language.

4. Training Methodology and Format Alignment

During post-training (SFT/RL), each vendor trains models on specific protocol formats using special tokens such as <|im_start|>, <think>, <skill>, etc. When assembling context, the Harness must strictly adhere to the vendor's training standards and message role sequencing. If the Harness forces a custom flat text concatenation (e.g., simple USER: ... ASSISTANT: ...), it deviates from the model's training distribution, directly weakening reasoning coherence, tool-calling accuracy, and causing KV Cache invalidation.

Reference Implementation: OpenCode's Multi-Model Integration

OpenCode's design for integrating various models can be studied in its source code: packages/opencode/src/session/system.ts Prompt directory:

https://github.com/anomalyco/opencode/tree/dev/packages/opencode/src/session/prompt

Additional reference:

https://www.dong-blog.fun/文章

Open Questions on Model Preferences and Automation

If each LLM has unique preferences, are those preferences only known internally, making the vendor's own agent the best fit? Similar to compilers or kernels with numerous parameters, will future models expose their preferences so that prompts, tool descriptions, and skills can be automatically optimized per model? Or will model capabilities converge? A key architectural challenge is identifying variation points and isolating them through abstraction.

Deep Technical Analysis

1. Attention Mechanism's Position Preference and Attention Dilution

LLM attention acts as a "similarity-based soft retrieval." It exhibits a clear position preference (Lost in the Middle) : higher attention is allocated to information at the beginning and end of the context, while information in the middle of long contexts is easily overlooked.

2. Conflict Between "Training Sync" and "Deployment Async"

LLM API specifications typically assume a strict synchronous sequence: the Assistant emits a tool_call, and the next message must be a tool_result; otherwise the interface returns an error. However, the physical world is asynchronous — tool execution takes time, and users may send interruptions or modified instructions mid-execution. This fundamental mismatch between the synchronous training protocol and asynchronous runtime reality is a core design tension for any Harness.