DeepSeek-V4.1-Flash Harness Showdown: Claude Code Wins on Quality & Efficiency

Benchmark Setup

The author tests DeepSeek-V4.1-Flash integrated with four different harnesses: Pi Agent, DeepSeek Harness (DSH) Standard mode, DSH Minimal mode, and Claude Code. The task is to implement a Space Invaders game in HTML/JavaScript, a benchmark originally used by Simon Willison (the creator of the viral "pelican riding a bicycle" test). The prompt used:

Write an HTML and JavaScript page implementing space invaders

Each harness is evaluated on completion time, token consumption, cache hit rate, output tokens per second (TPS), and functional correctness scored by an independent reviewer (MiBiTi 6-Astra).

Results per Harness

Pi Agent

Time: 36 min 9 sec. Tokens: 9.7M. Cache hit: 98.9%. TPS: not provided. The game runs but the first-screen style deviates from the target Astra layout. Ranked 2nd overall (score 8.9). Strengths: strongest visual effects, colorful enemies, starfield, glow, explosions, drag controls. Weaknesses: narrow-screen title overlaps score area, bottom text overlaps, difficulty high, context explosion (172 messages, 85 tool calls).

DeepSeek Harness Standard Mode

Time: 8 min 54 sec. Tokens: 4.2M. Cache hit: 98%. TPS: 275 tok/s. First screen and sidebar match expectations; body slightly weak. Ranked 3rd (score 8.7). Strengths: most stable, refresh and wide/narrow screens work, complete gameplay, very restrained. Weaknesses: computer and mobile screens too small; pause screen shows both "Resume" and "Start" prompts causing confusion.

DeepSeek Harness Minimal Mode

Time: 40 min 4 sec. Tokens: 1.5M. Cache hit: 99%. TPS: 295 tok/s. Task failed. Ranked 4th (score 7.2). Strengths: initial wide-screen effect good, colors, particles, shooting feedback complete. Weakness: page refresh freezes, leaving blank battlefield. Extreme context compression forces multi-round trial-and-error, increasing total time and hurting quality.

Claude Code

Time: 9 min 34 sec. Tokens: 3.0M. Cache hit: 98%. TPS: ~205.5 tok/s. Performance exceeded expectations; space droplet shows experience. Body layout less ideal. Ranked 1st (score 9.2). Strengths: best understands original arcade details — portrait orientation, enemy acceleration, UFO, bunker damage, professional shell and data panel. Weaknesses: pause button still shows "Pause"; Enter key vs "Restart" inconsistency; mobile zoom risk.

Functional Correctness Evaluation

The reviewer (MiBiTi 6-Astra) tested five criteria:

First load: all four versions normal.

Start, move, shoot, score: all four playable.

Pause and resume: all functional, but Claude Code and Standard have prompt text issues.

Page refresh: Claude Code, Pi, DSH Standard normal; DSH Minimal fails.

Narrow screen: Standard most stable but small; Claude Code needs vertical scroll; Pi homepage overlaps; DSH Minimal blank after refresh.

Assistive usability: Claude Code and DSH Standard have best button labeling; Pi Agent's mobile keys not keyboard/screen-reader friendly.

Comparative Analysis

Claude Code leads in delivery quality, code accuracy, and engineering integrity. Its context management is extremely concise, using only ~1/3 of Pi Agent's tokens and ~1/4 of the time to achieve the best result — a production-grade, high-ROI reference.

Pi Agent follows closely in quality, demonstrating powerful tool-chain invocation and deep reasoning, but at the cost of massive context bloat (172 messages, 85 tool calls), highest token consumption (9.7M, 6.4× minimal mode), and 36-minute end-to-end latency. Suitable for exploratory tasks where quality is paramount and cost/time are irrelevant.

DSH Standard mode tops speed at 8 min 54 sec with 275 tok/s, executing very smoothly. However, logical closure and detail handling on complex tasks lag behind Claude Code and Pi Agent, placing it third. Ideal for high-throughput daily development with moderate complexity and strict time constraints.

DSH Minimal mode achieves the lowest token usage (1.5M, saving 84.5% vs typical), but extreme context starvation deprives the model of global view, forcing repetitive multi-step probing and reflection. This backfires: total time balloons to 40 minutes (slowest) and final quality is worst.

Key Takeaway

If a model underperforms, consider switching its harness. The harness's system prompt, tool definitions, context management, and interaction protocol materially affect outcomes. The author cautions that this single-task, non-ideal isolation test is only for reference.