DeepSeek Harness Review: One-Day Intensive Test – Hit or Miss?

Introduction

This week DeepSeek launched the V4 Pro model and the long‑awaited open‑source DeepSeek Harness. The repository gathered nearly 100 k stars within two days, indicating strong community interest.

Concept and Design

DeepSeek describes a model as the "brain" and Harness as the "body"; an Agent is essentially "LLM + Harness". Unlike previous workflows where users integrated DeepSeek into external agents, DeepSeek now provides its own agent framework.

Plugin‑First Philosophy

The core idea is summed up in four Chinese characters: Everything Is a Plugin . Model, tools, strategies, storage, sandbox, context management, and UI are all modular components that can be added, removed, or swapped like building blocks, giving developers high flexibility.

Community Plugins

During the internal testing phase, developers contributed many UI plugins, ranging from anime‑style skins to terminal‑style (TUI) interfaces and retro portal pages. The plugin marketplace already contains over 100 extensions and continues to grow.

Installation and First Use

To install, send the following command to your preferred Agent:

帮我安装 DeepSeek Harness https://www.deepseek.com/harness/

After a few minutes the installation finishes. Then start the web UI with: dsh web The browser opens a page where you must paste your DeepSeek API key (tokens are billed) and ensure your account has sufficient credit.

UI Overview

The left pane shows conversation history, the middle pane is the task area where you select a working directory and a run mode, and the right pane lets you choose the model (e.g., DeepSeek V4 Pro or V4 Flash) and inference level.

Permission Levels

Read Only : The agent can view files but cannot modify them.

Workspace Write : The default level; the agent can edit files inside the selected workspace and will ask for confirmation before touching files outside the scope.

Full Access : Removes all safeguards; the agent can read and write anywhere on the system. Use only when you fully understand the upcoming actions.

Run Modes

Standard Mode : Suitable for everyday development tasks – reading files, editing code, running commands, invoking Skills, and spawning sub‑agents.

PTC Mode : For long, multi‑step workflows. The agent compiles repeated tool calls into a TypeScript script, reducing round‑trip latency and keeping the context compact.

Minimal Mode : Leaves only a Bash executor and a file editor; useful for “bare‑metal” testing of the model’s core capabilities.

Creation Mode : Allows developers to experiment with the runtime environment, combine plugins, and build a custom agent configuration.

Plugin Management

DeepSeek ships with more than 100 built‑in plugins. The settings page lists them, and the count is expected to increase as the community contributes.

Benchmark 01 – Eight‑Leg Spider Animation

A lengthy prompt asked the agent to generate a walking‑spider animation page.

Results:

dsh + DeepSeek V4 Pro (Max) completed the task in 15 minutes, displayed token usage and cache hit rate, but the initial page showed no spider (a failure).

Codex + GPT‑5.6 Sol finished in under 10 minutes with a correct, smooth spider animation.

Investigation revealed that V4 Pro cannot directly read images; it attempted a PNG decoder and then fell back to a text‑based analysis, which caused the missing spider.

After debugging, the corrected run produced a spider in 35 minutes. Lowering the inference level to High reduced the time to 17 minutes (minor leg merging). The V4 Flash (High) version also succeeded, though the legs appeared shorter.

Benchmark 02 – 3D Voxel Soccer Animation

The task asked for a voxel‑style soccer animation with continuous ball control.

dsh + V4 Pro delivered a smooth, uninterrupted sequence.

Codex + GPT‑5.6 Sol was faster and produced a clearer image, but a player fell without any collision, breaking realism.

Overall, DeepSeek wins on motion continuity, while GPT‑5.6 Sol wins on visual fidelity.

Benchmark 03 – 3D Rubik’s Cube

Prompt: create an HTML file that loads Three.js via CDN, builds a fully functional 3D Rubik’s cube, and supports automatic solving.

创建一个独立的 HTML 文件，
其中包含一个使用 Three.js（通过 CDN 引入）构建的、功能完整的 3D 魔方模拟器。
该魔方必须能够自动复原。

Codex + GPT‑5.6 Sol (Extreme) finished in 11 minutes, generated a page that could be played immediately, and allowed mouse‑drag rotation.

dsh + V4 Pro (Max) took about one hour, running roughly 500 internal test loops before producing a working cube with auto‑solve animation and mouse interaction.

Benchmark 04 – Jump‑Game Mini‑Game

A short prompt asked the agent to implement a “Jump” style game.

dsh + V4 Pro needed 30 minutes to produce a playable page; the first run scored only 2 points, but the core mechanics worked after a second attempt.

Codex + GPT‑5.6 Sol finished in 8 minutes, delivering a horizontal layout that also ran without obvious bugs.

Both agents succeeded, with GPT‑5.6 Sol winning on speed and DeepSeek winning on fidelity to the original “jump” style.

Benchmark 05 – Sliding Resistor ASCII Art

The author wanted an ASCII representation of a sliding resistor diagram because DeepSeek lacks image‑reading capability.

DeepSeek V4 Pro first tried to download the image and decode PNG pixels, which failed. It then converted the picture to text via a tool and produced a rough ASCII version.

GPT‑5.6 Sol (vision model) directly read the image and reproduced a much more faithful ASCII rendering.

Cost Analysis

The entire suite of tests consumed roughly 250 million tokens. At the pre‑price‑increase rate this equated to about 20 CNY (≈ $3), comparable to a cup of milk tea. After the recent price revision, the same usage would cost several hundred CNY. By contrast, the equivalent GPT‑5.6 Sol usage would be about 186 USD (≈ 1260 CNY), roughly ten times higher.

Final Thoughts

For developers, DeepSeek Harness’s "everything is a plugin" architecture is compelling: you can re‑assemble the model, tools, prompts, and UI to fit precise workflows. The detailed task‑trajectory view is invaluable for debugging and understanding tool‑call order.

However, the UI is heavily developer‑oriented. Casual users may find the numerous tabs, permission settings, and lack of a one‑click Skill marketplace confusing. A notable usability issue is that the web page’s timer continues even after the backend process has stopped, making it hard to tell whether a task is truly alive.

Despite these quirks, the framework is inexpensive, highly extensible, and already demonstrates impressive capabilities across diverse generation tasks. As the community ecosystem expands, DeepSeek Harness could become a powerful, customizable backbone for AI‑native applications.