How to Build a Fully Local Coding Agent: Best Practices and Benchmarks

Why Build a Local Coding Agent

Running both the LLM and the coding‑agent harness on‑premises gives predictable costs, privacy, offline availability, reproducibility (no surprise model upgrades) and control over data handling.

Choosing a Harness: Why Qwen‑Code?

Qwen‑Code is open‑source, optimized for Qwen models, and lets you run Qwen and Codex side‑by‑side. NVIDIA’s Polar: Agentic RL on Any Harness at Scale paper shows Qwen models perform best with the Qwen‑Code harness.

Hardware Requirements

Qwen3.6 35B‑A3B weighs ~22 GB and needs 30–40 GB RAM; it runs smoothly on an M4 Mac Mini and on DGX Spark. Cohere’s June benchmark ranks it as the strongest local model of its size.

Deploying the Model with Ollama

Ollama is chosen for its cross‑platform CLI friendliness. Install and pull the model:

uv run speed-memory-benchmark/ollama_speed_memory_bench.py --model qwen3.6:35b-mlx  # macOS (MLX version)
uv run speed-memory-benchmark/ollama_speed_memory_bench.py --model qwen3.6:35b      # Linux

The speed‑memory benchmark reports pre‑fill and generation speeds (tokens/sec) and memory usage for prompts from 1k to 50k tokens.

Raschka’s practical threshold is 20–30 tok/s, comparable to GPT‑5.5 high‑speed inference; both Qwen3.6 and the North Mini Code meet this line.

Capability Evaluation

A private 5‑question task set is used to compare models. Results:

Qwen‑Code: 3/5 (fails on some agentic decision questions)

North Mini Code: 2/5 (tool‑selection errors)

Gemma4:e2b: 0/5 (incorrect tool choice and clarification failures)

Additional benchmarks with Codex and Claude Code show similar success rates, but Claude Code consumes over twice the tokens of Codex for the same tasks.

Security Audit Before Installing a Harness

The harness can read, modify files and execute shell commands, so Raschka recommends auditing the open‑source code and running it in an isolated environment (e.g., a separate VM or dedicated hardware). A checklist covering ten risk dimensions—install scripts, package hooks, shell execution, file I/O, secrets, prompt‑injection, plugins, network calls, telemetry, auto‑updates—is provided.

The audit of Qwen‑Code found no red flags beyond typical coding‑agent risks.

Configuring Qwen‑Code with Ollama

Installation can be done via a one‑line script or npm:

curl -fsSL https://qwen-code-assets.oss-cn-hangzhou.aliyuncs.com/installation/install-qwen-standalone.sh | bash
npm install -g @qwen-code/qwen-code@latest

For full control, clone the repo and build from source. After installation, edit settings.json to add a custom provider, enable “thinking” mode, and select the local model.

Token‑Usage Insights

Token consumption is driven mainly by the harness, not the model. In the same harness, Qwen3.6, North Mini and Nemotron burn nearly identical token counts for the five benchmark tasks. Claude Code’s higher token usage stems from feeding back extensive input context each round, effectively doubling token cost and halving task speed.

Saving 50 % of tokens roughly halves execution time, making a token‑efficient harness a decisive advantage in local scenarios.