AWS Strands Harness: Same Model, 77% Lower Cost, Higher Accuracy

Benchmark: Same Model, Vastly Different Cost and Accuracy

AWS ran a harness benchmark across six suites (ALFWorld, ContextBench, GAIA, WebShop, τ³-bench, Terminal-Bench 2.1) using EC2 distributed infrastructure. All systems used the same underlying model, Claude Fable 5. Over 89 trials, Strands Harness cost $56.29 and achieved 69.7 accuracy; Claude Code cost $248.05 for 61.8 accuracy; oh-my-pi cost $86.83 for 69.7; OpenCode $73.42 for 66.3; DeepSeek Harness $40.30 for 59.5. Strands reduces cost ~77% versus Claude Code while improving accuracy by 7.9 points. DeepSeek Harness is cheaper but sacrifices accuracy. AWS notes the data is self-reported and the full paper is not yet public.

Key insight: the harness (runtime) contributes as much to cost/performance as the model itself.

Context Management: 1500-Token Offloading and 85% Compaction

Strands' ContextManager applies two default rules:

Tool-result truncation at ~1500 tokens : When a tool returns more than ~1500 tokens, the result is truncated to a head/tail preview; the full output is written to a Stash and can be retrieved via retrieve_context. This moves bulky tool outputs out of the hot context window.

Compaction at 85% context utilization : When the context window reaches 85% capacity, a summarization pass runs, preserving the most recent messages (e.g., last 4). If overflow persists, Context Recovery continues the agent loop instead of aborting with a context-length error.

Example: a tool returning 4000 tokens per call over 20 calls would accumulate 80,000 tokens of tool output. Offloading avoids repeated re-reading of that data in every subsequent model call, which is the primary driver of the cost gap.

Layered Memory: Context Window, Session, Long-Term Memory

Strands separates state into three tiers:

Context Window – working memory; aggressively managed via offloading and compaction.

Session – full conversation state persisted to disk ( ./.agent/sessions); restorable by session ID across process restarts.

Long-Term Memory – managed by MemoryManager with three capabilities: Recall (on-demand search), Injection (prepend relevant memories to prompt), Extraction (derive and store durable knowledge). Recall and Injection are enabled by default; Extraction requires explicit opt-in.

This mirrors traditional storage hierarchy: context ≈ RAM, session ≈ task state, memory store ≈ persistent knowledge.

Skills: Progressive Disclosure

Instead of loading all skill instructions into the system prompt, Strands uses Progressive Disclosure : only skill names and descriptions enter the initial prompt; full instructions are loaded via tool call when the model decides they are needed. This mirrors the tool-result offloading philosophy—don't pre-load what might not be used.

Prompt Caching and Cost Formula

Strands enables caching: auto, leveraging provider-level prompt caching (Bedrock, Anthropic) for system prompts, tool definitions, and stable message prefixes. The effective cost equation becomes:

Effective Context × Call Count – Cache Hits – Offloaded History + Compaction Cost + Subagent/Tool Call Cost

Thus identical model pricing can yield wildly different bills depending on harness implementation.

Model–Harness Decoupling and Portability

Strands SDK standardizes a Model Interface covering streaming, tool calling, and structured output. First-class providers: Amazon Bedrock, Anthropic, OpenAI, Google; community providers: Ollama, LiteLLM, Mistral, Llama API, llama.cpp, Vercel. The same harness can run on any Linux-container platform: Cloudflare Containers, Azure Container Apps, Google Cloud Run, Amazon ECS, Bedrock AgentCore, Modal. Two decouplings are achieved: model from harness, and harness from cloud platform.

Caveat: provider capabilities still differ (prompt caching, tool schema, reasoning effort, context window, parallel tool calls). Swapping models does not guarantee identical performance.

Evaluation Must Shift to Model × Harness × Workload

AWS's own benchmark demonstrates that Model × Harness is a combined variable, not independent. Harness controls what the model sees, retains, and when it calls tools; model capability determines how well it uses that information. Future evaluation should test specific Model × Harness combinations against real workloads (cost, success rate, step count) rather than relying on model leaderboards alone.

Conclusion

Strands Harness represents a shift: agent scaling is moving from model improvements to the runtime system surrounding the model. The "brain" (model) remains swappable, but the "body" (harness) is now a reusable, engineered component. While AWS's self-reported benchmarks need independent verification, the direction is clear: context, memory, tools, skills, caching, and execution loops determine how much of the model's potential is realized.

References:

Strands Agents, "Introducing Strands harness: frontier performance with 28% lower token cost", Sep 21, 2026.

Strands Agents Documentation: Harness, Configuration, Context Management, Memory, Skills.

The New Stack, "AWS open-sources an AI agent it says is 45% cheaper than Claude Code and Codex", Sep 21, 2026.