WorkflowArena Benchmark: Top LLMs Achieve Only 36.95% in Multi-Agent Orchestration

Introduction

Agents are moving from solo operation to multi-agent collaboration. Frameworks like Claude Code and Codex now allow models to spawn sub-agents at runtime and write executable workflows that schedule them: how to decompose tasks, what context each sub-agent receives, where to verify intermediate results, and how to integrate final outputs. This makes multi-agent orchestration a key capability—well-organized agents can achieve more with less, while poor organization leads to mutual interference. Existing benchmarks either only check final task completion or test within designer-specified collaboration structures, leaving the question unanswered: can large models themselves organize multi-agent systems for real long-horizon tasks?

WorkflowArena Benchmark Design

Einsia AI's Navers Lab released WorkflowArena , a benchmark dedicated to evaluating LLM-based multi-agent orchestration. It contains 145 long-horizon tasks across 8 professional domains : software engineering, enterprise operations, scientific research, mathematical derivation, finance, legal, content design & 3D production, covering 17 task types. The benchmark evaluates 9 frontier models across 25 reasoning configurations .

Paper : WorkflowArena: Benchmarking Executable Workflow Orchestration in LLM-Based Multi-Agent Systems

Project homepage : https://lab.einsia.ai/workflowarena

GitHub : https://github.com/Einsia/WorkflowArena

ArXiv : https://arxiv.org/pdf/2611.12856

Each task is a real professional deliverable: modifying an open-source codebase, reproducing a scientific result, completing a Lean formal proof, compiling a multi-entity consolidated financial report, analyzing a multi-defendant legal case, or building a complete architectural complex in Blender. The defining characteristic is that all parts are tightly coupled : code changes must pass existing regression tests, financial data must reconcile across entities, legal conclusions must stay consistent across documents, 3D scene components must fit seamlessly. Doing each piece correctly is insufficient; integration is the test.

Evaluation Setup

Models receive a blank "command console":

Orchestration is self-determined . The model can work alone or invoke the Workflow tool to schedule multiple sub-agents, inspecting, terminating, or modifying workflows at any time. Whether to delegate, how to delegate, and what context each sub-agent receives are entirely up to the model.

Orchestration consumes budget . The main agent and all sub-agents share a single inference budget; planning, scheduling, and rework all count.

Hidden grading, full process logging . Models cannot see the grader or reference answers; scoring depends solely on the delivered result, independent of how many sub-agents were used. Every scheduling decision is recorded.

This design reveals not only how well a model performs, but how it organizes the work .

Key Findings

Overall Performance: Ceiling at 36.95%, No Universal Winner

WorkflowArena uses a strict full solve rate : only deliveries meeting all acceptance criteria count as solved. Partial completions that fail integration are failures.

Top performer: Claude Opus 5.5 (max) with an average full solve rate of 36.95% . Even the strongest system fails on over 60% of tasks.

Large domain gaps: Formal mathematics near saturation (Claude Opus 5.5 solves 95.83% ), but software engineering peaks at 29.41% , finance at 18.75% , legal at only 9.09% .

No system dominates all domains : Claude Opus 5.5 leads in software engineering, science, and math; Claude Sonnet 5.5 wins legal and content design; GPT-6 Astra and Doubao-Seed-Evolving tie for first in enterprise operations.

Two Distinct Orchestration Styles: Claude vs. GPT

Because every scheduling action is logged, WorkflowArena exposes how different models organize multi-agent work.

Claude family: "Plan-then-execute" — few invocations, large scale. Claude Opus 5.5 and Sonnet 5.5 average 0.86 and 0.76 Workflow calls per task , but each call dispatches 5.55 and 6.14 sub-agents on average; workflows with 9+ sub-agents account for 30% of Sonnet's runs.

GPT family: "Small steps, fast iterations" — many invocations, small scale. GPT-6 Astra and GPT-6.1 Sol average 2.09 and 1.94 Workflow calls per task , but each call dispatches only 1.72 and 1.69 sub-agents ; ~99% of workflows involve ≤4 sub-agents.

Style varies by domain. In legal tasks, Claude Opus 5.5 averages 14.29 sub-agents per workflow vs. GPT-6 Astra's 1.73 . In math, Kimi K3.1 Pro makes 9.67 Workflow calls per task but only 1.74 sub-agents per call . High invocation frequency ≠ large collaboration scale.

Orchestration Transfer: Strong Orchestrator Boosts Weak Executors

To isolate orchestration value, researchers fixed the executor model and swapped only the workflow author. Claude Opus 5.5 and GPT-6 Astra served as orchestrators; Kimi K3.1 Pro, GPT-6 Luna, DeepSeek V4.1 Flash, Doubao-Seed-Evolving as executors. Orchestrators never touched files; all execution was done by the executor. Two modes: static (orchestrator writes workflow then exits) and dynamic (orchestrator stays online, adjusts based on feedback).

Opus-authored workflows improve every executor. Kimi K3.1 Pro rises from 35.75 to 46.61 (+10.86); DeepSeek V4.1 Flash from 18.37 to 25.24 (+6.87). Average gain across four executors: 6.41 points (static) and 6.27 points (dynamic) . Since Opus never edited files, gains come purely from workflow design and sub-agent instructions.

Self-execution skill ≠ orchestration skill. GPT-6 Astra scores 77.29 on the full 145 tasks (far above the four executors, max 62.73 for Kimi), yet its workflows hurt executors: static mode averages -2.53 points, dynamic mode only +1.34 , even lowering Kimi and DeepSeek scores.

This demonstrates that multi-agent orchestration is a distinct capability, not a byproduct of task-solving ability, and warrants separate measurement and optimization.

Cost-Effectiveness: Orchestration Quality Trumps Token Spend

Plotting full solve rate against average per-task inference cost (log scale) yields a Pareto frontier.

Orchestration costs vary by orders of magnitude. Cheapest configurations cost < $1 per task; most expensive cost tens of dollars.

Higher spend ≠ better orchestration. GPT-6 Astra spends several times more per task than Claude Opus 5.5 but never catches its solve rate. Raising inference intensity on the same model yields sharply diminishing returns.

The ceiling is set by how well the workflow is organized, not by how many tokens are burned.

Case Study: 3D Construction — Same Task, Different Orchestration Logic

Three Blender tasks (Palace, Cathedral, Fortress) require building complete architectural scenes from blueprints and change orders.

Palace : Opus 5.5 dispatches 7 sub-agents to build halls in parallel, then assembles, checks, and iteratively repairs ( 78.86 ); GPT-6 Astra builds a shared foundation first, then assigns a single sub-agent to assemble ( 73.86 ).

Cathedral : Opus 5.5 does two full builds ( 77.95 ); GPT-6 Astra splits by region among 3 sub-agents then integrates ( 81.59 ).

Fortress : Both use a single main builder, but Opus 5.5 first sends 7 sub-agents to study blueprints ( 64.31 ), while GPT-6 Astra builds while reviewing ( 55.35 ).

Same domain, different orchestration rationales: parallel component fabrication vs. pre-building context for a single builder. No universal orchestration pattern exists; effective orchestration adapts to the task's inherent structure.

Conclusion

As tasks grow longer and more complex, the bottleneck shifts from individual model strength to how the entire system is organized. WorkflowArena shows that today's LLMs have substantial room for improvement in multi-agent orchestration, and that a well-designed workflow can markedly lift the output of the same model pool. The benchmark's full delivery and workflow logs enable further research: heterogeneous systems where strong models orchestrate and weak models execute may be the next direction. The competition of system intelligence—from "one agent completes a task" to "one agent organizes a swarm to complete a task"—has just begun.