Supervisor Agent Pattern: Dynamic Task Routing, Bottleneck Risks, and Control Patterns

From Workflow to Supervisor: Why Dynamic Routing Matters

The previous article introduced Workflow where code controls a fixed sequence of nodes. However, real evaluation tasks often need dynamic routing: early findings (cost issues, compliance risks, performance anomalies) change what should happen next. A central Supervisor agent makes routing decisions based on current evidence, while runtime constraints (budget, permissions, timeouts, hard stop conditions) remain enforced by the system.

Control Relationship: Hub-and-Spoke

The Supervisor holds control: it understands the goal, selects child agents, defines subtask boundaries, judges result sufficiency, and synthesizes the final deliverable. Child agents only handle their own responsibilities and usually don't know each other's intermediate processes. This hub-and-spoke structure creates a clear audit trail — every dispatch originates from the Supervisor and every result returns to it.

User Task
  ↓
Central Supervisor
  ↙   ↓   ↘
Performance Agent  Security Agent  Business Agent
  \   ↓   /
   Results return to Supervisor
         ↓
    Final Plan

In traditional software engineering terms, the Supervisor combines scheduler, orchestrator, and controller. The key difference: traditional systems rely on queues, rules, and fixed state machines, while the Supervisor delegates intermediate judgments to a model. This brings flexibility but also instability, so hard guardrails must stay in the runtime and tool layers.

Dynamic Routing in Action: Evaluating a Model for Customer Service

The article walks through a concrete trace:

1. Supervisor reads goal, existing materials, and budget
2. Finds missing latency/cost data → calls Performance Agent
3. Performance Agent returns test results, flags peak latency anomaly
4. Supervisor adds Security Agent to check fallback paths and data flows
5. Security Agent finds external interface receiving sensitive fields → returns blocking condition
6. Supervisor asks Business Agent to evaluate alternatives, then produces unified technical plan

Step 4 is not a fixed next node; it is triggered by the latency anomaly. The security review examines fallback paths because latency issues may route more requests to backup interfaces, altering data flows. If a blocking risk appears, the Business Agent may never be called, and the task pauses for human confirmation.

Logging Requirements: Four Questions Every Dispatch Must Answer

To enable replay and debugging, logs must capture: subtask — Which agent was asked to solve what specific problem? reason — What current evidence triggered this dispatch? input_ref — Which version of the task and evidence did the child agent actually read? stop_condition — What result would make the Supervisor stop, retry, or escalate?

Without reason and input_ref, it becomes impossible to distinguish a Supervisor misjudgment from a child agent working with stale data.

Two Control Patterns: Agents as Tools vs. Handoff

Agents as Tools (OpenAI Agents SDK)

The Supervisor exposes specialists as callable tools. The specialist runs and returns a structured result; the Supervisor retains control and continues reasoning.

Supervisor ──call performance tool──> Performance Agent
Supervisor <──return structured result── Performance Agent
Supervisor continues reasoning and dispatching

This suits scenarios where the Supervisor must continuously compare multiple results, control final answer format, or keep all routing in the outer loop. It resembles traditional function calls and service orchestration: clear input/output contracts must be maintained.

Handoff (Control Transfer)

The current agent invokes a handoff tool, passing the conversation and necessary context to a designated agent. The recipient continues execution; whether control returns to the original Supervisor depends on subsequent handoff relationships. The deciding factor is who owns the next decision after handoff .

From an engineering perspective, handoff is an explicit responsibility transfer. If the task must return to the Supervisor for closure, that "return" must be encoded in the process. Without recorded ownership, it's hard to tell whether triage, execution, or closure failed.

In a Supervisor architecture, handoff can still serve central scheduling: child agents explicitly hand back to the Supervisor, which then decides the next step. If a child agent instead chooses another peer after handoff, the pattern shifts toward Swarm and should not be conflated just because both use the term "handoff".

Closure Is Not Just a Summary

The Supervisor's final text is a deliverable, not a system state. To decide whether to stop, the Supervisor must verify against the task context:

Evidence completeness — e.g., Performance Agent must return test environment, sample slices, latency results, and failure cases, not just "performance is good".

Blocking conditions handled — A security risk cannot be papered over by summary tone; it requires re-dispatch, pause, or human approval.

Budget exhaustion — Each dispatch adds model calls, context, and latency costs. Limits on agent count, tool calls, retries, and total time must exist; otherwise "consult one more expert" becomes an endless default.

Andrej Karpathy calls this context engineering : managing the task, evidence, versions, and budget inside the context window, not just writing prettier prompts. Child agents should return structured conclusions, evidence references, and failure reasons so the center can keep context within usable bounds.

Replayable Central State

A minimal auditable state includes:

task_id
goal
dispatch_history[]
subtask_results[]
evidence_refs[]
budget_remaining
quality_status
termination_reason

termination_reason

is critical: it must distinguish completion due to satisfied criteria from budget exhaustion, risk blocking, child agent timeout, or human takeover. Without it, success and abandonment look identical in the final text.

Security Boundaries: Meta's Rule of Two

Central control does not automatically solve security. Customer service data may come from external documents, involve user privacy, and trigger ticket writes. Meta's Agents Rule of Two advises that until prompt injection is reliably detected, a single session should not simultaneously possess three capabilities: handling untrusted input, accessing sensitive systems or private data, and changing state or communicating externally. If all three are needed, insert human approval, independent verification, or a new context window. The Supervisor can surface risks but cannot replace permission isolation and tool-level validation.

Why the Center Becomes a Bottleneck

Anthropic's orchestrator-subagent pattern matches this structure: a lead agent plans and dispatches, subagents work in isolated contexts, and results return to the center. It works well when subtask boundaries are clear and interdependencies are low (e.g., code review: security, test coverage, style, architecture can be checked independently then aggregated).

Anthropic identifies two limitations:

Information bottleneck — A new fact discovered by one subagent must pass through the center, which must also recognize that this fact affects another subagent; otherwise the dependency is not re-routed.

Default serial execution limits throughput — Running subagents sequentially incurs multi-agent token costs without parallel speed gains. Anthropic's Research system therefore launches multiple search agents in parallel, requiring each subtask to define goal, tools, output format, and boundaries. They observed failure modes: over-dispatching for simple queries, duplicate searches, and excessive updates.

Google Research's controlled evaluation ( Towards a science of scaling agent systems ) tested 180 agent configurations. On the parallelizable Finance-Agent task, centralized architecture improved over single-agent by 80.9% ; on the strictly sequential PlanCraft task, multi-agent approaches decreased performance by 39% to 70% . This is not a universal guarantee for Supervisor; it shows centralized scheduling only helps when the task is genuinely decomposable.

The same study reports error amplification: independent parallel systems without mutual verification amplified errors 17.2× ; center-coordinated systems amplified 4.4× . The center can be a verification bottleneck, but only if it actually checks results rather than merely concatenating text.

Karpathy's autoresearch experiments hit the same inflection point. He viewed synchronous single-threaded code commits as a scaling limit and envisioned agents asynchronously exploring different research directions. In another thread, he described an agent planning successive experiments based on continuous results, accumulating ~700 adjustments to reduce a metric from 2.02 hours to 1.80 hours. The key takeaway is not the "700" number but that every round had a verifiable result, giving grounds for the next dispatch .

Back to the example: Supervisor suits dynamic tasks where intermediate results can change the route, yet each branch has clear acceptance criteria. Simply "dispatch a few more agents to look" without evaluation standards quickly turns the center into a queue point and summarizer.

When to Choose Supervisor

The author uses four criteria:

Subtask boundaries can be clearly defined, each with independent inputs, outputs, and acceptance criteria.

The next step depends on intermediate discoveries; a fixed order would require frequent changes.

A unified deliverable is needed, or a single point must centrally enforce budget, permissions, and stop decisions.

Subagents do not need to continuously exchange large amounts of intermediate state directly.

Open-ended research, complex analysis, and domain-split evaluation tasks often fit this shape.

For stable sequences where every node can be predefined and failure handling rules are fixed, revert to Workflow. Letting the model decide the next step only adds latency and uncertainty.

Conversely, if subagents need long-lived context and continuous task ownership, intermediate discoveries must notify multiple collaborators directly, or the task requires peer-level agents freely relaying work, Supervisor may be insufficient. The former suggests Agent Teams; the latter points to shared state, message buses, or Swarm. Piling more dispatch prompts onto a central supervisor usually won't solve these.

Design Checklist: Beyond a "Manager Role"

The difficulty lies in turning central decisions into an inspectable control plane. For the customer service evaluation task, the author checks four things:

Dispatch must be explainable — Routing rationale should rest on task state and evidence, not solely on an untraceable model output.

Stop conditions must be explicit — Criteria satisfaction, budget exhaustion, risk blocking, and human takeover should be distinct states.

Results must return under contract — Subagents return structured results, evidence references, failure reasons, and versions, not unreusable long text.

Center failure must be handled — Supervisor timeout, context bloat, or repeated dispatching should trigger pause, resume, budget reduction, or human fallback.

Thus, Supervisor is not about concentrating "intelligence" in one agent, but concentrating dynamic routing in an auditable location . Whether the center can close the loop depends on whether state, evidence, and termination conditions are more reliable than the final prose.

When a Supervisor must manage multiple Supervisors, the problem shifts from "how the center dispatches" to "how multiple centers isolate context and responsibility." The next article covers Hierarchical architectures.

References

Anthropic, Multi-agent coordination patterns: Five approaches and when to use them (https://claude.com/blog/multi-agent-coordination-patterns)

Anthropic, How we built our multi-agent research system (https://www.anthropic.com/engineering/multi-agent-research-system)

Google Research, Towards a science of scaling agent systems: When and why agent systems work (https://research.google/blog/towards-a-science-of-scaling-agent-systems-when-and-why-agent-systems-work/)

OpenAI Agents SDK, Tools: Agents as tools (https://openai.github.io/openai-agents-python/tools/)

OpenAI Agents SDK, Handoffs (https://openai.github.io/openai-agents-python/handoffs/)

Andrej Karpathy, X post on async large-scale agent collaboration (https://x.com/karpathy/status/2030705271627284816)

Andrej Karpathy, X post on autoresearch experiment iteration and agent swarm (https://x.com/karpathy/status/2031135152349524125)

Andrej Karpathy, X post on context engineering (https://x.com/karpathy/status/1937902205765607626)

Meta AI, Agents Rule of Two: A Practical Approach to AI Agent Security (https://ai.meta.com/blog/practical-ai-agent-security/)

Simon Willison, New prompt injection papers: Agents Rule of Two and The Attacker Moves Second (https://simonwillison.net/2025/Nov/2/new-prompt-injection-papers/)