Rethinking Agent Composition: Single-Loop Skills vs. Sub-Agent Handoffs

The Problem with Default Multi-Agent Topologies

When building customer-service agents, teams often start by splitting domains into sub-agents — one for orders, one for after-sales, one for products — with a top-level orchestrator. This pattern appears in nearly every agent framework's landing page. However, in long-running sessions this topology creates structural friction: a single user utterance like "I want to return the blue coat I bought last week" simultaneously touches orders (last week's purchase), products (coat, color), and after-sales (return). Whichever sub-agent receives the turn first, the other two domains' context remains with the orchestrator. When the sub-agent finishes and control returns, session history, user preferences, and cart state must be moved across the boundary. Each handoff becomes a lossy state operation that also consumes extra model turns and latency.

A Production-Grade Alternative: Single Loop with On-Demand Skills

Anthropic's open-source commerce-agents reference implementation takes a different approach: a single agent loop owns the entire conversation, and long-tail capabilities are packaged as skills that load on demand. This is not a toy demo; it fully implements both shopper and merchant agents across retail, travel, telecom, and entertainment verticals, with the same definitions running on three different runtimes.

Reframing Composition Granularity as a Cache and State Problem

The key insight is that composition granularity should be decided by caching and state considerations, not code organization.

State

In a single loop, conversation context stays in one place — no handoffs, no leakage channels at boundaries.

Cache

Model-side context caches hit on prefix stability. Skills loaded on demand appear only in the few turns where they are used, so they do not pollute the common prefix of every turn. Once loaded, they sit at a stable position and can be reused from cache. Conversely, the resident system prompt appears in every turn's prefix, making it the most stable and cache-friendly segment.

Frequency-Driven Placement Rule

Where a capability lives depends on how often it is used:

Used in nearly every session (e.g., product search in a shopping scenario) → place in the system prompt, resident in the prefix cache, no extra load turn.

Used in a minority of sessions → make it a skill; pay one model turn to load it when hit, gaining a smaller, more stable prefix for all other turns.

The empirical threshold is roughly "used in more than one-third of sessions." Safety, compliance, and brand guardrails fall outside this spectrum — they always stay in the system prompt because skills are optional and you cannot gamble on them being loaded.

Skill Loading Is Not Free: Predictive Pre-Injection

Loading a skill consumes a model turn. The reference implementation mitigates this by pre-injecting skills before the first model call when observability signals (e.g., the landing page indicates the user will likely enter a specific flow) predict they will be needed. Pre-injection avoids the discovery turn.

When Sub-Agents Are Still Justified

The reference implementation identifies only two narrow cases:

Truly self-contained narrow tasks — e.g., deep research that carries its own independent context window and returns only a compact conclusion without sharing process state with the main session.

Genuine ownership transfer — handing the user to an independent domain agent that takes over the conversation. The litmus test is ownership: handoff means the receiver runs the conversation; delegation means the orchestrator retains the main session and shuttles context in and out on every call, which structurally guarantees per-turn degradation. Many teams mistake delegation for handoff — that is where the real problem hides.

Implications for Evaluation

Domain-split architectures tend to evaluate routing correctness and per-sub-agent performance. With a single-loop-plus-skills design, evaluation should target session state rather than conversation path. Test cases can start from a crafted mid-conversation state instead of a clean slate. Each "should refuse" case pairs with a "should serve" case; each "should ask first" pairs with a "should act directly." Missing negative cases are the most common gap. A single case spanning orders, inventory, and pricing simultaneously validates that the composition architecture does not leak state at boundaries.

Cost Model Driven by Cache Layout

Placing per-turn mutable content (timestamps, current page) at the top of the system prompt is the most common way to shatter the cache prefix. Such volatile fields belong at the end of the turn. A three-segment ordering that works well in production:

[Global segment] System prompt + tool definitions      ← cross-session bytes unchanged, hottest
[Session segment] User profile + turn history          ← stable within session
[Volatile segment] Current time / current page         ← always last

Production implementations with good hit rates achieve 90%+ cache hits; cache reads cost roughly one-tenth of a fresh creation. This gap means composition granularity decisions ultimately show up on the bill, not just in architecture review slides.

Takeaway: Three Questions Before You Draw the Topology

The conclusion is not that sub-agents should be eliminated — they remain the right tool for certain tasks. The valuable takeaway is a questioning framework: composition design should not start from a framework's example topology. Instead, answer three questions first:

What is the usage frequency of this capability?

Does it need to share state with the main session?

Who truly owns the conversation at this boundary?

For most long-session applications, the answers converge on a single loop with skills, relegating sub-agents to the two niches where they are irreplaceable. Next time you see a polished sub-agent topology diagram in a review, ask: are these boundaries transferring state, or merely transferring problems?