Workflow in Multi-Agent Systems: Code-Controlled Routing, Node Checkpoints, and Replayable Pipelines

Problem Context: Enterprise LLM Integration Reveals Gaps

In an enterprise project integrating LLMs into online customer service, the model could retrieve documents and call tools, but pre-launch issues emerged: incomplete version tracking of reference materials, inability to reproduce benchmark tests, risk of sensitive fields leaking to unauthorized services, and lack of auditable justification for the release decision. These checks must happen sequentially — each gate must pass before the next begins.

Since the path is already fixed, why let an agent decide the next step at runtime? Workflow addresses this contradiction: when the path is stable, code owns routing; agents only reason inside nodes; checkpoints sit at node boundaries so failures can be retried, paused, or resumed from a stable node, making the entire chain replayable and auditable.

Three-Layer Responsibility Map

The article separates concerns into three layers:

Agent Node : Responsible for retrieval, analysis, generation, tool calls. Does not decide global routing or skip checkpoints.

Workflow Control Flow : Responsible for sequence, conditions, retries, timeouts, termination. Does not replace node-internal reasoning and tool operations.

Runtime State : Tracks current node, result versions, evidence, recovery points. Is not equivalent to chat history.

Workflow is not a "lite agent" nor merely chaining model calls with arrows. It keeps uncertainty inside nodes while moving routing, state, and recovery outside the model.

Four Architectures Compared: Who Decides the Next Step?

Four architectures compared on who decides the next step, information flow, and main cost:

Workflow : Next step decided by code and rules. Information flow: fixed chain or DAG. Main cost: rule changes usually require process changes.

Supervisor : Next step decided by a central agent. Information flow: central dispatch and aggregation. Main cost: central context and decision bottleneck.

Hierarchical : Next step decided by multi-level supervisors. Information flow: hierarchical up/down. Main cost: cross-layer debugging and permission boundaries more complex.

Swarm / Handoff : Next step decided by current agent. Information flow: peer relay. Main cost: routing rationale harder to audit uniformly.

These architectures can combine, but control ownership must stay unambiguous. For any framework, ask: does it give "next step" to code, a central agent, or the currently working agent?

Running a Concrete Pipeline: Model Evaluation for Customer Service

Minimal Workflow:

Data Collection → Benchmark Test → Security Review → Draft Proposal

Each node emits versioned, evidenced results:

Data Collection: leaves sourced model info and version records.

Benchmark Test: leaves environment, samples, results, and reproduction commands.

Security Review: leaves risk items, evidence, and handling recommendations.

Draft Proposal: consumes only results that passed prior checkpoints.

Inter-node payloads must carry version and evidence, not just "test completed". Downstream needs to know what was tested, in which environment, which cases failed, and whether the result can proceed.

Required Node Output Fields

input_ref      upstream result version
node           current node
attempt        current execution count
output_ref     this run's result version
quality_status passed / failed / pending
next_action    computed by runtime rules (retry, proceed, pause, escalate)

next_action

must not be decided by an agent casually writing "suggest continue". Node results persist first; runtime then computes the next step based on checkpoints, retry budgets, and permission rules. This distinguishes "agent didn't finish" from "runtime didn't release".

Agent Does Node, Code Does Control Flow

In the Benchmark Test node, the agent may choose test tools, diagnose failures, generate reports, and retry once within budget. Runtime manages: current node, max retries, timeout handling (fail vs. human review), which results may enter Security Review, and where to resume after scheduler restart.

This separation makes Workflow practical: models retain exploration space, but global order, state transitions, and recovery points are visible in code.

Runtime must also record an event chain, not just final text:

node_started
tool_called
artifact_written
quality_checked
node_succeeded
node_failed
retry_scheduled
workflow_paused

These event names aren't framework-mandated, but logging them gives handles for debugging and replay. When a task halts, you know the last successfully written result; on replay, you resume from the latest stable node, avoiding re-running completed model calls.

Checkpoints at Node Boundaries

Fixed order doesn't mean restart-from-scratch on failure. If Benchmark Test times out on the third test group, runtime saves timeout reason, input version, and logs; retries per node policy; if budget exceeded, marks task failed or pending — never lets Draft Proposal treat the test as passed.

Similarly, if Security Review finds an interface sending sensitive fields to a disallowed external service, the task stops at Security Review or rolls back to modify input. The Draft agent cannot "write away" the risk with an explanation.

Each node needs explicit completion criteria:

Data Collection : Minimum completion condition — sources accessible, version and timestamp recorded. Action on failure — add sources or pause.

Benchmark Test : Minimum completion condition — environment, data slice, and results reproducible. Action on failure — retry inside node; escalate after budget.

Security Review : Minimum completion condition — risk items have evidence; blocking conditions handled or explicitly escalated. Action on failure — return for modification or human confirmation.

Draft Proposal : Minimum completion condition — references only checkpoint-passed results. Action on failure — do not generate release conclusion.

Checkpoints keep errors near their source. Once a flawed result moves downstream, later agents tend to elaborate it rather than re-verify facts.

Figure 2: Checkpoints split failures into retryable and pause-needing categories; human intervention resumes from a stable node instead of re-running the whole chain.

Critical: every failure branch must have inspectable state — which input version a retry starts from, what evidence a pause leaves, which stable node recovery resumes from. Without these records, "recovery" is just a full re-run.

Same Workflow Expressed in Five Frameworks

The article maps the same fixed chain into LangGraph, CrewAI, Microsoft Agent Framework, OpenAI Agents SDK, and Claude Agent SDK. Differences appear in where control flow lives. This is not a capability ranking but an expression comparison.

LangGraph : Fixed chain usually lives in StateGraph, nodes, edges, conditional edges; state is an explicit object. Points to verify separately: routing, state, persistence, and debugging can be inspected independently. Docs separate predefined Workflows from dynamic tool-choosing Agents; also provide prompt chaining, parallelization, evaluator-optimizer patterns.

CrewAI : Fixed chain usually lives in roles, tasks, and sequential flow or Flow organizing nodes. Points to verify separately: business role/task mapping is intuitive, but must separately verify current version's state persistence, streaming, and failure recovery.

Microsoft Agent Framework : Fixed chain usually lives in application code composing sequential, conditional, and parallel steps. Points to verify separately: better for separating "what a node is" from "how steps connect"; specific Builder/API changes across versions — don't rely on old class names.

OpenAI Agents SDK : Fixed chain usually lives in application code controlling call order; sub-agents can be invoked as agents as tools. Points to verify separately: when sub-agent acts as tool, control stays in outer layer; handoff transfers control to another agent — don't mix their information flows.

Claude Agent SDK : If used as node execution unit, fixed order still needs external scheduling layer. Points to verify separately: shows that providing an Agent Loop ≠ providing business Workflow; node contracts, checkpoints, recovery strategies remain application responsibility.

LangGraph docs explicitly show StateGraph, conditional edges, persistence, debugging; OpenAI Agents SDK docs clearly distinguish agents-as-tools from handoffs. CrewAI, Microsoft Agent Framework, and Claude Agent SDK specifics must be checked against their current official docs and actual runs.

For this task: LangGraph may draw four nodes and state fields as a graph; CrewAI may start from roles and task relations; OpenAI Agents SDK requires app code to decide when to call which sub-agent; Claude Agent SDK can serve as execution node while outer layer maintains sequence and acceptance. All can participate in Workflow, but "who owns routing" doesn't vanish by switching SDKs.

Workflow's Explicit Trade-offs

Predictable latency but end-to-end latency accumulates across nodes; parallel gains limited when steps can't start simultaneously.

Easy replay but upstream errors propagate downstream. Google Research's analysis of agent systems places sequential dependency, communication cost, and error propagation in one framework: task dependency structure should precede agent count in architecture choice.

Clear audit trail but process changes require code changes. Frequent rule changes and growing branches can turn Workflow into an unmaintainable condition tree.

Practical signal: if runtime starts repeatedly asking the model "which agent now?", "skip this step?", "where to go after failure?", code is simulating a Supervisor. Stuffing more conditions into a fixed pipeline may be no simpler than admitting dynamic scheduling is needed.

When to Choose Workflow First

First check if the path is stable, then decide if multiple agents are needed. Approval flows, ETL, fixed content pipelines, model evaluation, and deployment checks typically exhibit:

Step order mostly unchanged.

Inputs/outputs clearly definable.

Node results verifiable by rules, tests, or human approval.

Failures retryable or pausable at boundaries.

Need for audit, replay, and explicit permission scopes.

Conversely, if the next step depends on newly discovered facts, requires ad-hoc expert selection, or user intent shifts continuously in conversation, fixed pipelines become clumsy. Then evaluate Supervisor, hierarchical collaboration, or handoff — because the problem shifts from "how nodes execute" to "who decides next step".

Put Deterministic Parts in Code First

In multi-agent systems, models most easily take over routing: what's next, when to stop, where to roll back on failure. For rules that can be written clearly, prefer code. Agents keep exploration space inside nodes; runtime owns state, retries, timeouts, checkpoints, and recovery boundaries — so when something breaks, you can pinpoint the exact step.

Workflow's value lands in replayability, auditability, and the ability to pause and resume.

When the path cannot be predetermined, the question becomes: who decides the next step? That's the subject of the next article on Supervisor.

References

Anthropic, Multi-agent coordination patterns: Five approaches and when to use them (https://claude.com/blog/multi-agent-coordination-patterns)

Google Research, Towards a science of scaling agent systems: When and why agent systems work (https://research.google/blog/towards-a-science-of-scaling-agent-systems-when-and-why-agent-systems-work/)

LangGraph, Workflows and agents (https://docs.langchain.com/oss/python/langgraph/workflows-agents)

OpenAI Agents SDK, Tools: Agents as tools (https://openai.github.io/openai-agents-python/tools/)

OpenAI Agents SDK, Handoffs (https://openai.github.io/openai-agents-python/handoffs/)

CrewAI, Flows (https://docs.crewai.com/en/concepts/flows)

Microsoft Agent Framework, Workflows (https://learn.microsoft.com/en-us/agent-framework/workflows/)

Claude Agent SDK, Overview (https://platform.claude.com/docs/en/agent-sdk/overview)