7 Multi‑Agent Design Patterns Every AI Engineer Should Know

Introduction

Multi‑agent systems combine several specialized agents with mechanisms such as reflection, planning, reasoning, memory, retrieval‑augmented generation (RAG), tools, and a multi‑agent coordination protocol (MCP). This summary outlines the principal architectural patterns and practical considerations for building such systems.

Seven Multi‑Agent Design Patterns

1. Workflow (Prompt Chaining)

Agents are executed sequentially, each passing its output as the next agent’s input. Typical use‑cases include code generation → code review → deployment, ETL pipelines, and multi‑step reasoning where earlier context guides later processing.

2. Routing

The routing pattern introduces conditional logic so a controller agent dynamically assigns a request to the most appropriate specialized agent. Four common implementations are:

LLM routing – the controller uses a language model to output an explicit routing identifier or wraps downstream agents as tool functions (implicit routing).

Embedding routing – queries are embedded and matched to the most semantically similar route.

Rule‑based routing – hard‑coded keywords, patterns, or structured data determine the path.

Small‑model routing – a lightweight classifier trained on a small labeled dataset makes routing decisions.

Example pseudo‑code for LLM routing:

class Router:
    def __init__(self):
        self.model = ChatModel(model_name="gpt-4", api_key="", stream=False)
        self.routes = {
            'code_help': {
                'description': 'programming, code assistance',
                'handler': self.handle_code_question
            },
            'general_chat': {
                'description': 'casual conversation',
                'handler': self.handle_general_chat
            }
        }
        # Pre‑compute embeddings for route descriptions
        self.route_embeddings = {}
        for name, info in self.routes.items():
            self.route_embeddings[name] = self.model.encode([info['description']])

    def route_query(self, user_question):
        q_emb = self.model.encode([user_question])
        similarities = {name: cosine_similarity(q_emb, emb)[0][0]
                        for name, emb in self.route_embeddings.items()}
        best_route = max(similarities, key=similarities.get)
        handler = self.routes[best_route]['handler']
        response = handler(user_question)
        return {'route': best_route, 'confidence': similarities[best_route], 'response': response}

3. Parallel

Multiple agents run concurrently, each handling a sub‑task such as web crawling, retrieval, or summarization. Their outputs are merged into a single result, reducing latency in high‑throughput pipelines (e.g., document parsing, API orchestration).

4. Loop (Iterative)

Agents repeatedly refine their output until a quality threshold is met. This pattern is suited for proofreading, report generation, or creative iteration where the system “thinks → acts → re‑thinks” until the final answer satisfies the criteria.

5. Aggregation

Several agents generate partial results, and a master agent consolidates them into a final output. Common in retrieval‑augmented generation, voting systems, and consensus‑building scenarios.

6. Network

Agents communicate freely without a strict hierarchy, sharing context dynamically. This pattern powers simulations, multi‑agent games, and collective reasoning systems. The agentscope‑samples repository demonstrates a nine‑agent “Werewolf” game.

7. Hierarchical

A top‑level planner assigns sub‑tasks to worker agents, monitors progress, and makes the final decision—mirroring a manager‑team structure. Used for intent recognition, middleware orchestration (e.g., Redis, Elasticsearch), and other scenarios requiring coordinated delegation.

Key Design Considerations

Effective multi‑agent systems minimize friction between agents: avoid duplicate work, ensure each agent knows when to act or wait, and orchestrate communication so the collective behavior exceeds that of any single component.

Open‑Source Agent Frameworks

AutoGPT – GitHub ★ 180 k stars

Dify – GitHub ★ 118 k stars

AutoGen – GitHub ★ 51.4 k stars

CrewAI – GitHub ★ 40.1 k stars

LangGraph – GitHub ★ 20.6 k stars

When to Use an Agent Framework

Agent frameworks excel when a problem cannot be exhaustively enumerated, requires cross‑system verification, and needs dialogue‑driven clarification, negotiation, and decision‑making. Pure static workflows become brittle under such conditions.

Illustrative Logistics Scenario

For a query like “My package hasn’t arrived, what should I do?” a static workflow would need a combinatorial explosion of branches (status × user tier × policy × exception). An agent framework can dynamically route the request, retrieve real‑time logistics data, clarify missing information, and apply policy reasoning without pre‑defining every branch.

Core Capabilities Example (AutoGen / CrewAI)

Intent Recognition + Clarification – A planner agent splits the user request into distinct intents (e.g., delivery issue, refund request, insurance query) and asks follow‑up questions for missing details.

Cross‑System Evidence Gathering – Specialized agents query order‑management systems, logistics APIs, billing records, CRM, and insurance databases to collect relevant evidence.

Policy Reasoning & Compliance – A policy agent applies business rules (e.g., holiday delay + premium membership + insurance) to compute compensation ranges and decide whether human review is required.