Mastering AI Agents with the Plan-and-Solve Design Pattern

Plan-and-Solve design pattern

Planner decomposes a complex request into an ordered list of executable sub‑tasks and returns a JSON‑encoded plan. Solver runs each step sequentially, invoking tools when needed and triggering replanning if a step fails. This "think‑then‑act" approach turns an open‑ended prompt into a concrete roadmap.

Why it improves on ReAct

ReAct interleaves reasoning and execution (“think‑while‑doing”). For simple queries this works, but on multi‑step problems the model can lose sight of the original goal, repeat steps, or omit sub‑goals. By keeping the full plan visible during execution, Plan-and-Solve maintains goal fidelity, reduces loops, and raises success rates.

Core roles

Planner : analyses the task, selects appropriate tools, and outputs a JSON array where each element contains step_number, description, optional tool, and expected_output.

Solver : executes the steps, calls the indicated tool (or the LLM directly), records success, and can request a new sub‑plan when a step fails.

End‑to‑end workflow

Receive task : e.g., "Help me plan a birthday party".

Generate plan : Planner returns a list such as

Determine theme and budget

Set date and venue

Compile guest list

Arrange food and drinks

Decorate the venue

Execute plan : Solver runs each step, calling tools (search, file I/O, calculator, etc.) as specified.

Aggregate results : After all steps succeed, Solver synthesises a final answer from the step outputs.

Concrete example – weekly report generation

Using ReAct, the model might start writing the report, forget project A, then scramble to add it, producing a disordered document. With Plan-and-Solve the process is:

Review completed tasks (query Jira/Tapd).

Summarise next‑week plan.

List encountered issues.

Generate the report document.

Each step is executed in order, guaranteeing a complete and logically ordered report.

Implementation details

The following Python class demonstrates a full Plan-and-Solve agent built on OpenAI’s function‑calling API. It includes planning, execution, dynamic replanning, and result aggregation.

class PlanAndSolveAgent:
    def __init__(self, llm):
        self.llm = llm

    def run(self, task, max_retries=2):
        # Stage 1: Planning
        plan = self.planner(task)
        # Stage 2: Execution with possible replanning
        results = []
        retry_count = 0
        i = 0
        while i < len(plan):
            step = plan[i]
            result = self.execute_step(step)
            results.append(result)
            if not result.success:
                if retry_count < max_retries:
                    new_plan = self.replanner(task, step, results)
                    plan = plan[:i] + new_plan
                    retry_count += 1
                else:
                    print(f"⚠️ Step {step.step_number} failed after retries, continue")
            i += 1
        # Stage 3: Aggregate results
        final_result = self.aggregate_results(task, results)
        return final_result

Key implementation points:

Plans must be executable – each step is a clear action with an optional tool.

Dynamic adjustment – on failure the replanner generates a new sub‑plan that replaces the remaining steps.

Result aggregation – a final LLM prompt combines successful step outputs into a coherent answer.

Planner prompt (simplified)

The planner asks the model to produce a JSON array. Example prompt fragment:

请为以下任务制定详细的执行计划：
任务：{task}
可用工具：
- search_web: 搜索网络信息
- read_file: 读取文件内容
- write_file: 写入文件内容
- calculator: 进行数学计算

返回 JSON 数组，每个元素包含 step_number、description、tool（如有）和 expected_output。

Execution logic

For each PlanStep, execute_step either calls a mock tool (search, read, calculator) or sends the description back to the LLM via self.llm.chat.completions.create. Success is recorded in a StepResult dataclass.

Replanning

If a step fails, the replanner receives the original task, the failed step description, and the list of already successful step numbers. It asks the LLM to generate a new JSON plan starting from the failed step, thereby avoiding the dead‑end.

Result aggregation

The aggregator builds a prompt that lists all successful step outputs and asks the LLM to synthesize a final, coherent answer.

Practical tips

Plan granularity: 3‑7 steps strike a balance between detail and overhead.

Leave slack: ensure the replanner can recover without a full restart.

Tool descriptions must be concise; the Planner selects tools based on these strings.

Avoid naïve concatenation of step outputs; use a synthesis prompt to produce a polished final answer.

Cold knowledge

Origin: Wang et al. (2023) *Plan-and-Solve Prompting: Improving Zero‑Shot Chain‑of‑Thought Reasoning by Large Language Models* (arXiv:2305.04091).

Relation to Chain‑of‑Thought: CoT = think‑while‑talking; Plan‑and‑Solve = think‑then‑act, better for multi‑step, goal‑driven tasks.

Hierarchical planning: a Planner‑generated sub‑task can itself be fed back into the Plan-and‑Solve loop, creating nested plans.

Analogy to classic project‑management techniques such as Work‑Breakdown Structure (WBS) and Sprint Planning.

Limitations: highly exploratory or creative work (e.g., novel writing) may benefit more from ReAct’s flexible “think‑while‑doing” style.

References

Plan-and-Solve paper: https://arxiv.org/abs/2305.04091

LangChain Plan‑and‑Execute docs: https://python.langchain.com/docs/modules/agents/agent_types/plan_and_execute.html

OpenAI Function Calling guide: https://platform.openai.com/docs/guides/function-calling