Can Large Language Models Truly Plan? Unpacking Agent Frameworks

Why the interview question matters

Interviewers often ask whether a large language model (LLM) can plan on its own to test a candidate’s understanding of the underlying mechanisms of agent frameworks. The correct answer reveals that most apparent planning is actually scripted in prompts or code, not genuine model reasoning.

Current state: pseudo‑planning

In mainstream agents like ReAct, BabyAGI, and AutoGPT, the planning step is embedded in the prompt or program structure. For example, a prompt may instruct the model to follow a "Think → Decide → Act → Observe" cycle:

You are an AI assistant. To complete tasks, always think step by step, consider tools you have, and reason before acting.
Use this format:
Think
Decide
Act
Observe

This template tells the model how to "pretend" to plan; each step is generated deterministically, akin to filling in a worksheet rather than autonomous reasoning.

"Task → Decompose → Execute → Record → Review → Continue" – the LLM only produces the textual description for each step.

Can LLMs truly plan?

LLMs can produce a static plan when asked (e.g., "Design a two‑week mobile app schedule"), but they cannot dynamically adjust the plan, incorporate environmental feedback, or continuously revise goals without external orchestration.

Dynamic plan adjustment

Feedback‑driven next‑step selection

Continuous goal correction

These capabilities are currently beyond the model itself, which is why frameworks add external logic to "fill the brain".

Where to put the logic: Prompt vs. Program

Three scenarios help decide the placement of planning logic:

1️⃣ Use prompts when you need creativity and flexibility

Complex, open‑ended tasks (e.g., brainstorming a promotion campaign)

Frequently changing processes where hard‑coding would be cumbersome

Divergent content generation such as copywriting or outline creation

2️⃣ Hard‑code in the program for high control, compliance, or safety

Fixed, low‑tolerance workflows (e.g., login, payment processing)

Regulated domains requiring auditability (legal, risk, medical)

Operations that depend on external services (databases, APIs) and must follow strict order

3️⃣ Hybrid approach is most common

Combine a stable code‑defined backbone (e.g., order → payment → shipment) with prompt‑driven flexible components (e.g., customer replies, recommendation text).

Fixed main flow + localized flexible planning

Practical case breakdowns

Case 1: E‑commerce customer‑service bot

Program part: authentication, return policy, API queries

Prompt part: conversational Q&A, product recommendations

Rules are hard‑coded; language interaction is delegated to the model.

Case 2: Enterprise knowledge‑base QA

Program part: access control, document retrieval

Prompt part: summarization, comparison, natural‑language explanation

The model speaks; the program fetches data.

Case 3: Contract review & compliance

Program part: legal rule engine, approval workflow, risk scoring

Prompt part: clause analysis, amendment suggestions

Compliance logic is fixed; textual analysis is handed to the model.

Case 4: Internal project‑management agent

Program part: task assignment, permission checks, reminders

Prompt part: requirement breakdown, communication advice, risk tips

The framework controls process; the model handles content.

How major frameworks implement planning

1️⃣ ReAct – instant planning (think‑act loop)

Prompt template forces a three‑step cycle: Thought → Action → Observation → Thought … . The model decides the next action, external code parses and executes it. Planning stays inside the loop, but the trajectory is bounded by the prompt.

Planning embedded in the loop

Model decides step‑by‑step

External code executes actions

2️⃣ MRKL – modular reasoning system

The model acts as a central brain, selecting from a list of tools (weather API, calculator, database, etc.) based on reasoning. Prompt lists available tools; each turn the model chooses which tool to invoke.

Strong dependence on prompt templates

Planning expressed as tool selection

Execution still handled by external code

3️⃣ BabyAGI – task‑loop scheduler

Maintains a task list and repeatedly executes: run current task → create new tasks from results → reprioritize → repeat. Core modules: Task Creation Agent, Task Prioritization Agent, Execution Agent. The LLM only fills in tasks; the loop is scripted.

Planning appears in task generation and ordering

External program drives the loop

LLM continuously “adds tasks”

BabyAGI seems self‑growing, but the script predetermines its trajectory.

4️⃣ AutoGPT – reinforced autonomous loop

Given a goal, the model can generate commands, browse the web, read/write files, summarize, update its plan, and continue. The main loop (generate action → execute → record → feed back → generate next action) is hard‑coded in code.

Model generates commands

Code executes and stores memory

Overall resembles “simulated free will”

AutoGPT feels the most autonomous, yet each step follows a predefined format.

Takeaway

Current “intelligent planning” in agent systems is authored by humans—prompt templates plus external code—rather than emergent model insight. Achieving genuine autonomous planning will likely require reinforcement learning, deeper multi‑agent architectures, and self‑reflection mechanisms.