LLM Agent Fundamentals: ReAct Reasoning-Action Loop & Tool Calling Evolution

1. What Is an Agent?

The term "agent" has many definitions. The classic AI textbook Artificial Intelligence: A Modern Approach defines an agent as anything that perceives its environment through sensors and acts upon it through actuators. Google defines an AI agent as a software system that uses AI to achieve goals on behalf of users, exhibiting reasoning, planning, memory, and autonomy. Wikipedia notes that an LLM alone is not an autonomous agent because it cannot interact with dynamic environments, recall past behaviors, or plan future actions; it becomes an agent when supporting elements such as role, environment, memory, and tool use are added.

2. ReAct

Shunyu Yao's ReAct paradigm is a widely accepted foundation for LLM-based agents.

Before ReAct, LLMs used two main strategies:

Reason-only (e.g., Chain-of-Thought): the model reasons internally. This decomposes problems but suffers from hallucination and error accumulation.

Act-only : the model outputs actions to interact with external environments or APIs (e.g., querying a knowledge base). This fetches external information but lacks high-level reasoning and planning.

ReAct's core idea is to interleave reasoning and action , forming a more robust task-solving paradigm: the model alternately generates reasoning traces and action commands, then incorporates actual execution results into further reasoning, achieving synergy between reasoning and action.

2.1. Thought → Act → Observe

The workflow, illustrated with a paper example, proceeds in three steps:

LLM thinks about the task goal and historical feedback, establishing the logical basis for the next action.

LLM generates the next action (in the example, a Search command).

The action is executed against the external environment, and the observation is fed back to the LLM for the next TAO cycle.

The author demonstrates this manually using a prompt from an older LangChain version.

2.2. Manual Act Example

System prompt (from old LangChain):

Answer the following questions as best you can. You have access to the following tools:

Search: A search engine. Useful for when you need to answer questions about current events. Input should be a search query.
Calculator: Useful for when you need to answer questions about math.

The way you use the tools is by specifying a json blob.
Specifically, this json should have a `action` key (with the name of the tool to use) and a `action_input` key (with the input to the tool going here).

The only values that should be in the "action" field are: Search, Calculator

The $JSON_BLOB should only contain a SINGLE action, do NOT return a list of multiple actions. Here is an example of a valid $JSON_BLOB:

{
  "action": $TOOL_NAME,
  "action_input": $INPUT
}

ALWAYS use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action:
$JSON_BLOB
Observation: the result of the action
... (this Thought/Action/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin! Reminder to always use the exact characters `Final Answer` when responding.

The LLM call uses stop = ["Observation:"] to halt generation after the action.

client.chat.completions.create(
    model="deepseek-chat",
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_prompt},
    ],
    temperature=0,
    stream=False,
    stop = ["Observation:"]
)

Round 1

User question: "Who is Zhao Ergou's son? What is the square of his current age?"

LLM response:

Thought: I need to first find out who Zhao Ergou's son is, then get his current age, and finally calculate the square of his age. Since this is a question about a specific person, I need to use the search tool to look up relevant information.

Action:
{
  "action": "Search",
  "action_input": "赵二狗 儿子"
}

Manual search result: "Zhao Ergou's son is Qian Xiaomao."

Round 2

The previous thought, action, and observation are fed back. LLM responds:

Thought: Now I know Zhao Ergou's son is Qian Xiaomao. Next I need to find Qian Xiaomao's current age. Since age may change over time, I need to search for the latest information.

Action:
{
  "action": "Search",
  "action_input": "钱小猫 年龄"
}

Manual search result: "Qian Xiaomao is 9 years old this year."

Round 3

History updated. LLM responds:

Thought: Now I know Qian Xiaomao is 9 years old. Next I need to calculate the square of his age, i.e., 9 squared. This can be done with the calculator tool.

Action:
{
  "action": "Calculator",
  "action_input": "9 ** 2"
}

Manual calculation result: "Answer: 81"

Round 4

Full history provided. LLM produces final answer:

Zhao Ergou's son is Qian Xiaomao, he is 9 years old this year, the square of his age is 81.

Final Answer: Zhao Ergou's son is Qian Xiaomao, the square of his current age is 81.

2.3. Tool Evolution

When Yao et al. published ReAct in 2022, tools were defined only via prompts. Since then, tool-related technology has advanced rapidly in definition, invocation, and scope.

One track is training models to use tools . Meta's Toolformer (2023) is a milestone; today's mainstream large models all perform post-training with similar objectives, though implementations are far more complex and systematic than the original paper's pipeline.

Another track is standardization of tool-use APIs . OpenAI introduced Function Calling in 2023, allowing developers to define function interfaces (signatures/JSON schemas) in the Chat Completions API, letting the model intelligently output function names and arguments for execution. In 2025, OpenAI released a new Agents platform and Responses API, making it easier to build agents with stronger model-tool collaboration. (Details on tool-use APIs will be covered in subsequent articles.)

3. Summary

While a universal definition of "agent" remains elusive, ReAct is the foundational paradigm for LLM-based agents. From this perspective, two key characteristics emerge:

LLM serves as the agent's planner : the model autonomously generates Thought, plans Action, and in more complex paradigms performs Reflection.

Agents interact with the outside world via tools : the resulting Observations feed back into subsequent planning.

References:

Google Cloud: What are AI agents? - https://cloud.google.com/discover/what-are-ai-agents

Wikipedia: Large language model - https://en.wikipedia.org/wiki/Large_language_model

ReAct official page - https://react-lm.github.io/

Toolformer paper (arXiv:2302.04761) - https://arxiv.org/abs/2302.04761

OpenAI: Function Calling and other API updates - https://openai.com/index/function-calling-and-other-api-updates/

OpenAI Help: Function Calling in the OpenAI API - https://help.openai.com/en/articles/8555517-function-calling-in-the-openai-api