Unlocking Agentic Reasoning: A Deep Dive into the New LLM Paradigm

Agentic Reasoning Overview

Agentic Reasoning reframes large language models (LLMs) as autonomous agents that can plan , act , and learn . The paradigm is organized into three hierarchical layers—Foundational, Self‑Evolving, and Collective—each of which can be optimized either in‑context (prompt‑level) or via post‑training (fine‑tuning or reinforcement learning).

1. Foundational Reasoning

This layer provides the core capabilities required for a single agent operating in relatively static environments.

1.1 Planning

In‑Context Planning : task decomposition, workflow design, and tree‑search algorithms such as BFS, DFS, A*, and Monte‑Carlo Tree Search (MCTS). Formal plan representations may use PDDL or executable code.

Post‑Training Planning : reward‑shaping and optimal‑control techniques, including trajectory optimization and diffusion‑based control, to improve long‑term decision making.

1.2 Tool Use

Contextual Integration : zero‑shot or few‑shot prompting (e.g., ReAct , ART , ChatCoT ) to invoke APIs, run code, or interact with external systems.

Post‑Training Integration : fine‑tuning or reinforcement‑learning (e.g., Toolformer , ToolLLM , ToolRL ) to learn reliable tool‑calling policies.

Orchestration Integration : coordinated multi‑tool pipelines that manage dependencies (e.g., HuggingGPT , OctoTools , ToolChain* ).

1.3 Search

Dynamic retrieval, knowledge‑graph traversal, and web browsing to acquire up‑to‑date information. Representative systems include Self‑RAG , DeepRAG , and WebGPT .

2. Self‑Evolving Reasoning

This layer enables continuous improvement from experience.

2.1 Feedback Mechanisms

Reflective Feedback : self‑critique and trajectory correction (e.g., Reflexion , Self‑Refine ).

Parameter Adaptation : incorporation of new training data to update model weights (e.g., AgentTuning , ReST , Distill‑CoT ).

Validator‑Driven Selection : external signals guide output selection (e.g., ReZero , CodeRL , SWE‑bench ).

2.2 Memory Dimensions

Contextual Use : store dialogue history, workflow state, and execution trajectories for immediate context augmentation.

Structured Representation : knowledge graphs and multimodal memory structures support relational and cross‑modal reasoning.

Post‑Training Control : reinforcement‑learning‑based memory management that decides when to update, summarize, or forget information.

2.3 Evolution of Core Abilities

Planning Evolution : automatic task generation and strategy refinement (e.g., SCA , Self‑Rewarding , RAGEN ).

Tool Evolution : synthesis of new tools from language descriptions (e.g., LATM , CRAFT , ToolMaker ).

Search Evolution : knowledge synthesis and adaptive retrieval policies (e.g., Reflexion , MemOS ).

3. Collective Multi‑Agent Reasoning

Extends intelligence from a single agent to coordinated systems.

3.1 Role Taxonomy

Leader/Coordinator : decomposes global goals and arbitrates conflicts.

Worker/Executor : carries out concrete actions.

Critic/Evaluator : checks output quality and detects risks.

Memory Keeper : maintains long‑term knowledge bases.

Communication Facilitator : manages messaging protocols.

Domain‑specific roles (e.g., software‑engineering, finance, law, healthcare, education, biomedicine, music) are built on this taxonomy.

3.2 Collaboration & Division of Labor

Manual Pipelines : hand‑crafted cascades that are interpretable but inflexible.

LLM‑Driven Orchestration : systems such as AutoGen , Magentic‑One , and MAS‑GPT dynamically allocate tasks.

Graph Topology Optimization : learning optimal communication structures with GommFormer , AgentPrune , and AFlow .

Strategy‑Based Training : reinforcement‑learning approaches ( MAGRPO , MHGPO , COPY ) that optimize collaborative policies.

3.3 Multi‑Agent Memory Management

Architecture : hierarchical (e.g., G‑Memory) vs. flat (Intrinsic Memory Agents).

Topology : centralized (SEDM), distributed (Collaborative Memory), or shared‑pool designs.

Content : semantic decomposition (MIRIX), task‑oriented chunks (LEGOMem), or cognitive‑stage representations (MAPLE).

Management : summary‑forget strategies (Lyfe Agents) or filter‑validate pipelines (AGENT‑KB).

4. Application Domains

Mathematical Exploration & Code Generation : AlphaEvolve , OpenHands , Cursor .

Scientific Discovery : ChemCrow , Coscientist , The AI Scientist .

Embodied Intelligence : Voyager , SayCan , CosmosReason1 .

Healthcare : MedAgent‑Pro , TxAgent , MDAgents .

Autonomous Web Exploration : WebArena , Mind2Web , DeepResearcher .

5. Evaluation Benchmarks

Benchmarks are grouped by the capability they assess.

Tool Use : ToolBench , APIBench , T‑Eval (accuracy of single‑ and multi‑turn tool calls).

Search : WebArena , Mind2Web , FinBrowseComp (retrieval and integration).

Memory & Planning : LOCOMO , LongMemEval , ALFWorld (long‑term memory retention and planning consistency).

Multi‑Agent Collaboration : AgentBench , MultiAgentBench , MAgIC (cooperation, competition, social reasoning).

https://arxiv.org/pdf/2601.12538
Github: https://github.com/weitianxin/Awesome-Agentic-Reasoning