Augmented Language Models: Reasoning and External Tool Utilization

After AlphaGo’s 2016 victory and the emergence of ChatGPT in late 2022, large language models (LLMs) have attracted intense interest across many domains, including programming, knowledge retrieval, mathematics, and even claims of possessing child‑level cognition.

The author reflects on a recent survey of Augmented Language Models (ALM) and identifies two prominent abilities that emerge in sufficiently large LLMs:

Reasoning ability : the model can decompose complex tasks into a series of simpler sub‑tasks and solve them step‑by‑step, covering commonsense, mathematical, and symbolic reasoning.

External‑tool manipulation ability : the model can invoke external resources such as search engines, databases, calculators, or even physical devices like robotic arms.

These abilities are considered emergent, appearing only when model parameters exceed a certain scale (typically >10 B). Prompt engineering is a key technique to elicit reasoning, with variants such as zero‑shot prompts, few‑shot prompts, and Chain‑of‑Thought (CoT) prompting. CoT prompts provide intermediate reasoning steps, improving performance on benchmarks like GSM8K, especially for larger models.

Explicit methods such as the “scratchpad” approach train models to generate and follow intermediate computation steps, effectively supervising multi‑step reasoning during fine‑tuning.

Beyond reasoning, LLMs can augment their knowledge by retrieving external information. Retrieval can be sparse (bag‑of‑words) or dense (vector‑based). Works like REALM and IRCoT interleave retrieval with reasoning, allowing the model to fetch relevant passages and incorporate them into the reasoning process.

Recent developments also enable LLMs to act in the world: plugins for ChatGPT connect the model to thousands of external services; systems like WebGPT and BlenderBot use reinforcement learning from human feedback to browse the web or query retrievers when needed; and planning frameworks (e.g., PEER, ReAct) let LLMs generate plans and execute actions in simulated or real environments, including controlling robotic arms.

Overall, the survey highlights that LLMs’ reasoning and tool‑use capabilities are complementary: reasoning helps decompose tasks, while external tools provide up‑to‑date knowledge and physical actuation, moving LLMs closer to general artificial intelligence.