Enhancing Interactive Agents with Large Language Models: The SwiftSage Framework and Benchmark Analysis

With the rise of large language models (LLMs), the ability of models to reason in interactive environments has become a key focus. Traditional LLM usage is limited to pure text tasks, but recent work aims to enable agents that understand and act in physical worlds, such as moving a teapot onto a shelf.

Two benchmark environments are highlighted: AFL World, a simple textual simulation with limited challenges, and ScienceWorld, a more complex benchmark from AI2 containing over 30 tasks, 10 rooms per task, 25 actions, and 200 objects with varied physical states.

The article outlines the four interaction dimensions used in these environments: Observation (feedback after each action), Environment (visible objects and their states), Inventory (items the agent has collected), and Score (continuous reward reflecting task progress).

Baseline approaches largely rely on reinforcement learning (e.g., DRNN) or behavior cloning, but they struggle with large action spaces and long‑term planning. Knowledge‑augmented RL and language‑model‑based re‑ranking have shown modest improvements.

Three recent LLM‑driven methods are described: SayCan (generates K candidate actions and re‑ranks them), ReAct (adds self‑generated reasoning before each action), and Reflexion (reflects on failed attempts to improve future actions).

Building on these ideas, the SwiftSage framework combines a small T5‑large model for fast action prediction with a large LLM for planning when the small model encounters difficulties. The small model receives detailed task descriptions, recent action history, environment state, and inventory information, while the large model answers five targeted questions to produce a comprehensive plan that is then grounded into concrete actions.

Experimental results on ScienceWorld show that SwiftSage outperforms traditional baselines and other LLM‑based methods in both score and efficiency, achieving higher rewards with roughly one‑third the token consumption of pure LLM approaches.

The authors acknowledge limitations such as dependence on high‑quality oracle data, reliance on commercial LLM APIs, and challenges in obtaining rich feedback from real‑world robots. Future work includes expanding to more complex tasks, distilling planning ability into open‑source models, and tighter integration with physical robotic systems.

A brief Q&A addresses the feasibility of using RL for the small model, differences from Google’s multi‑agent work, and practical considerations for model selection.