How Ant Group’s Ragent Redefines LLM‑Based AI Agents on Ray

Background

Ray is the underlying distributed framework used by OpenAI for large‑model training. Ant Group joined Ray early, contributed over 26% of its core code, and now operates more than 1.5 million CPU cores, making it the second‑largest contributor worldwide and the largest user in China.

Motivation

Building on years of Ray experience, Ant developed multiple engines—Realtime, Mobius, Mars, and the Multi‑Tenant architecture—to support streaming, machine learning, inference, and scientific computing, culminating in the Unified AI Serving framework that integrates offline, online, and AI deployment workloads.

Design & Implementation

The Ragent framework structures an LLM‑based agent into four essential modules:

Profile : Defines the agent’s persona and role, such as a gentle travel assistant that can manage itineraries and perform data analysis.

Memory : Consists of Knowledge (domain and prior knowledge) and Experience (recorded dialogues, user queries, reasoning steps, and outcomes) to enable continual learning and error avoidance.

Planning : Decomposes complex tasks into manageable subtasks using algorithms like Chain‑of‑Thought or Tree‑of‑Thought, similar to flowcharts in software design.

Action : Executes tasks based on memory and plans, featuring Function Calling to invoke external services or interact with physical devices such as robotic arms.

These four modules constitute the core components required for a robust LLM‑based AI agent.