Inside Ant Group’s Ragent: Building Scalable AI Agents on Ray

Background

Ant Group has been collaborating with the open‑source Ray project since 2017, contributing over 26% of Ray’s core code and becoming the second‑largest contributor worldwide. Today Ant operates more than 1.5 million CPU cores and runs a large Ray community in China.

Motivation

To support the era of large‑model AI, Ant built Unified AI Serving, a framework that unifies offline, online, inference and deployment workloads across its massive infrastructure. The need for a multi‑tenant, Ray‑based agent system led to the creation of Ragent.

Design & Implementation

Ragent follows a modular architecture for LLM‑based agents, consisting of four essential components:

Profile : defines the agent’s persona and role, such as a gentle travel assistant that can manage itineraries and perform data analysis.

Memory : split into Knowledge (domain and prior knowledge) and Experience (past dialogues, user queries, reasoning steps, and action outcomes) to help the agent learn from history and avoid repeated mistakes.

Planning : breaks complex tasks into manageable sub‑tasks using algorithms like Chain‑of‑Thought or Tree‑of‑Thought, similar to flow‑chart decomposition in software design.

Action : executes real‑world tasks based on the plan and experience. A key feature is Function Calling, which lets the model invoke external services or interact with physical devices such as robotic arms.

These four modules constitute the core components Ant considers necessary for any large‑language‑model‑driven agent.