How Ant Group’s Ray‑Powered Ragent Redefines LLM‑Based AI Agents

This article introduces Ragent, Ant Group’s latest distributed agent framework built on Ray, and outlines its key components and motivations.

Background

Ray, originally developed by OpenAI for large‑model training, has been adopted by Ant Group since 2017. Ant contributed over 26% of Ray’s core code, becoming the second‑largest contributor worldwide, and now operates more than 1.5 million CPU cores in production while maintaining a vibrant Ray community in China.

Motivation

During the 2023‑2024 era of large models, Ant built Unified AI Serving in the United States, integrating offline, online, inference, and deployment workflows into a single framework that powers a core business scenario across its massive compute fleet.

Design & Implementation

Ragent is organized around four essential modules that together enable a large‑language‑model (LLM) based agent:

Profile : Defines the agent’s persona, such as a gentle travel assistant that can manage itineraries and perform data analysis.

Memory : Consists of Knowledge (domain and prior knowledge) and Experience (recorded dialogues, user queries, reasoning steps, and action outcomes) to improve future behavior.

Planning : Breaks complex tasks into manageable sub‑tasks using algorithms like Chain‑of‑Thought or Tree‑of‑Thought, similar to flowcharts in software design.

Action : Executes real‑world tasks based on experience and plans, featuring Function Calling to invoke external services or interact with physical devices such as robotic arms.

These four modules constitute the core components required for an LLM‑based agent in financial and other domains.