How Ant Group’s Ragent Redefines Distributed LLM Agents with Ray

Ant Group shares its latest Ray‑based distributed agent framework, Ragent, which builds on the open‑source Ray platform that powers large‑model training at OpenAI.

Background

Ant Group has been contributing to Ray since 2017, launching its first workflow engine Geaflow in 2018. Over the years it built several Ray‑based engines such as Realtime, Mobius, Mars, and introduced a multi‑tenant architecture ahead of the Ray community.

Motivation

In the 2023‑2024 large‑model era, Ant developed Unified AI Serving to combine offline, online, inference and deployment for its 1.5 million‑core workloads. The next step is an AI Agent framework that leverages Ray.

Design & Implementation

The agent architecture consists of four essential modules:

Profile : defines the agent’s persona and role, e.g., a gentle travel assistant.

Memory : includes Knowledge (domain and prior knowledge) and Experience (past dialogues, user queries, reasoning and actions) to improve future behavior.

Planning : breaks complex tasks into sub‑tasks using algorithms such as Chain‑of‑Thought or Tree‑of‑Thought.

Action : executes tasks based on experience and plans, featuring function calling and possible interaction with physical devices.

These four modules constitute the core components required for a large‑language‑model‑based agent.