How to Build Scalable Multi‑Agent AI Systems with RocketMQ’s Asynchronous Messaging

Problem Statement

Multi‑Agent AI systems suffer from three core issues when using synchronous request‑reply communication:

Synchronous blocking – the supervisor must wait for each sub‑agent to finish, causing cascading bottlenecks and low concurrency.

Lack of fault tolerance – a failure or timeout in any sub‑agent aborts the whole task chain because there is no retry or checkpoint mechanism.

Imbalanced resource consumption – differing throughput among agents leads to overload of some agents and idle compute resources for others.

Solution Overview

RocketMQ for AI replaces the request‑reply pattern with an event‑driven, asynchronous architecture. Tasks are published as messages to a queue; the supervisor continues processing while sub‑agents consume tasks independently. The platform provides a lightweight topic model (LiteTopic) that supports millions of lightweight resources and dynamic subscriptions, enabling elastic scaling.

Key Features

Asynchronous communication – eliminates cascading blocking and allows the supervisor to dispatch many tasks concurrently.

Persistence & retry – requests and results are stored in the queue, providing checkpointing, automatic retries, and dead‑letter handling.

Fine‑grained scheduling – consumption rates, priority queues, and selective delivery can be configured to smooth traffic spikes and maximize AI compute utilization.

Typical Architecture

A three‑agent system consists of:

Supervisor Agent – receives user queries, publishes tasks to dedicated request topics, and subscribes to a response LiteTopic.

Weather Agent – consumes messages from WeatherAgentTask topic, performs weather queries, and publishes results to the response LiteTopic.

Travel Agent – consumes messages from TravelAgentTask topic, creates travel itineraries, and publishes results to the response LiteTopic.

Resource and Topic Setup

# 1. Create a VPC and an ECS instance (Linux) to host the agents.
# 2. Provision a RocketMQ for AI instance.
# 3. Create three topics:
#    - WeatherAgentTask   (standard message)
#    - TravelAgentTask    (standard message)
#    - WorkerAgentResponse (LiteTopic for lightweight responses)
# 4. Create consumer groups:
#    - WeatherAgentTaskConsumerGroup   (CLUSTERING mode)
#    - TravelAgentTaskConsumerGroup    (CLUSTERING mode)
#    - WorkerAgentResponseConsumerGroup (LITE_SELECTIVE mode, ordered delivery)

Agent Deployment

Enable the Baidu AI model service (百炼) and obtain an API key.

Create two AI agents on the Baidu platform using the provided model parameters and prompts: a weather‑assistant agent and a travel‑assistant agent.

Deploy the agents on the ECS server using the supplied startup scripts (e.g., sh start_weather_agent.sh and sh start_travel_agent.sh).

Workflow Execution

The user sends a query (e.g., “Plan a weekend driving itinerary around Hangzhou”).

The Supervisor Agent publishes the query to WeatherAgentTask.

The Weather Agent consumes the message, calls the weather model, and publishes the weather data to WorkerAgentResponse.

The Supervisor receives the weather result, composes a second request containing the weather information, and publishes it to TravelAgentTask.

The Travel Agent consumes the request, generates a detailed itinerary, and publishes the result to WorkerAgentResponse.

The Supervisor Agent consumes the final itinerary and returns it to the user.

All messages can be traced in the RocketMQ console by topic or LiteTopic, confirming that the system operates without tight coupling, with automatic retries, and with controlled consumption rates that protect downstream agents from overload.