Evaluating JoyAgent‑JDGenie: A Lightweight Multi‑Agent AI Framework in Action

Background and Purpose

JoyAgent‑JDGenie is a newly open‑source multi‑agent product that works out‑of‑the‑box for general questions and can be extended with sub‑agents or MCP‑like tools for custom scenarios. It is lightweight and has no platform dependencies.

The author conducted a comprehensive test using both the online demo and a locally deployed instance to assess overall performance.

Test URLs: https://autobots.jd.com/genie

Source code: https://github.com/jd-opensource/joyagent-jdgenie

Test Cases

1. From the official site, request a recommendation of barbecue restaurants near Shichahai, Beijing, within three subway stations and an average price of 100 CNY.

2. Deploy the service locally and ask for a 5‑day travel plan from Beijing to Alshan for two people with a budget of 5,000 CNY each.

Evaluation Focus

Overall planning time

Interaction experience

Result quality

Complexity of local deployment

Detailed Evaluation

3.1 Restaurant Recommendation

Question: “I want to visit Shichahai in Beijing; recommend three barbecue restaurants within three subway stations and an average price of 100 CNY.”

Analysis: The agent must identify location, subway range, price, and cuisine.

Result:

a. The initial reasoning chain (shown in the image) demonstrates rapid and accurate task parsing.

b. Interaction interface: a clean workspace on the right shows browser content, real‑time query results, and dynamically generated markdown.

c. Final output took about 4 minutes, fulfilling all requirements, providing query results and a markdown document. The workspace offers three saving options: export to Joyspace, download the markdown file, or copy to clipboard.

The generated content matches the criteria (three‑station subway, 100 CNY average, barbecue) and includes detailed explanations for each area, enhancing the user experience.

3.2 Travel Planning

The author installed a local version to test out‑of‑the‑box capability.

3.2.1 Dependency Installation and Environment Setup

a. Run sh check_dep_port.sh to list missing dependencies and follow the prompts to install them.

b. Edit application.yml and env_template to configure the LLM; the author used OpenAI’s gpt‑4.1 model.

c. Start the service with sh Genie_start.sh.

The installation and startup are straightforward with minimal platform dependencies.

3.2.2 Executing the Travel Planning Task

Question: “Two people, each with a budget of 5,000 CNY, want to travel from Beijing to Alshan for 5 days starting July 30. Provide a detailed itinerary.”

Analysis: Identify start and destination, duration, budget, travel dates, and required extensions such as weather, local cuisine, and attractions.

Thought process is illustrated in the following screenshots.

Process log:

Result screenshots:

The entire planning took just over five minutes. The basic itinerary is correct, daily schedules are reasonable, and the extended content provides detailed attraction and food introductions.

Conclusion and Recommendations

This framework, as a fully open‑source lightweight solution, performed excellently in the tests:

High‑efficiency interaction : rapid responses, concise operation paths, and reasonable request frequency.

Smart and convenient result saving : flexible archiving strategies reduce user effort.

Lightweight, out‑of‑the‑box deployment : minimal dependencies, no complex configuration.

Comprehensive and reliable content generation : thorough coverage of basic and extended information with stable output quality.

Minor optimizations are possible, but overall performance is outstanding, especially the interaction layer, providing a solid foundation for further development and testing.