JD Merchant Intelligent Assistant – Multi‑Agent System Architecture, Planning, and Evaluation

JD Merchant Intelligent Assistant is a 24/7 AI‑driven service for e‑commerce merchants, built on a large‑language‑model (LLM) based multi‑agent system that mimics real‑world merchant team collaboration.

The system evolved through three stages: (1) B‑mall ticket auto‑reply using LLM + RAG without tool calls; (2) JD 招商站 with a single agent, low accuracy and hallucinations; (3) JD Mai Assistant introducing a master‑subagent multi‑agent architecture, achieving >90% decision accuracy and second‑level response latency.

Multi‑Agent Planning key technologies integrate four model categories: LLM for goal extraction, Embedding for fast tool matching, Tools DAG for multi‑path inverse reasoning, and Operations‑research optimization for planning efficiency. The ReAct paradigm enables dynamic planning updates at each execution step.

输入总结模型的目标是针对用户历史的会话记录与本轮的提问分析其具体意图，作为Master Agent的思考的核心环节，需要对其意图总结效果进行评价。
1、自动化评价方案：
1.评价方法：以高阶模型（例如：GPT‑4o）作为裁判模型，结合用户当前轮次提问与历史的会话记录，对线上推理的准确性进行评价。
2.自动化评分指令（简化）：
你是一个擅长问题意图理解的专家。现在需要你评估一个电商平台AI助手对于商家用户提问的意图理解质量，并要求你从以下维度对回答进行评估，评分为0‑10分，分数必须是整数:1.正确性:意图是否正确表达出用户当前的问题；2.关联性:意图是否正确关联历史对话；3.和人工理解相似程度:与人工标注的意图相似度。

Evaluation methods include full‑link ReAct performance scoring, local Reward Models for thought/action/observation assessment, and customizable business rules. Both automated and manual reference cases are supported.

Online inference uses a master‑agent/sub‑agents hierarchy for task decomposition and distributed scheduling, with a standard communication protocol ensuring efficient multi‑step collaboration and global chain‑of‑thought planning.

Sample generation techniques cover offline standardized corpus creation for LLM training and continuous online inference labeling via Reward Models, enabling scalable data accumulation and model improvement.

The article concludes with an outlook on the universality and replicability of the multi‑agent approach for other industries, its status as a benchmark case in e‑commerce AI, and a recruitment invitation for engineers experienced in SFT, RLHF, RAG, Multi‑Agent, and related technologies.