How JD Retail’s AI Assistant Uses Multimodal LLMs to Boost E‑Commerce

JD Retail has built an AI assistant that mimics the collaborative workflow of a merchant team by coordinating a Master Agent with multiple Sub Agents, breaking complex business problems into sub‑tasks and dramatically improving solution accuracy and efficiency.

Technical Foundations

The system leverages the ReAct paradigm, multi‑model ensemble, and dynamic planning updates to precisely understand user intent. By incorporating embeddings and a Tools DAG, it reduces hallucinations in large language models (LLMs) and raises tool‑calling precision.

To handle the diverse multimodal product descriptions (text, images, video, live streams) on JD’s platform, the team adopted a Mixture‑of‑Experts (MoE) architecture that fuses different expert modules and task heads, enabling unified representation of heterogeneous media.

Key Applications

Merchants can interact with the assistant in natural language to obtain sales forecasts, marketing spend recommendations, pricing advice, and product selection guidance. An interactive “JD‑Jingyan AI Shopping Assistant” demonstrates how the model can suggest suitable televisions or other items based on user queries.

In recommendation scenarios, JD fine‑tuned state‑of‑the‑art multimodal models on e‑commerce data, producing richer product embeddings that combine textual and visual signals. Instead of fusing ID and modality features at the feature level, the model merges them at the representation layer, training with CTR as a supervision signal and later blending with traditional predictors for overall performance gains.

Team Culture and Growth Opportunities

The JD Retail technology team operates like a university research group, encouraging open discussion, cross‑functional collaboration, and rapid prototyping. Engineers are encouraged to engage directly with business operations—such as accompanying delivery staff—to deeply understand pain points.

Career development includes both professional and managerial tracks, regular internal tech talks, hackathons, and opportunities to work on high‑impact projects. The team emphasizes that technology must serve business value, and engineers are expected to translate model insights into concrete, quantifiable solutions for real‑world problems.