How JD’s AI Generates Multimodal Product Summaries to Boost E‑Commerce

With the rapid development of the Internet, information overload has become a barrier for people to obtain and understand needed information. This has led to the emergence of concise summary services, such as 60‑second voice briefs, quick book talks, and quick movie summaries, which distill core information for audiences.

In recent years, AI technology, especially natural language processing for text generation, has made huge progress. OpenAI’s 175‑billion‑parameter GPT‑3 released in 2020 can write at a level comparable to humans.

JD.com has applied large‑scale text generation technology in its business. Its Yanshi model, built on the domain‑pretrained K‑PLUG, can generate product copy for over 3,000 third‑level categories, achieving over 90% human‑review approval and having generated more than 30 billion characters of copy. The model is used in the “Discover Good Products” channel, combo purchases, AI live‑streaming sales, and more, contributing over 300 million CNY in GMV. Yanshi can also write poems, couplets, and calligraphy.

The main challenges of product‑summary generation come from three aspects: abundant information sources (titles, specifications, posters), the need to process multimodal and structured data (text, images, specification tables), and the requirement for an intelligent system that can fully mine product selling points and deliver personalized recommendations at the right moment.

Based on this, JD Yanshi released anonymized real‑scene data and partnered with NLPCC 2022 to host a multimodal product‑summary challenge. The task requires generating a concise text summary for a given product, using detailed textual description, a product knowledge graph, and product images. The article introduces the task definition, dataset, and evaluation methods.