Multimodal and Human‑Computer Interaction Technologies for E‑commerce Live Streaming: From Q&A to Live Broadcast

The presentation introduces the rapid growth of e‑commerce live streaming and the need for scalable, cost‑effective solutions, highlighting Alibaba's "Xiaomi" digital human as a case study that evolved from window‑style Q&A to multi‑dimensional live interaction.

Key challenges are identified: high talent cost for human anchors, difficulty in providing personalized support, and the necessity of handling diverse, multimodal content (text, images, video) during live sessions.

To address these, the speaker outlines a multimodal script generation pipeline that combines structured data (keywords, product attributes) with unstructured assets (text, images, videos), leveraging text‑to‑text generation, story‑telling, and knowledge‑graph‑driven outline creation to produce coherent, brand‑aligned narratives.

Advanced reading‑comprehension (MRC) and QAMaker techniques are described for extracting answers from FAQs, product documents, and visual content, enabling both answer‑to‑question and question‑to‑answer generation while reducing manual configuration effort.

The talk also presents the LiveQA framework, which integrates audio‑visual streams, ASR, entity detection, and multimodal pre‑training to support real‑time, low‑latency question answering and interactive experiences in live broadcasts.

Finally, the speaker summarizes that multimodal AI, knowledge‑graph augmentation, and efficient pre‑training have become essential for building human‑like virtual anchors that can deliver personalized, high‑quality interactions in fast‑paced live‑streaming environments.