LLM‑Powered Live Stream Analysis and Automation for E‑commerce

The self‑operated live‑stream team at Taobao faced a heavy workload when manually watching benchmark live rooms to extract best practices for their own streams.

To reduce manual effort, they built an end‑to‑end pipeline that leverages large language models (LLMs) to automatically learn from benchmark live streams.

Step 1 – Video & Audio Acquisition : Using ffmpeg -i "http://example.com/playlist.m3u8" -c copy output.mp4 to download the m3u8 replay video, and ffmpeg -i "http://example.com/playlist.m3u8" -vn -acodec libmp3lame output.mp3 to extract the audio.

Step 2 – Speech‑to‑Text : The audio files are fed into an internal transcription tool (听悟) to obtain time‑aligned transcripts in various formats (srt, docx, pdf).

Step 3 – LLM Summarization : Custom prompts are crafted to extract sales‑talk highlights, visual cues, and user comments. Example prompt snippets include instructions for product‑talk analysis, visual analysis, and comment classification. The LLM (GPT‑4o) generates concise summaries, actionable recommendations for product managers, and structured insights for the operations team.

The pipeline also includes prompt‑tuning guidelines (step‑by‑step decomposition, brevity, role specification) and two video‑size handling strategies: bitrate compression via ffmpeg and splitting long videos into smaller clips.

Results show that the automated summaries match the quality of manual notes, freeing operators to focus on core tasks. The system has already driven new features such as automatic coupon push, intelligent product recommendation, and standardized talk‑script templates.

Future work will focus on further prompt optimization, real‑time traffic factor integration, and expanding LLM usage to product selection copilot, interactive components, and traffic analysis.