Short Video Analysis in Local Life Scenarios: Techniques and Practices at Meituan

Background – The rapid growth of video content, driven by advances in hardware and software, has created a massive amount of information in both user and content dimensions. AI algorithms can add value across creation, review, editing, and distribution of videos, especially in local‑life scenarios where Meituan operates.

Video Analysis Practices at Meituan

1. Multi‑label Video Classification – Traditional metadata‑only approaches are insufficient; Meituan builds a tag‑based understanding of video content to support operations, user profiling, search, recommendation, and advertising. Challenges include constructing a robust label taxonomy, ensuring accuracy and coverage, and enabling incremental learning. The initial model leverages the public YouTube‑8M dataset with aggregation‑style architectures, followed by a semi‑supervised teacher‑student pipeline to adapt to Meituan’s domain.

Label System Expansion – Horizontal expansion uses feature clustering and manual refinement, while vertical refinement adds fine‑grained food and scene tags using specialized image classification models. Continuous data iteration is achieved through active learning, confidence‑based sampling, and weak supervision from multimodal (visual‑text) signals.

2. Intelligent Video Cover – Covers act as the thumbnail for videos. Two scenarios are addressed: (a) generic covers selected by importance metrics (clarity, motion, information density) using either end‑to‑end or interpretable scoring pipelines; (b) semantic covers aligned with user intent, derived from weakly supervised segment‑level tagging that combines visual and textual cues.

3. Video Generation – Automated generation of short promotional videos for merchants (e.g., restaurants, hotels) involves AI‑driven material selection, quality assessment, deduplication, aesthetic ranking, smart cropping, and motion rendering. The pipeline processes multi‑modal inputs (images, video, audio, text) and produces a cohesive output for distribution.

Summary and Outlook – With continued advances in AI, 5G, and multimodal learning, video will play an increasingly pivotal role in local‑life services. Future work will focus on unsupervised/self‑supervised learning and deeper multimodal understanding to unlock more value from massive video data.