Tencent Advertising Multimedia AI Technology: Research and Application

This article, authored by Liu Wei, Director and Distinguished Scientist of Tencent's Advertising Multimedia AI Center, presents a comprehensive overview of multimedia AI technology research and applications in Tencent's advertising ecosystem. The article introduces a complete multimedia AI technology matrix built on the Taiji Machine Learning Platform, featuring the Hunyuan AI large model and advertising-specific models.

The core content covers five major technology platforms: Jue (巨阙) - Advertising Content Understanding, which provides multi-dimensional and multi-granularity semantic understanding for products, creatives, and landing pages, including multi-modal pre-training for product classification, large-scale logo retrieval, QA-style attribute recognition, and temporal video understanding; Qiankun (乾坤) - Intelligent Advertising Creation, an automated video creation engine supporting video adaptation, image-to-video generation, video-to-video derivation, and virtual effects; Shenzhen (神针) - Intelligent Ad Review, a platform with 100+ AI review capabilities covering automatic judgment, similarity reuse, negative detection, and rule engine; Tianyin (天印) - Ad Fingerprint System, providing hierarchical fingerprint IDs and embeddings for similarity detection across 4 levels; and Hunyuan (混元) - AI Large Model, featuring multi-modal content understanding, multi-modal copy generation, and cross-modal retrieval capabilities.

The technical innovations include multi-modal pre-training models using tens of millions of advertising data, DML-based logo retrieval systems, multi-modal QA frameworks for attribute extraction, video temporal segmentation and spatiotemporal detection, multi-modal auto-judgment review technology, and advanced hashing algorithms like Angular Quantization and Hash Bit Selection. The cross-modal retrieval model has achieved Top 1 results on five authoritative international cross-modal retrieval benchmarks.