2024 Advances in Advertising Creative Generation and Selection
In 2024 the advertising team deployed an end‑to‑end AIGC pipeline that automatically creates high‑quality ad images, uses the multimodal Reliable Feedback Network and the million‑size RF1M dataset to filter outputs, builds rich offline and online multimodal representations with contrastive and list‑wise learning, and optimizes ranking architecture to deliver scalable, personalized creative selection.
High‑quality advertising images are crucial for e‑commerce success, but manual design is costly. The 2023 advertising team leveraged AIGC to increase creative diversity, yet low‑quality outputs limited coverage. In 2024 they achieved automatic generation of high‑quality ad creatives and personalized recommendation at scale.
Reliable Feedback Network (RFNet) : To raise the usable rate of generated images, a multimodal feedback model simulates human review. RFNet integrates auxiliary modalities (e.g., NER, background color, brand logo) to judge image validity. The architecture is illustrated below:
The team also released the RF1M dataset, containing over one million human‑annotated generated ad images to facilitate research on realistic human feedback.
Offline Representation Construction : Using MLLM technology, explicit (e.g., NER, logo) and implicit (e.g., promotion cues) features are extracted from creative images and texts. Contrastive learning (MOCO v3) treats other creatives of the same SKU as negative samples, improving discriminability. Representation quality is evaluated with the Fassi retrieval tool.
Multimodal Representation for Selection : The selection pipeline is split into element selection and combination selection. Multimodal embeddings are aligned and optimized to enhance cold‑start performance. List‑wise objectives are added to the CTR estimator, and the following formula (illustrated) captures the upgraded loss:
Online Architecture Optimization : The online system decomposes the <user, sku, creative> ranking into a <user, sku> prediction and a subsequent creative ranking. Joint training of the ranking model and the creative selection model reduces combinatorial explosion and eases serving pressure.
Summary & Outlook : The presented techniques—reliable feedback, multimodal representation, and list‑wise modeling—significantly improve AIGC image usability and personalized ad delivery. Future work will focus on deeper multimodal fusion and user‑specific personalization.
JD Retail Technology
Official platform of JD Retail Technology, delivering insightful R&D news and a deep look into the lives and work of technologists.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.