Self-Supervised Learning for Image Embeddings in Recommendation Systems: SwAV and M6 Applications at Meiping Meiwu

This article presents the practical exploration and empirical validation of self-supervised learning techniques—specifically SwAV and M6—for generating high-quality image and multimodal embeddings in the recommendation system of Meiping Meiwu, a home decor content platform within Alibaba's Taobao ecosystem.

The authors first introduce the motivation and theoretical background of self-supervised learning, contrasting it with supervised and unsupervised paradigms, and highlight its relevance to real-world scenarios where labeled data is scarce. They detail two major approaches evaluated: SwAV (a clustering-based contrastive learning method) and M6 (a sparse Mixture-of-Experts multimodal pretraining model).

SwAV’s core idea—using swapped prediction and cluster assignment consistency across augmented views—is explained with mathematical formulations and visual comparisons to standard contrastive learning (e.g., MoCo). The article includes key equations for contrastive loss, cluster assignment projection, and uniformity constraints.

M6’s sparse MoE architecture is also described, with its gate function selecting top-k experts per forward pass to reduce computational cost: h = Σ_i w_i(x) · E_i(x), where w_i(x) = softmax(top_k(g(x)))_i and E_i denotes expert i.

The paper then evaluates embedding quality across three application scenarios:

KNN-based scene/style consistency : SwAV (+whitening) achieves 71.6% scene consistency, outperforming the online baseline (62.8%); M6 (+whitening) reaches 71.6% style consistency.

Ranking model AUC improvements : Adding M6 embeddings (+whitening) yields the highest gains: +0.00124 AUC in CTR, +0.00371 in CTCVR, and +0.00545 in dwell behavior AUC over baseline (online embedding). Notably, improvements are more pronounced for newly published content.

Downstream tasks : SwAV backbone improves image classification (Top-1: 74.67% vs. ResNet50’s 73.72%) and image retrieval (R@1: 14.23% vs. 11.56%) in the “Tantan Haohuo” scenario.

Finally, the authors summarize that self-supervised embeddings significantly enhance semantic representation, especially for cold-start content, and validate their deployment in production—achieving +2.6% pCTCVR and +1.51% avg IPv lift in A/B tests.