How View-Specific Information Boosts Multi-View Multi-Label Learning (SIMM)

Research Motivation

Real‑world objects often have diverse descriptions and rich semantics. For example, a landscape image can be represented by HSV histograms, Gist, SIFT, etc., and annotated with tags like {snow, pavilion, lake}. Traditional multi‑label methods either concatenate heterogeneous features—causing high dimensionality and over‑fitting—or sum them, which fails when feature dimensions differ. The key challenge is to integrate diverse view‑specific and label information effectively.

Proposed Method (SIMM)

The authors introduce SIMM (View‑Specific Information extraction for Multi‑view Multi‑label learning), which simultaneously extracts shared subspace representations and view‑specific information.

Shared Subspace Mining

Inspired by adversarial multi‑task learning, SIMM minimizes an adversarial loss L_adv to confuse a discriminator D that tries to identify the originating view of a shared representation c^v. A monotonic decreasing function F is used so that D cannot distinguish views, encouraging the shared subspace to contain only common information. To avoid pure noise, a multi‑label loss L_sml is added, ensuring semantic relevance.

View‑Specific Feature Extraction

View‑specific information is defined as the residual after removing shared information. An orthogonal loss L_specific enforces orthogonality between the view‑specific vector s^v (extracted by layer E^v) and the shared vector c, encouraging them to capture complementary information.

Overall Framework

The model consists of shared subspace extractor H, view‑specific extractors E^v, and a discriminator D. During training, all components are optimized jointly. At test time, given an unseen example x^*, the final prediction is obtained by combining shared and view‑specific outputs.

Experiments

Eight multi‑view multi‑label datasets were used, including six benchmark sets and a Youku video annotation set. Six baseline methods were compared: two SIMM‑related baselines, ML‑kNN with two different inputs, and two multi‑view multi‑label methods (F2L21F, LSAMML). Six evaluation metrics were reported: Hamming Loss, Average Precision, One Error, Coverage, Micro‑F1, and another standard metric. Results (10‑fold cross‑validation) show that SIMM achieves the best performance on 87.5% of metric‑dataset combinations and ranks second on 10.4%.

Additional ablation studies set the balance parameters α and β to zero, removing L_shared and L_specific. The performance drops on Pascal and Youku15w datasets, confirming the importance of both shared and view‑specific losses.

Conclusion

SIMM jointly optimizes an adversarial loss and a multi‑label loss to capture shared information, while an orthogonal constraint extracts discriminative view‑specific features. Across eight datasets, six baselines, and six metrics, SIMM consistently outperforms traditional multi‑label and multi‑view methods, demonstrating the benefit of integrating shared and private view information.