Multimodal Attribute-Level Sentiment Analysis for Social Media: Background, Tasks, and Recent Advances

The rapid evolution of social media platforms such as Twitter and Weibo has shifted user posts from pure text to rich multimodal content, creating new challenges for sentiment analysis that must jointly model text, images, and videos.

Four core sub‑tasks are defined for multimodal attribute‑level sentiment analysis (MABSA): 1) Multimodal Attribute Extraction (MATE) – extracting attribute words from multimodal posts. 2) Multimodal Named Entity Recognition (MNER) – classifying extracted attributes into predefined entity types using visual cues. 3) Multimodal Attribute Sentiment Classification (MASC) – assigning sentiment polarity to each attribute. 4) Joint Multimodal Attribute‑Sentiment Extraction (JMASA) – extracting attribute‑sentiment pairs simultaneously.

Representative recent works include:

Unified Multimodal Transformer (UMT) (ACL 2020) – combines BERT for text and ResNet for images, introduces a multimodal interaction module and an auxiliary entity‑span detection task to improve MNER.

Coarse‑to‑Fine Image‑Target Matching (IJCAI‑ECAI 2022) – first filters out irrelevant image regions with a binary coarse‑level relevance classifier, then aligns the target entity with the most relevant image region using cross‑model transformers.

Vision‑Language Pre‑Training for MABSA (VLP‑MABSA) (ACL 2022) – a BART‑based generative framework pre‑trained on five vision‑language tasks (MLM, AOE, MRM, AOG, MSP) and fine‑tuned on MABSA datasets, achieving state‑of‑the‑art results on Twitter‑15, Twitter‑17, and MVSA‑Multi.

Extensive experiments show that these models consistently outperform prior baselines, especially in low‑resource settings where pre‑training yields large gains.

Future directions highlighted are improving model interpretability (e.g., visualizing attention), exploring adversarial robustness by swapping modalities, and extending multimodal techniques to related tasks such as information extraction, entity linking, and knowledge‑graph construction.