A Survey of Multimodal Recommendation Systems: From Background to Future Directions

Multimodal recommendation systems aim to alleviate information overload by leveraging text, images, audio, and video alongside traditional ID features, improving recommendation accuracy and addressing cold‑start problems.

The overall workflow can be divided into three stages: raw feature representation, feature interaction, and recommendation generation.

Modal Encoders extract dense representations from each modality, with visual encoders (CNN, ResNet, Transformer), textual encoders (NLP models), and other encoders for video/audio.

Feature Interaction includes three strategies: connection (building user‑item graphs to capture cross‑modal relations), fusion (attention‑based or MLP‑based merging of modality embeddings), and filtering (removing noisy or spurious cross‑modal signals via causal learning).

Feature Enhancement tackles data sparsity by learning shared and exclusive semantics across modalities, using disentangled representation learning and contrastive learning with modality masking.

Model Optimization balances lightweight recommendation models with heavyweight modal encoders, employing either end‑to‑end training or a two‑stage paradigm with pre‑trained encoders, prompt tuning, and knowledge distillation to improve efficiency.

Future directions highlighted include unified frameworks, model interpretability, computational complexity, handling incomplete or biased multimodal data, and integrating large language models (MLLMs) for richer item representations.

The article concludes with a Q&A session addressing modality missingness and the comparative benefits of multimodal versus ID‑based recommendation approaches.