{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,6]],"date-time":"2026-07-06T08:17:16Z","timestamp":1783325836271,"version":"3.54.6"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T00:00:00Z","timestamp":1780012800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T00:00:00Z","timestamp":1780012800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Multimed Info Retr"],"published-print":{"date-parts":[[2026,6]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Adopting multimodal embeddings in recommender systems has become standard practice, typically leveraging text and visual embeddings derived from pre-trained multimodal foundation models. While these embeddings are widely assumed to improve recommendation performance, systematic empirical evidence remains limited. In this work, we present a comprehensive reproducibility study that critically examines the actual contribution of multimodal embeddings to recommender systems. Through large-scale experiments on 14 widely used state-of-the-art models, we evaluate the effectiveness of multimodal embeddings both jointly and per modality. Using a modality knockout strategy, an architecture-preserving diagnostic intervention, we measure model sensitivity to text and image embeddings by replacing them with constants or noise, rather than removing or altering architectural components. Our results indicate that multimodal embeddings generally improve performance, particularly in graph-based fusion models, while simpler fusion methods yield marginal gains that are difficult to attribute purely to multimodal semantics. Notably, text embeddings alone often match full multimodal performance, whereas image embeddings alone provide limited benefit. These findings clarify the practical value of multimodal embeddings and offer reproducible insights for future multimodal recommendation research. Our code and datasets are available under\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/github.com\/GAIR-Lab\/MKF4MMRec\" ext-link-type=\"uri\">https:\/\/github.com\/GAIR-Lab\/MKF4MMRec<\/jats:ext-link>\n                    .\n                  <\/jats:p>","DOI":"10.1007\/s13735-026-00408-9","type":"journal-article","created":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T11:05:52Z","timestamp":1780052752000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A Reproducibility Study of Multimodal Embeddings for Recommender Systems"],"prefix":"10.1007","volume":"15","author":[{"given":"Yu","family":"Ye","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junchen","family":"Fu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Song","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kaiwen","family":"Zheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joemon M.","family":"Jose","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,5,29]]},"reference":[{"key":"408_CR1","doi-asserted-by":"crossref","unstructured":"Wei Y, Wang X, Nie L, He X, Hong R, Chua T-S (2019) Mmgcn: Multi-modal graph convolution network for personalized recommendation of micro-video. In: Proceedings of the 27th ACM International Conference on Multimedia. pp 1437\u20131445","DOI":"10.1145\/3343031.3351034"},{"key":"408_CR2","doi-asserted-by":"crossref","unstructured":"Liu Q, Zhu J, Yang Y, Dai Q, Du Z, Wu X-M, Zhao Z, Zhang R, Dong Z (2024) Multimodal pretraining, adaptation, and generation for recommendation: A survey. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. pp 6566\u20136576","DOI":"10.1145\/3637528.3671473"},{"key":"408_CR3","doi-asserted-by":"crossref","unstructured":"Yu P, Tan Z, Lu G, Bao B-K (2023) Multi-view graph convolutional network for multimedia recommendation. In: Proceedings of the 31st ACM International Conference on Multimedia. pp 6576\u20136585","DOI":"10.1145\/3581783.3613915"},{"key":"408_CR4","doi-asserted-by":"crossref","unstructured":"Reimers N, Gurevych I (2019) Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084","DOI":"10.18653\/v1\/D19-1410"},{"key":"408_CR5","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"408_CR6","unstructured":"Dosovitskiy A (2020) An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929"},{"key":"408_CR7","doi-asserted-by":"crossref","unstructured":"Zhou X, Shen Z (2023) A tale of two graphs: Freezing and denoising graph structures for multimodal recommendation. In: Proceedings of the 31st ACM International Conference on Multimedia. pp 935\u2013943","DOI":"10.1145\/3581783.3611943"},{"key":"408_CR8","unstructured":"Zhou H, Zhou X, Zeng Z, Zhang L, Shen Z (2023) A comprehensive survey on multimodal recommender systems: Taxonomy, evaluation, and future directions. arXiv preprint arXiv:2302.04473"},{"key":"408_CR9","doi-asserted-by":"crossref","unstructured":"Fu J, Ge X, Xin X, Karatzoglou A, Arapakis I, Zheng K, Ni Y, Joemon JMJ (2025) Efficient and effective adaptation of multimodal foundation models in sequential recommendation. IEEE Trans Knowl Data Eng","DOI":"10.1109\/TKDE.2025.3608071"},{"key":"408_CR10","doi-asserted-by":"crossref","unstructured":"Ge X, Fu J, Chen F, An S, Sebe N, Jose JM (2024) Towards end-to-end explainable facial action unit recognition via vision-language joint learning. In: Proceedings of the 32nd ACM International Conference on Multimedia. pp 8189\u20138198","DOI":"10.1145\/3664647.3681443"},{"key":"408_CR11","doi-asserted-by":"crossref","unstructured":"Zhuang Z, Du H, Han H, Li Y, Fu J, Jose JM, Ni Y (2025) Bridging the gap: Teacher-assisted wasserstein knowledge distillation for efficient multi-modal recommendation. In: Proceedings of the ACM on Web Conference 2025. pp 2464\u20132475","DOI":"10.1145\/3696410.3714852"},{"key":"408_CR12","doi-asserted-by":"crossref","unstructured":"He R, McAuley J (2016) Vbpr: visual bayesian personalized ranking from implicit feedback. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30","DOI":"10.1609\/aaai.v30i1.9973"},{"key":"408_CR13","doi-asserted-by":"crossref","unstructured":"Zhou X, Zhou H, Liu Y, Zeng Z, Miao C, Wang P, You Y, Jiang F (2023) Bootstrap latent representations for multi-modal recommendation. In: Proceedings of the ACM Web Conference 2023, pp. 845\u2013854","DOI":"10.1145\/3543507.3583251"},{"key":"408_CR14","doi-asserted-by":"crossref","unstructured":"Ye Y, Fu J, Song Y, Zheng K, Jose JM (2026) Are Multimodal Embeddings Truly Beneficial for Recommendation? A Deep Dive into Whole vs. Individual Modalities. arxiv:2508.07399","DOI":"10.1007\/978-3-032-21324-2_5"},{"issue":"1","key":"408_CR15","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1007\/s11257-016-9174-x","volume":"26","author":"J Beel","year":"2016","unstructured":"Beel J, Breitinger C, Langer S, Lommatzsch A, Gipp B (2016) Towards reproducibility in recommender-systems research. User Model User-Adap Inter 26(1):69\u2013101","journal-title":"User Model User-Adap Inter"},{"key":"408_CR16","doi-asserted-by":"crossref","unstructured":"Ferrari Dacrema M, Cremonesi P, Jannach D (2019) Are we really making much progress? a worrying analysis of recent neural recommendation approaches. In: Proceedings of the 13th ACM Conference on Recommender Systems. pp 101\u2013109","DOI":"10.1145\/3298689.3347058"},{"key":"408_CR17","doi-asserted-by":"crossref","unstructured":"Rendle S, Krichene W, Zhang L, Anderson J (2020) Neural collaborative filtering vs. matrix factorization revisited. In: Proceedings of the 14th ACM Conference on Recommender Systems. pp 240\u2013248","DOI":"10.1145\/3383313.3412488"},{"key":"408_CR18","doi-asserted-by":"crossref","unstructured":"Petrov A, Macdonald C (2022) A systematic review and replicability study of bert4rec for sequential recommendation. In: Proceedings of the 16th ACM Conference on Recommender Systems. pp 436\u2013447","DOI":"10.1145\/3523227.3548487"},{"key":"408_CR19","doi-asserted-by":"crossref","unstructured":"Sun F, Liu J, Wu J, Pei C, Lin X, Ou W, Jiang P (2019) Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. pp 1441\u20131450","DOI":"10.1145\/3357384.3357895"},{"issue":"3","key":"408_CR20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3569930","volume":"41","author":"Y Ji","year":"2023","unstructured":"Ji Y, Sun A, Zhang J, Li C (2023) A critical study on data leakage in recommender system offline evaluation. ACM Trans Inf Syst 41(3):1\u201327","journal-title":"ACM Trans Inf Syst"},{"key":"408_CR21","unstructured":"Li R, Deng W, Cheng Y, Yuan Z, Zhang J, Yuan F (2023) Exploring the upper limits of text-based collaborative filtering using large language models: Discoveries and insights. arXiv preprint arXiv:2305.11700"},{"key":"408_CR22","doi-asserted-by":"crossref","unstructured":"Yuan Z, Yuan F, Song Y, Li Y, Fu J, Yang F, Pan Y, Ni Y (2023) Where to go next for recommender systems? id-vs. modality-based recommender models revisited. In: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2639\u20132649","DOI":"10.1145\/3539618.3591932"},{"key":"408_CR23","doi-asserted-by":"crossref","unstructured":"Fu J, Yuan F, Song Y, Yuan Z, Cheng M, Cheng S, Zhang J, Wang J, Pan Y (2024) Exploring adapter-based transfer learning for recommender systems: Empirical studies and practical insights. In: Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pp. 208\u2013217","DOI":"10.1145\/3616855.3635805"},{"key":"408_CR24","unstructured":"Ni Y, Cheng Y, Liu X, Fu J, Li Y, He X, Zhang Y, Yuan F (2023) A content-driven micro-video recommendation dataset at scale. arXiv preprint arXiv:2309.15379"},{"key":"408_CR25","doi-asserted-by":"crossref","unstructured":"Zhang J, Cheng Y, Ni Y, Pan Y, Yuan Z, Fu J, Li Y, Wang J, Yuan F (2024) Ninerec: A benchmark dataset suite for evaluating transferable recommendation. IEEE Transactions on Pattern Analysis and Machine Intelligence","DOI":"10.1109\/TPAMI.2024.3373868"},{"key":"408_CR26","doi-asserted-by":"crossref","unstructured":"Fu J, Ge X, Xin X, Karatzoglou A, Arapakis I, Wang J, Jose JM (2024) Iisan: Efficiently adapting multimodal representation for sequential recommendation with decoupled peft. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. pp 687\u2013697","DOI":"10.1145\/3626772.3657725"},{"key":"408_CR27","doi-asserted-by":"crossref","unstructured":"Wei Y, Wang X, Nie L, He X, Chua T-S (2020) Graph-refined convolutional network for multimedia recommendation with implicit feedback. Proceedings of the 28th ACM International Conference on Multimedia. pp 3541\u20133549","DOI":"10.1145\/3394171.3413556"},{"key":"408_CR28","doi-asserted-by":"crossref","unstructured":"Guo Z, Li J, Li G, Wang C, Shi S, Ruan B (2024) Lgmrec: Local and global graph learning for multimodal recommendation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 8454\u20138462","DOI":"10.1609\/aaai.v38i8.28688"},{"key":"408_CR29","doi-asserted-by":"crossref","unstructured":"Zhang J, Zhu Y, Liu Q, Wu S, Wang S, Wang L (2021) Mining latent structures for multimedia recommendation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3872\u20133880","DOI":"10.1145\/3474085.3475259"},{"key":"408_CR30","doi-asserted-by":"crossref","unstructured":"Xv G, Li X, Xie R, Lin C, Liu C, Xia F, Kang Z, Lin L (2024) Improving multi-modal recommender systems by denoising and aligning multi-modal content and user feedback. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3645\u20133656","DOI":"10.1145\/3637528.3671703"},{"key":"408_CR31","doi-asserted-by":"crossref","unstructured":"Ong RK, Khong AW (2025) Spectrum-based modality representation fusion graph convolutional network for multimodal recommendation. Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining. pp 773\u2013781","DOI":"10.1145\/3701551.3703561"},{"key":"408_CR32","doi-asserted-by":"crossref","unstructured":"Shi K, Zhang Y, Zhang M, Xiao K, Yu D, Zhou Y, Huang W, Li Z (2025) Heterogeneous environment-aware multimodal recommendation with modality alignment. Information Fusion","DOI":"10.1016\/j.inffus.2025.103989"},{"key":"408_CR33","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2025.113766","volume":"324","author":"K Shi","year":"2025","unstructured":"Shi K, Zhang Y, Zhang M, Xiao K, Hou X, Li Z (2025) Noise-enhanced graph contrastive learning for multimodal recommendation systems. Knowl-Based Syst 324:113766","journal-title":"Knowl-Based Syst"},{"key":"408_CR34","doi-asserted-by":"crossref","unstructured":"Shi K, Zhang Y, Zhang M, Chen L, Yi J, Xiao K, Hou X, Li Z (2025) Structurally refined graph transformer for multimodal recommendation. IEEE Transactions on Multimedia. arXiv:2511.00584","DOI":"10.1109\/TMM.2026.3673527"},{"key":"408_CR35","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2025.114807","volume":"331","author":"Q He","year":"2026","unstructured":"He Q, Li K, Zhang Y, Li X, Zhou W (2026) Smooth diffusion model for multimodal recommendation. Knowl-Based Syst 331:114807","journal-title":"Knowl-Based Syst"},{"key":"408_CR36","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2025.130763","volume":"304","author":"J Liu","year":"2025","unstructured":"Liu J, Zhang Y, He Q, Yang R, Wang X, Zhou W (2025) Prototype learning based hierarchical decoupling for multimodal recommendation. Expert Syst Appl 304:130763","journal-title":"Expert Syst Appl"},{"key":"408_CR37","doi-asserted-by":"crossref","unstructured":"Pomo C, Attimonelli M, Danese D, Narducci F, Noia TD (2025) Do recommender systems really leverage multimodal content? a comprehensive analysis on multimodal representations for recommendation. arXiv:2508.04571 [cs.IR]","DOI":"10.1145\/3746252.3761398"},{"key":"408_CR38","unstructured":"Zhou H, Zhang Y, Sun A, Shen Z (2025) Does multimodality improve recommender systems as expected? a critical analysis and future directions. arXiv:2508.05377 [cs.IR]"},{"key":"408_CR39","doi-asserted-by":"crossref","unstructured":"He K, Chen X, Xie S, Li Y, Doll\u00e1r P, Girshick R (2022) Masked autoencoders are scalable vision learners. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. pp 16000\u201316009","DOI":"10.1109\/CVPR52688.2022.01553"},{"key":"408_CR40","doi-asserted-by":"crossref","unstructured":"Ni J, Li J, McAuley J (2019) Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 188\u2013197","DOI":"10.18653\/v1\/D19-1018"},{"key":"408_CR41","unstructured":"Reimers N, Gurevych I (2020) all-MiniLM-L6-v2: A Sentence-BERT model. https:\/\/huggingface.co\/sentence-transformers\/all-MiniLM-L6-v2. Accessed: 2025-09-05"},{"key":"408_CR42","doi-asserted-by":"publisher","first-page":"1074","DOI":"10.1109\/TMM.2021.3138298","volume":"25","author":"Q Wang","year":"2021","unstructured":"Wang Q, Wei Y, Yin J, Wu J, Song X, Nie L (2021) Dualgnn: Dual graph neural network for multimedia recommendation. IEEE Trans Multimedia 25:1074\u20131084","journal-title":"IEEE Trans Multimedia"},{"key":"408_CR43","doi-asserted-by":"publisher","first-page":"5107","DOI":"10.1109\/TMM.2022.3187556","volume":"25","author":"Z Tao","year":"2022","unstructured":"Tao Z, Liu X, Xia Y, Wang X, Yang L, Huang X, Chua T-S (2022) Self-supervised learning for multimedia recommendation. IEEE Trans Multimedia 25:5107\u20135116","journal-title":"IEEE Trans Multimedia"},{"key":"408_CR44","doi-asserted-by":"crossref","unstructured":"Zhou H, Zhou X, Zhang L, Shen Z (2023) Enhancing dyadic relations with homogeneous graphs for multimodal recommendation. In: ECAI 2023, pp. 3123\u20133130. IOS Press, ???","DOI":"10.3233\/FAIA230631"},{"key":"408_CR45","doi-asserted-by":"crossref","unstructured":"Yu P, Tan Z, Lu G, Bao B-K (2025) Mind individual information! principal graph learning for multimedia recommendation. Proceedings of the AAAI Conference on Artificial Intelligence, vol 39. pp 13096\u201313105","DOI":"10.1609\/aaai.v39i12.33429"},{"key":"408_CR46","doi-asserted-by":"crossref","unstructured":"He X, Liao L, Zhang H, Nie L, Hu X, Chua T-S (2017) Neural collaborative filtering. In: Proceedings of the 26th International Conference on World Wide Web. pp. 173\u2013182","DOI":"10.1145\/3038912.3052569"},{"issue":"1","key":"408_CR47","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3682075","volume":"3","author":"L Zhang","year":"2024","unstructured":"Zhang L, Zhou X, Zeng Z, Shen Z (2024) Multimodal pre-training for sequential recommendation via contrastive learning. ACM Trans Recomm Syst 3(1):1\u201323","journal-title":"ACM Trans Recomm Syst"}],"container-title":["International Journal of Multimedia Information Retrieval"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13735-026-00408-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s13735-026-00408-9","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13735-026-00408-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,6]],"date-time":"2026-07-06T07:42:58Z","timestamp":1783323778000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s13735-026-00408-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,29]]},"references-count":47,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,6]]}},"alternative-id":["408"],"URL":"https:\/\/doi.org\/10.1007\/s13735-026-00408-9","relation":{},"ISSN":["2192-6611","2192-662X"],"issn-type":[{"value":"2192-6611","type":"print"},{"value":"2192-662X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,29]]},"assertion":[{"value":"3 February 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 May 2026","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 May 2026","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 May 2026","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"14"}}