{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,22]],"date-time":"2026-05-22T05:09:03Z","timestamp":1779426543189,"version":"3.53.1"},"reference-count":60,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T00:00:00Z","timestamp":1778889600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Digital"],"abstract":"<jats:p>Children increasingly consume online video content, creating a growing need for scalable approaches to support content moderation workflows. However, directly identifying harmful or policy-violating content, such as violence, sexual content, or self-harm, remains a complex task that typically requires specialized classifiers and domain-specific annotations. In this context, sentiment analysis can provide complementary information by capturing affective signals expressed through language and visual cues. This study does not treat sentiment polarity as a direct indicator of unsafe or policy-violating content. Instead, it explores multimodal sentiment analysis as an auxiliary triage signal that may help prioritize content for human review or identify segments requiring further inspection. This paper investigates the feasibility of using large vision\u2013language models (LVLMs) for zero-shot multimodal sentiment analysis on utterance-aligned video segments. We evaluate two LVLMs, LLaVA-OneVision-7B and Qwen2.5-VL-7B, under three input settings: text-only, vision-only, and multimodal, using a conversational TV-series dataset consisting of short utterance-level video segments and transcripts. The results show that multimodal sentiment inference can provide useful screening signals without task-specific fine-tuning, although the benefits are model-dependent. LLaVA-OneVision-7B consistently outperforms Qwen2.5-VL-7B and benefits more clearly from combining textual and visual inputs, whereas Qwen2.5-VL-7B shows limited improvement across modality settings. We also analyze the trade-off between frame sampling and image resolution. Finally, we discuss limitations related to dataset scope, annotation subjectivity, class imbalance, and the need for broader validation before real-world deployment.<\/jats:p>","DOI":"10.3390\/digital6020040","type":"journal-article","created":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T12:36:40Z","timestamp":1779107800000},"page":"40","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Zero-Shot Multimodal Sentiment Analysis Using LVLMs as a Triage Signal for Video Platform Moderation"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9102-7089","authenticated-orcid":false,"given":"Anggi","family":"Hanafiah","sequence":"first","affiliation":[{"name":"Department of Informatics Engineering, Universitas Islam Riau, Pekanbaru 28284, Indonesia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4637-5735","authenticated-orcid":false,"given":"Winda","family":"Monika","sequence":"additional","affiliation":[{"name":"Department of Library Information, Universitas Lancang Kuning, Pekanbaru 28266, Indonesia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6283-3217","authenticated-orcid":false,"given":"Arbi Haza","family":"Nasution","sequence":"additional","affiliation":[{"name":"Department of Informatics Engineering, Universitas Islam Riau, Pekanbaru 28284, Indonesia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9434-5880","authenticated-orcid":false,"given":"Aytu\u011f","family":"Onan","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, Faculty of Engineering, Izmir Institute of Technology, Izmir 35430, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8310-2007","authenticated-orcid":false,"given":"Yohei","family":"Murakami","sequence":"additional","affiliation":[{"name":"Faculty of Information Science and Engineering, Ritsumeikan University, Ibaraki, Osaka 567-8570, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-8913-6657","authenticated-orcid":false,"given":"Hafiza Oktasia","family":"Nasution","sequence":"additional","affiliation":[{"name":"Department of Management, Universitas Riau, Pekanbaru 28292, Indonesia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,5,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"5731","DOI":"10.1007\/s10462-022-10144-1","article-title":"A survey on sentiment analysis methods, applications, and challenges","volume":"55","author":"Wankhade","year":"2022","journal-title":"Artif. Intell. Rev."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"47","DOI":"10.54254\/2977-3903\/12\/2024128","article-title":"A comprehensive survey on multimodal sentiment analysis: Techniques, models, and applications","volume":"12","author":"Zhang","year":"2024","journal-title":"Adv. Eng. Innov."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Wu, S., Wang, X., Wang, L., He, D., and Dang, J. (2024). Enriching Multimodal Sentiment Analysis through Textual Emotional Descriptions of Visual-Audio Content. arXiv.","DOI":"10.1609\/aaai.v39i2.32152"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.imavis.2017.08.003","article-title":"A survey of multimodal sentiment analysis","volume":"65","author":"Soleymani","year":"2017","journal-title":"Image Vis. Comput."},{"key":"ref_5","first-page":"1","article-title":"Dynamic Weighted Gating for Enhanced Cross-Modal Interaction in Multimodal Sentiment Analysis","volume":"21","author":"Wang","year":"2024","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Xie, Y., Zhu, Z., Lu, X., Huang, Z., and Xiong, H. (2024, January 20\u201325). InfoEnh: Towards Multimodal Sentiment Analysis via Information Bottleneck Filter and Optimal Transport Alignment. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), Torino, Italy.","DOI":"10.63317\/2a4kxypqhzvh"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1007\/s13735-024-00340-w","article-title":"MuAL: Enhancing multimodal sentiment analysis with cross-modal attention and difference loss","volume":"13","author":"Deng","year":"2024","journal-title":"Int. J. Multimed. Inf. Retr."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wang, W., Ding, L., Shen, L., Luo, Y., Hu, H., and Tao, D. (2024). Wisdom: Improving multimodal sentiment analysis by fusing contextual world knowledge. Proceedings of the 32nd ACM International Conference on Multimedia, ACM.","DOI":"10.1145\/3664647.3681403"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"102862","DOI":"10.1016\/j.inffus.2024.102862","article-title":"Multi-source multi-modal domain adaptation","volume":"117","author":"Zhao","year":"2025","journal-title":"Inf. Fusion"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10844-024-00858-9","article-title":"Multi-task learning and mutual information maximization with crossmodal transformer for multimodal sentiment analysis","volume":"63","author":"Shi","year":"2024","journal-title":"J. Intell. Inf. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Chen, J., Yu, K., Wang, F., Zhou, Z., Bi, Y., Zhuang, S., and Zhang, D. (2024). Temporal convolutional network-enhanced real-time implicit emotion recognition with an innovative wearable fNIRS-EEG dual-modal system. Electronics, 13.","DOI":"10.3390\/electronics13071310"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"91060","DOI":"10.1109\/ACCESS.2023.3307716","article-title":"Interpretable multimodal sentiment classification using deep multi-view attentive network of image and text data","volume":"11","author":"Pashazadeh","year":"2023","journal-title":"IEEE Access"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1016\/j.inffus.2022.09.025","article-title":"Multimodal sentiment analysis: A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions","volume":"91","author":"Gandhi","year":"2023","journal-title":"Inf. Fusion"},{"key":"ref_14","unstructured":"Li, B., Zhang, Y., Guo, D., Zhang, R., Li, F., Zhang, H., Zhang, K., Zhang, P., Li, Y., and Liu, Z. (2024). Llava-onevision: Easy visual task transfer. arXiv."},{"key":"ref_15","unstructured":"Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., and Zhou, J. (2023). Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond. arXiv."},{"key":"ref_16","unstructured":"Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., and Ge, W. (2024). Qwen2-VL: Enhancing Vision-Language Model\u2019s Perception of the World at Any Resolution. arXiv."},{"key":"ref_17","unstructured":"Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., and Tang, J. (2025). Qwen2.5-VL Technical Report. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"124","DOI":"10.11591\/ijai.v12.i1.pp124-136","article-title":"A convolutional neural network framework for classifying inappropriate online video contents","volume":"12","author":"Tanantong","year":"2023","journal-title":"IAES Int. J. Artif. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Balat, M., Gabr, M., Bakr, H., and Zaky, A.B. (2024). TikGuard: A Deep Learning Transformer-Based Solution for Detecting Unsuitable TikTok Content for Kids. 2024 6th Novel Intelligent and Leading Emerging Sciences Conference (NILES), IEEE.","DOI":"10.1109\/NILES63360.2024.10753192"},{"key":"ref_20","unstructured":"Zhao, C., Yang, L., Kuang, J., and Yan, Z. (2025). Protecting Children from Violent Short Videos: A Child-Attentive Multimodal Multitask Learning Approach. Pacific Asia Conference on Information Systems, AIS Electronic Library. Available online: https:\/\/aisel.aisnet.org\/pacis2025\/aiandml\/aiandml\/19\/."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1049\/icp.2024.3981","article-title":"Research for the methods of text sentiment analysis","volume":"2024","author":"Xu","year":"2025","journal-title":"IET Conf. Proc."},{"key":"ref_22","first-page":"161","article-title":"Research on text sentiment analysis in natural language processing","volume":"Volume 2024","author":"Jiao","year":"2024","journal-title":"Proceedings of the International Conference on Electrical Engineering and Intelligent Control (EEIC 2024)"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1007\/s10723-018-9466-y","article-title":"Semantic orientation based decision making framework for big data analysis of sporadic news events","volume":"17","author":"Raza","year":"2019","journal-title":"J. Grid Comput."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1162\/coli.08-012-R1-06-90","article-title":"Recognizing contextual polarity: An exploration of features for phrase-level sentiment analysis","volume":"35","author":"Wilson","year":"2009","journal-title":"Comput. Linguist."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1162\/COLI_a_00049","article-title":"Lexicon-based methods for sentiment analysis","volume":"37","author":"Taboada","year":"2011","journal-title":"Comput. Linguist."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Pang, B., Lee, L., and Vaithyanathan, S. (2002). Thumbs up? Sentiment classification using machine learning techniques. arXiv.","DOI":"10.3115\/1118693.1118704"},{"key":"ref_27","first-page":"2996","article-title":"A novel classification approach based on Na\u00efve Bayes for Twitter sentiment analysis","volume":"11","author":"Song","year":"2017","journal-title":"KSII Trans. Internet Inf. Syst. TIIS"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Naz, S., Sharan, A., and Malik, N. (2018). Sentiment classification on twitter data using support vector machine. Proceedings of the 2018 IEEE\/WIC\/ACM International Conference on Web Intelligence (WI), IEEE.","DOI":"10.1109\/WI.2018.00-13"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Kim, Y. (2014). Convolutional neural networks for sentence classification. arXiv.","DOI":"10.3115\/v1\/D14-1181"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Tai, K.S., Socher, R., and Manning, C.D. (2015). Improved semantic representations from tree-structured long short-term memory networks. arXiv.","DOI":"10.3115\/v1\/P15-1150"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Yue, W., and Li, L. (2020). Sentiment analysis using word2vec-cnn-bilstm classification. Proceedings of the 2020 Seventh International Conference on Social Networks Analysis, Management and Security (SNAMS), IEEE.","DOI":"10.1109\/SNAMS52053.2020.9336549"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1109\/MMUL.2021.3080305","article-title":"End-to-end learning for multimodal emotion recognition in video with adaptive loss","volume":"28","author":"Huynh","year":"2021","journal-title":"IEEE MultiMedia"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Pennington, J., Socher, R., and Manning, C.D. (2014, January 25\u201329). Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1162"},{"key":"ref_34","first-page":"4171","article-title":"Bert: Pre-training of deep bidirectional transformers for language understanding","volume":"Volume 1: Long and short papers","author":"Devlin","year":"2019","journal-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Machajdik, J., and Hanbury, A. (2010). Affective image classification using features inspired by psychology and art theory. Proceedings of the 18th ACM International Conference on Multimedia, ACM.","DOI":"10.1145\/1873951.1873965"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Verma, B., Meel, P., and Vishwakarma, D.K. (2024). Visual Sentiment Recognition via Popular Deep Models on the Memotion Dataset. Proceedings of the 2024 IEEE 9th International Conference for Convergence in Technology (I2CT), IEEE.","DOI":"10.1109\/I2CT61223.2024.10543569"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Yang, J., She, D., and Sun, M. (2017). Joint Image Emotion Classification and Distribution Learning via Deep Convolutional Neural Network. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17), IJCAI Inc.","DOI":"10.24963\/ijcai.2017\/456"},{"key":"ref_38","first-page":"1","article-title":"Sentiment Recognition in Images leveraging ResNet18 vs Vit Architecture","volume":"Volume 1","author":"Sai","year":"2024","journal-title":"Proceedings of the 2024 Second International Conference on Advances in Information Technology (ICAIT)"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Jhadi, K., Tiwari, N., and Chawla, M. (2024). Visual Sentiment-based on FER for Improving Feedback Analysis using Transfer Learning. Proceedings of the 2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT), IEEE.","DOI":"10.1109\/ICCCNT61001.2024.10725403"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Limami, F., Hdioud, B., and Oulad Haj Thami, R. (2024). Contextual emotion detection in images using deep learning. Front. Artif. Intell., 7.","DOI":"10.3389\/frai.2024.1386753"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Liu, S., and Li, T. (2026). A Review of Multimodal Sentiment Analysis in Online Public Opinion Monitoring. Informatics, 13.","DOI":"10.3390\/informatics13010010"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Li, H., Lu, Y., and Zhu, H. (2024). Multi-modal sentiment analysis based on image and text fusion based on cross-attention mechanism. Electronics, 13.","DOI":"10.3390\/electronics13112069"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"474","DOI":"10.1007\/s42486-025-00195-y","article-title":"Multimodal sentiment analysis based on slice aggregation and dynamic fusion","volume":"7","author":"Zhan","year":"2025","journal-title":"CCF Trans. Pervasive Comput. Interact."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Almousa, O., Tashtoush, Y., AlSobeh, A., Zahariev, P., and Darwish, O. (2026). SiAraSent: From Features to Deep Transformers for Large-Scale Arabic Sentiment Analysis. Big Data Cogn. Comput., 10.","DOI":"10.3390\/bdcc10020049"},{"key":"ref_45","unstructured":"Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., and Anadkat, S. (2023). Gpt-4 technical report. arXiv."},{"key":"ref_46","unstructured":"Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozi\u00e8re, B., Goyal, N., Hambro, E., and Azhar, F. (2023). Llama: Open and efficient foundation language models. arXiv."},{"key":"ref_47","unstructured":"Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., and Huang, F. (2023). Qwen technical report. arXiv."},{"key":"ref_48","unstructured":"Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., and Ruan, C. (2024). Deepseek-v3 technical report. arXiv."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"71876","DOI":"10.1109\/ACCESS.2024.3402809","article-title":"ChatGPT Label: Comparing the Quality of Human-Generated and LLM-Generated Annotations in Low-Resource Language NLP Tasks","volume":"12","author":"Nasution","year":"2024","journal-title":"IEEE Access"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1007\/s40558-024-00309-9","article-title":"Discovering sentiment insights: Streamlining tourism review analysis with Large Language Models","volume":"27","author":"Guidotti","year":"2025","journal-title":"Inf. Technol. Tour."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Dmonte, A., Ko, E., and Zampieri, M. (2024). An Evaluation of Large Language Models in Financial Sentiment Analysis. Proceedings of the 2024 IEEE International Conference on Big Data (BigData), IEEE.","DOI":"10.1109\/BigData62323.2024.10825272"},{"key":"ref_52","first-page":"1","article-title":"Large Language Models Powered Aspect-Based Sentiment Analysis for Enhanced Customer Insights","volume":"21","author":"Carrasco","year":"2025","journal-title":"Tour. Manag. Stud."},{"key":"ref_53","unstructured":"Zhou, C., Song, D., Tian, Y., Wu, Z., Wang, H., Zhang, X., Yang, J., Yang, Z., and Zhang, S. (2024). A Comprehensive Evaluation of Large Language Models on Aspect-Based Sentiment Analysis. arXiv."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Khalila, Z., Nasution, A.H., Monika, W., Onan, A., Murakami, Y., Radi, Y.B.I., and Osmani, N.M. (2025). Investigating Retrieval-Augmented Generation in Quranic Studies: A Study of 13 Open-Source Large Language Models. Int. J. Adv. Comput. Sci. Appl., 16.","DOI":"10.14569\/IJACSA.2025.01602134"},{"key":"ref_55","unstructured":"Thresa Jeniffer, J., Swetha, M., Raghuvaran, E., Deepa, R., and Surendran, R. (2025). Enhancing Sentiment Analysis with Multimodal Large Language Models. 2025 6th International Conference for Emerging Technology (INCET), IEEE."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1575","DOI":"10.1109\/TNNLS.2025.3622470","article-title":"Multimodal Aspect-Based Sentiment Analysis with Plugin-Enhanced Large Language Models","volume":"37","author":"Tian","year":"2026","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3708349","article-title":"Vision-Language Models Learn Super Images for Efficient Partially Relevant Video Retrieval","volume":"21","author":"Nishimura","year":"2023","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl."},{"key":"ref_58","unstructured":"Zhu, D., Chen, J., Shen, X., Li, X., and Elhoseiny, M. (2023). Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv."},{"key":"ref_59","first-page":"973","article-title":"Utterance-level multimodal sentiment analysis","volume":"Volume 1: Long Papers","author":"Mihalcea","year":"2013","journal-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"19083","DOI":"10.1109\/ACCESS.2022.3151048","article-title":"MLCM: Multi-label confusion matrix","volume":"10","author":"Heydarian","year":"2022","journal-title":"IEEE Access"}],"container-title":["Digital"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2673-6470\/6\/2\/40\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,22]],"date-time":"2026-05-22T04:45:06Z","timestamp":1779425106000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2673-6470\/6\/2\/40"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,16]]},"references-count":60,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,6]]}},"alternative-id":["digital6020040"],"URL":"https:\/\/doi.org\/10.3390\/digital6020040","relation":{},"ISSN":["2673-6470"],"issn-type":[{"value":"2673-6470","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,16]]}}}