{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T04:43:42Z","timestamp":1784263422984,"version":"3.55.0"},"reference-count":47,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2022,5,10]],"date-time":"2022-05-10T00:00:00Z","timestamp":1652140800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The increasing popularity of social networks and users\u2019 tendency towards sharing their feelings, expressions, and opinions in text, visual, and audio content have opened new opportunities and challenges in sentiment analysis. While sentiment analysis of text streams has been widely explored in the literature, sentiment analysis from images and videos is relatively new. This article focuses on visual sentiment analysis in a societally important domain, namely disaster analysis in social media. To this aim, we propose a deep visual sentiment analyzer for disaster-related images, covering different aspects of visual sentiment analysis starting from data collection, annotation, model selection, implementation, and evaluations. For data annotation and analyzing people\u2019s sentiments towards natural disasters and associated images in social media, a crowd-sourcing study has been conducted with a large number of participants worldwide. The crowd-sourcing study resulted in a large-scale benchmark dataset with four different sets of annotations, each aiming at a separate task. The presented analysis and the associated dataset, which is made public, will provide a baseline\/benchmark for future research in the domain. We believe the proposed system can contribute toward more livable communities by helping different stakeholders, such as news broadcasters, humanitarian organizations, as well as the general public.<\/jats:p>","DOI":"10.3390\/s22103628","type":"journal-article","created":{"date-parts":[[2022,5,10]],"date-time":"2022-05-10T21:52:11Z","timestamp":1652219531000},"page":"3628","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":49,"title":["Visual Sentiment Analysis from Disaster Images in Social Media"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3830-9869","authenticated-orcid":false,"given":"Syed Zohaib","family":"Hassan","sequence":"first","affiliation":[{"name":"SimulaMet, 0167 Oslo, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0931-9275","authenticated-orcid":false,"given":"Kashif","family":"Ahmad","sequence":"additional","affiliation":[{"name":"Information and Computing Technology Division, College of Science and Engineering, Hamad Bin Khalifa University (HBKU), Doha 34110, Qatar"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steven","family":"Hicks","sequence":"additional","affiliation":[{"name":"SimulaMet, 0167 Oslo, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2073-7029","authenticated-orcid":false,"given":"P\u00e5l","family":"Halvorsen","sequence":"additional","affiliation":[{"name":"SimulaMet, 0167 Oslo, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ala","family":"Al-Fuqaha","sequence":"additional","affiliation":[{"name":"Information and Computing Technology Division, College of Science and Engineering, Hamad Bin Khalifa University (HBKU), Doha 34110, Qatar"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7858-0928","authenticated-orcid":false,"given":"Nicola","family":"Conci","sequence":"additional","affiliation":[{"name":"Department of Information Engineering and Computer Science, University of Trento, 38123 Trento, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Riegler","sequence":"additional","affiliation":[{"name":"SimulaMet, 0167 Oslo, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1016\/j.tele.2017.10.006","article-title":"Sentiment analysis on Twitter: A text mining approach to the Syrian refugee crisis","volume":"35","author":"Ayvaz","year":"2018","journal-title":"Telemat. Inform."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/j.osnem.2017.12.002","article-title":"Politics, sentiments, and misinformation: An analysis of the Twitter discussion on the 2016 Austrian presidential elections","volume":"5","author":"Strembeck","year":"2018","journal-title":"Online Soc. Netw. Media"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2745","DOI":"10.1007\/s11063-019-10049-1","article-title":"A Robust Sentiment Analysis Method Based on Sequential Combination of Convolutional and Recursive Neural Networks","volume":"50","author":"Sadr","year":"2019","journal-title":"Neural Process. Lett."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1177\/1529100619832930","article-title":"Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements","volume":"20","author":"Barrett","year":"2019","journal-title":"Psychol. Sci. Public Interest"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1109\/MIS.2018.2882362","article-title":"Multimodal Sentiment Analysis: Addressing Key Issues and Setting Up the Baselines","volume":"33","author":"Poria","year":"2018","journal-title":"IEEE Intell. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"31267","DOI":"10.1007\/s11042-019-07942-1","article-title":"Natural disasters detection in social media and satellite imagery: A survey","volume":"78","author":"Said","year":"2019","journal-title":"Multimed. Tools Appl."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"102261","DOI":"10.1016\/j.ipm.2020.102261","article-title":"Using AI and Social Media Multimodal Content for Disaster Response and Management: Opportunities, Challenges, and Future Directions","volume":"57","author":"Imran","year":"2020","journal-title":"Inf. Process. Manag."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2837","DOI":"10.1007\/s11042-018-5982-9","article-title":"Social media and satellites","volume":"78","author":"Ahmad","year":"2018","journal-title":"Multimed. Tools Appl."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Hassan, S.Z., Ahmad, K., Al-Fuqaha, A., and Conci, N. (2019, January 9\u201313). Sentiment analysis from images of natural disasters. Proceedings of the International Conference on Image Analysis and Processing, Trento, Italy.","DOI":"10.1007\/978-3-030-30645-8_10"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1016\/j.chb.2015.09.011","article-title":"Follow me and like my beautiful selfies: Singapore teenage girls\u2019 engagement in self-presentation and peer comparison on social media","volume":"55","author":"Chua","year":"2016","journal-title":"Comput. Hum. Behav."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1109\/TAFFC.2014.2317187","article-title":"Are they different? Affect, feeling, emotion, sentiment, and opinion detection in text","volume":"5","author":"Munezero","year":"2014","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2980","DOI":"10.1109\/TMM.2018.2827782","article-title":"Building emotional machines: Recognizing image emotions through deep neural networks","volume":"20","author":"Kim","year":"2018","journal-title":"IEEE Trans. Multimed."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.imavis.2017.08.003","article-title":"A survey of multimodal sentiment analysis","volume":"65","author":"Soleymani","year":"2017","journal-title":"Image Vis. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"58683","DOI":"10.1109\/ACCESS.2020.2982970","article-title":"Face Segmentation: A Journey From Classical to Deep Learning Paradigm, Approaches, Trends, and Directions","volume":"8","author":"Khan","year":"2020","journal-title":"IEEE Access"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Badjatiya, P., Gupta, S., Gupta, M., and Varma, V. (2017, January 3\u20137). Deep Learning for Hate Speech Detection in Tweets. Proceedings of the 26th International Conference on World Wide Web Companion, Perth, Australia.","DOI":"10.1145\/3041021.3054223"},{"key":"ref_16","unstructured":"Araque, O., Gatti, L., Staiano, J., and Guerini, M. (2019). DepecheMood++: A Bilingual Emotion Lexicon Built through Simple Yet Powerful Techniques. IEEE Trans. Affect. Comput."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"e1253","DOI":"10.1002\/widm.1253","article-title":"Deep learning for sentiment analysis: A survey","volume":"8","author":"Zhang","year":"2018","journal-title":"Wiley Interdiscip. Rev. Data Min. Knowl. Discov."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1440","DOI":"10.1049\/iet-ipr.2019.1270","article-title":"Survey on Visual Sentiment Analysis","volume":"14","author":"Ortis","year":"2020","journal-title":"IET Image Process."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Machajdik, J., and Hanbury, A. (2010, January 25\u201329). Affective image classification using features inspired by psychology and art theory. Proceedings of the 18th ACM International Conference on Multimedia, Firenze, Italy.","DOI":"10.1145\/1873951.1873965"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Borth, D., Ji, R., Chen, T., Breuel, T., and Chang, S.F. (2013, January 21\u201325). Large-Scale Visual Sentiment Ontology and Detectors Using Adjective Noun Pairs. Proceedings of the 21st ACM International Conference on Multimedia, Barcelona, Spain.","DOI":"10.1145\/2502081.2502282"},{"key":"ref_21","unstructured":"Chen, T., Borth, D., Darrell, T., and Chang, S.F. (2014). Deepsentibank: Visual sentiment concept classification with deep convolutional neural networks. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Chandrasekaran, G., and Hemanth, D.J. (2021, January 22\u201324). Efficient Visual Sentiment Prediction Approaches Using Deep Learning Models. Proceedings of the Iberoamerican Knowledge Graphs and Semantic Web Conference, Kingsville, TX, USA.","DOI":"10.1007\/978-3-030-91305-2_20"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Pournaras, A., Gkalelis, N., Galanopoulos, D., and Mezaris, V. (2021, January 4\u20135). Exploiting Out-of-Domain Datasets and Visual Representations for Image Sentiment Classification. Proceedings of the 2021 16th International Workshop on Semantic and Social Media Adaptation & Personalization (SMAP), Corfu, Greece.","DOI":"10.1109\/SMAP53521.2021.9610801"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Al-Halah, Z., Aitken, A.P., Shi, W., and Caballero, J. (2019, January 27\u201328). Smile, Be Happy :) Emoji Embedding for Visual Sentiment Analysis. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea.","DOI":"10.1109\/ICCVW.2019.00550"},{"key":"ref_26","first-page":"1","article-title":"Attention-Based Modality-Gated Networks for Image-Text Sentiment Analysis","volume":"16","author":"Huang","year":"2020","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Gelli, F., Uricchio, T., He, X., Del Bimbo, A., and Chua, T.S. (2019, January 21\u201325). Learning subjective attributes of images from auxiliary sources. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.","DOI":"10.1145\/3343031.3350574"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"You, Q., Jin, H., and Luo, J. (2017, January 4\u20139). Visual sentiment analysis by attending on local image regions. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.10501"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ou, H., Qing, C., Xu, X., and Jin, J. (2021). Multi-Level Context Pyramid Network for Visual Sentiment Analysis. Sensors, 21.","DOI":"10.3390\/s21062136"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Wu, L., Zhang, H., Deng, S., Shi, G., and Liu, X. (2021). Discovering Sentimental Interaction via Graph Convolutional Network for Visual Sentiment Prediction. Appl. Sci., 11.","DOI":"10.3390\/app11041404"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1007\/s00530-020-00656-7","article-title":"A deep learning architecture of RA-DLNet for visual sentiment analysis","volume":"26","author":"Yadav","year":"2020","journal-title":"Multimed. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., Wang, X., and Tang, X. (2017, January 21\u201326). Residual attention network for image classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.683"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Wang, X., Jia, J., Yin, J., and Cai, L. (2013, January 15\u201318). Interpretable aesthetic features for affective image classification. Proceedings of the 2013 IEEE International Conference on Image Processing, Melbourne, VIC, Australia.","DOI":"10.1109\/ICIP.2013.6738665"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"22323","DOI":"10.1007\/s11042-019-08312-7","article-title":"Exploiting objective text description of images for visual sentiment analysis","volume":"80","author":"Ortis","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Katsurai, M., and Satoh, S. (2016, January 20\u201325). Image sentiment analysis using latent correlations among visual, textual, and sentiment views. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.","DOI":"10.1109\/ICASSP.2016.7472195"},{"key":"ref_36","unstructured":"Wang, J., Fu, J., Xu, Y., and Mei, T. (2016, January 9\u201315). Beyond Object Recognition: Visual Sentiment Analysis with Deep Coupled Adjective and Noun Neural Networks. Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI), New York, NY, USA."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Peng, K.C., Sadovnik, A., Gallagher, A., and Chen, T. (2016, January 25\u201328). Where do emotions come from? Predicting the emotion stimuli map. Proceedings of the 2016 IEEE international conference on image processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7532430"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"E7900","DOI":"10.1073\/pnas.1702247114","article-title":"Self-report captures 27 distinct categories of emotion bridged by continuous gradients","volume":"114","author":"Cowen","year":"2017","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_39","unstructured":"Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., and Oliva, A. (2014, January 8\u201313). Learning deep features for scene recognition using places database. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_40","first-page":"39","article-title":"How Deep Features Have Improved Event Recognition in Multimedia: A Survey","volume":"15","author":"Ahmad","year":"2019","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl. (TOMM)"},{"key":"ref_41","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20138). Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_42","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_45","unstructured":"Iandola, F., Moskewicz, M., Karayev, S., Girshick, R., Darrell, T., and Keutzer, K. (2014). Densenet: Implementing efficient convnet descriptor pyramids. arXiv."},{"key":"ref_46","unstructured":"Tan, M., and Le, Q.V. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv."},{"key":"ref_47","first-page":"1","article-title":"Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning","volume":"18","author":"Nogueira","year":"2017","journal-title":"J. Mach. Learn. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/10\/3628\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:08:45Z","timestamp":1760137725000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/10\/3628"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,10]]},"references-count":47,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["s22103628"],"URL":"https:\/\/doi.org\/10.3390\/s22103628","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,10]]}}}