{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T02:19:23Z","timestamp":1772504363762,"version":"3.50.1"},"reference-count":42,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2025,6,20]],"date-time":"2025-06-20T00:00:00Z","timestamp":1750377600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["72071010"],"award-info":[{"award-number":["72071010"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>When a disaster occurs, a large number of social media posts on platforms like Weibo attract public attention with their combination of text and images. However, the consistency between textual descriptions and visual representations varies significantly. Consistent multi-modal data are crucial for helping the public understand the disaster situation and support rescue efforts. This study aims to develop a systematic framework for assessing the consistency of multi-modal disaster-related data on social media. This study explored how the congruence between text and image content affects public engagement and informs strategies for efficient emergency responses. Firstly, the Clip (Contrastive Language-Image Pre-Training) model was used to mine the disaster correlation, loss category, and severity of the images and text. Then, the consistency of image\u2013text pairs was qualitatively analyzed and quantitatively calculated. Finally, the influence of graphic consistency on social concern was discussed. The experimental findings reveal that the consistency of text and image data significantly influences the degree of public concern. When the consistency increases by 1%, the social attention index will increase by about 0.8%. This shows that consistency is a key factor for attracting public attention and promoting the dissemination of information related to important disasters. The proposed framework offers a robust, systematic approach to analyzing disaster loss information consistency. It allows for the efficient extraction of high-consistency data from vast social media data sets, providing governments and emergency response agencies with timely, accurate insights into disaster situations.<\/jats:p>","DOI":"10.3390\/systems13070498","type":"journal-article","created":{"date-parts":[[2025,6,20]],"date-time":"2025-06-20T13:13:04Z","timestamp":1750425184000},"page":"498","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Harnessing Multi-Modal Synergy: A Systematic Framework for Disaster Loss Consistency Analysis and Emergency Response"],"prefix":"10.3390","volume":"13","author":[{"given":"Siqing","family":"Shan","sequence":"first","affiliation":[{"name":"School of Economics and Management, Beihang University, Beijing 100191, China"}]},{"given":"Jingyu","family":"Su","sequence":"additional","affiliation":[{"name":"School of Economics and Management, Beihang University, Beijing 100191, China"}]},{"given":"Junze","family":"Li","sequence":"additional","affiliation":[{"name":"School of Economics and Management, Beihang University, Beijing 100191, China"}]},{"given":"Yinong","family":"Li","sequence":"additional","affiliation":[{"name":"School of Economics and Management, Beihang University, Beijing 100191, China"}]},{"given":"Zhongbao","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Economics and Management, Beihang University, Beijing 100191, China"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"103193","DOI":"10.1016\/j.ipm.2022.103193","article-title":"Joint multimodal sentiment analysis based on information relevance","volume":"60","author":"Chen","year":"2023","journal-title":"Inf. Process. Manag."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"102095","DOI":"10.1016\/j.ijdrr.2021.102095","article-title":"Social media-based disaster research: Development, trends, and obstacles","volume":"55","author":"Tang","year":"2021","journal-title":"Int. J. Disaster Risk Reduct."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Wu, Z., Chen, L., and Song, Y. (2023, January 19\u201321). A Model for Classifying Emergency Events Based on Social Media Multimodal Data. Proceedings of the International Work-Conference on Artificial Neural Networks, Ponta Delgada, Portugal.","DOI":"10.1007\/978-3-031-43085-5_25"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1017\/dmp.2020.226","article-title":"Lack of alignment in emergency response by systems and the public: A Dutch disaster health literacy case study","volume":"16","year":"2022","journal-title":"Disaster Med. Public Health Prep."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s10708-007-9111-y","article-title":"Citizens as sensors: The world of volunteered geography","volume":"69","author":"Goodchild","year":"2007","journal-title":"GeoJournal"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Min, S., Ahuja, R., Liu, Y., Zaidi, A., Phu, C., Nocera, L., and Shahabi, C. (2021, January 2\u20135). CrowdMap: Spatiotemporal Visualization of Anonymous Occupancy Data for Pandemic Response. Proceedings of the 29th International Conference on Advances in Geographic Information Systems, Beijing, China.","DOI":"10.1145\/3474717.3484269"},{"key":"ref_7","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18\u201324). Learning transferable visual models from natural language supervision. Proceedings of the International Conference on Machine Learning, Online."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"107733","DOI":"10.1016\/j.chb.2023.107733","article-title":"Exploring the impact of sentiment on multi-dimensional information dissemination using COVID-19 data in China","volume":"144","author":"Luo","year":"2023","journal-title":"Comput. Hum. Behav."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1016\/j.ssci.2019.02.029","article-title":"Disaster management 2.0: A real-time disaster damage assessment model based on mobile social media data\u2014A case study of Weibo (Chinese Twitter)","volume":"115","author":"Shan","year":"2019","journal-title":"Saf. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"101760","DOI":"10.1016\/j.ijdrr.2020.101760","article-title":"Leveraging multimodal social media data for rapid disaster damage assessment","volume":"51","author":"Hao","year":"2020","journal-title":"Int. J. Disaster Risk Reduct."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1087","DOI":"10.1080\/19475705.2022.2064774","article-title":"Mining typhoon victim information based on multi-source data fusion using social media data in China: A case study of the 2019 Super Typhoon Lekima. Geomatics","volume":"13","author":"Wu","year":"2022","journal-title":"Nat. Hazards Risk"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"102418","DOI":"10.1016\/j.ijdrr.2021.102418","article-title":"Real-time assessment of human loss in disasters based on social media mining and the truth discovery algorithm","volume":"62","author":"Shan","year":"2021","journal-title":"Int. J. Disaster Risk Reduct."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"102200","DOI":"10.1016\/j.ijdrr.2021.102200","article-title":"Crowdsourced social media and mobile phone signaling data for disaster impact assessment: A case study of the 8.8 Jiuzhaigou earthquake","volume":"58","author":"Xing","year":"2021","journal-title":"Int. J. Disaster Risk Reduct."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"108401","DOI":"10.1016\/j.compeleceng.2022.108401","article-title":"Multi-label disaster text classification via supervised contrastive learning for social media data","volume":"104","author":"Xie","year":"2022","journal-title":"Comput. Electr. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"106136","DOI":"10.1016\/j.engappai.2023.106136","article-title":"An ALBERT-based TextCNN-Hatt hybrid model enhanced with topic knowledge for sentiment analysis of sudden-onset disasters","volume":"123","author":"Zhang","year":"2023","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"112891","DOI":"10.1016\/j.measurement.2023.112891","article-title":"Automatic detection of actual water depth of urban floods from social media images","volume":"216","author":"Li","year":"2023","journal-title":"Measurement"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"110467","DOI":"10.1016\/j.knosys.2023.110467","article-title":"Scanning, attention, and reasoning multimodal content for sentiment analysis","volume":"268","author":"Liu","year":"2023","journal-title":"Knowl. -Based Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"105405","DOI":"10.1016\/j.cageo.2023.105405","article-title":"Real-time social media sentiment analysis for rapid impact assessment of floods","volume":"178","author":"Godsall","year":"2023","journal-title":"Comput. Geosci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"107164","DOI":"10.1016\/j.engappai.2023.107164","article-title":"Image-based preliminary emergency assessment of damaged buildings after earthquake: Taiwan case studies","volume":"126","author":"Cheng","year":"2023","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.isprsjprs.2022.11.010","article-title":"Rapid identification of damaged buildings using incremental learning with transferred data from historical natural disaster cases","volume":"195","author":"Ge","year":"2023","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"116132","DOI":"10.1016\/j.engstruct.2023.116132","article-title":"Geometric consistency enhanced deep convolutional encoder-decoder for urban seismic damage assessment by UAV images","volume":"286","author":"Wang","year":"2023","journal-title":"Eng. Struct."},{"key":"ref_22","first-page":"103483","article-title":"InCR: Inception and concatenation residual block-based deep learning network for damaged building detection using remote sensing images","volume":"123","author":"Tasci","year":"2023","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"100225","DOI":"10.1016\/j.dajour.2023.100225","article-title":"An integrated convolutional neural network and sorting algorithm for image classification for efficient flood disaster management","volume":"7","author":"Islam","year":"2023","journal-title":"Decis. Anal. J."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"104467","DOI":"10.1016\/j.scs.2023.104467","article-title":"Flood vulnerability assessment of urban buildings based on integrating high-resolution remote sensing and street view images","volume":"92","author":"Xing","year":"2023","journal-title":"Sustain. Cities Soc."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"110502","DOI":"10.1016\/j.knosys.2023.110502","article-title":"TeFNA: Text-centered fusion network with crossmodal attention for multimodal sentiment analysis","volume":"269","author":"Huang","year":"2023","journal-title":"Knowl. Based Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"110515","DOI":"10.1016\/j.knosys.2023.110515","article-title":"VABDC-Net: A framework for Visual-Caption Sentiment Recognition via spatio-depth visual attention and bi-directional caption processing","volume":"269","author":"Pandey","year":"2023","journal-title":"Knowl. -Based Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"103378","DOI":"10.1016\/j.ipm.2023.103378","article-title":"Multimodal negative sentiment recognition of online public opinion on public health emergencies based on graph convolutional networks and ensemble learning","volume":"60","author":"Zeng","year":"2023","journal-title":"Inf. Process. Manag."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"103229","DOI":"10.1016\/j.ipm.2022.103229","article-title":"PS-mixer: A polar-vector and strength-vector mixer model for multimodal sentiment analysis","volume":"60","author":"Lin","year":"2023","journal-title":"Inf. Process. Manag."},{"key":"ref_29","unstructured":"Wang, Z., Liu, X., Li, H., Sheng, L., Yan, J., Wang, X., and Shao, J. (November, January 27). Camp: Cross-modal adaptive message passing for text-image retrieval. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Liu, C., Mao, Z., Liu, A.-A., Zhang, T., Wang, B., and Zhang, Y. (2019, January 21\u201325). Focus your attention: A bidirectional focal attention network for image-text matching. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.","DOI":"10.1145\/3343031.3350869"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Diao, H., Zhang, Y., Ma, L., and Lu, H. (2021, January 2\u20139). Similarity reasoning and filtration for image-text matching. Proceedings of the AAAI Conference on Artificial Intelligence, Online.","DOI":"10.1609\/aaai.v35i2.16209"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Rizk, Y., Jomaa, H.S., Awad, M., and Castillo, C. (2019, January 8\u201312). A computationally efficient multi-modal classification approach of disaster-related Twitter images. Proceedings of the 34th ACM\/SIGAPP Symposium on Applied Computing, Limassol, Cyprus.","DOI":"10.1145\/3297280.3297481"},{"key":"ref_33","unstructured":"Mouzannar, H., Rizk, Y., and Awad, M. (2018, January 4\u20137). Damage Identification in Social Media Posts using Multimodal Deep Learning. Proceedings of the ISCRAM, Rochester, NY, USA."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10479-020-03514-x","article-title":"A deep multi-modal neural network for informative Twitter content classification during emergencies","volume":"319","author":"Kumar","year":"2022","journal-title":"Ann. Oper. Res."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"10223","DOI":"10.1007\/s12652-020-02791-5","article-title":"Multi-modal classification of Twitter data during disasters for humanitarian response","volume":"12","author":"Madichetty","year":"2021","journal-title":"J. Ambient Intell. Humaniz. Comput."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1177\/00222437231169711","article-title":"Words meet photos: When and why photos increase review helpfulness","volume":"61","author":"Ceylan","year":"2024","journal-title":"J. Mark. Res."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1459","DOI":"10.25300\/MISQ\/2020\/14870","article-title":"Enhancing social media analysis with visual data analytics: A deep learning approach","volume":"44","author":"Shin","year":"2020","journal-title":"MIS Q. Manag. Inf. Syst."},{"key":"ref_38","unstructured":"Ofli, F., Alam, F., and Imran, M. (2020). Analysis of social media data using multimodal deep learning for disaster response. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Alam, F., Ofli, F., and Imran, M. (2018, January 25\u201328). Crisismmd: Multimodal twitter datasets from natural disasters. Proceedings of the International AAAI Conference on Web and Social Media, Palo Alto, CA, USA.","DOI":"10.1609\/icwsm.v12i1.14983"},{"key":"ref_40","unstructured":"Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Lu, Y., Liu, Q., Dai, D., Xiao, X., Lin, H., Han, X., Sun, L., and Wu, H. (2022). Unified structure generation for universal information extraction. arXiv.","DOI":"10.18653\/v1\/2022.acl-long.395"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1057\/s41599-022-01278-2","article-title":"Influence of information attributes on information dissemination in public health emergencies","volume":"9","author":"Cai","year":"2022","journal-title":"Humanit. Soc. Sci. Commun."}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/13\/7\/498\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:56:05Z","timestamp":1760032565000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/13\/7\/498"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,20]]},"references-count":42,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2025,7]]}},"alternative-id":["systems13070498"],"URL":"https:\/\/doi.org\/10.3390\/systems13070498","relation":{},"ISSN":["2079-8954"],"issn-type":[{"value":"2079-8954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,20]]}}}