{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:18:44Z","timestamp":1760059124185,"version":"build-2065373602"},"reference-count":57,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2025,5,24]],"date-time":"2025-05-24T00:00:00Z","timestamp":1748044800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>During natural disasters, social media platforms, such as X (formerly Twitter), become a valuable source of real-time information, with eyewitnesses and affected individuals posting messages about the produced damage and the victims. Although this information can be used to streamline the intervention process of local authorities and to achieve a better distribution of available resources, manually annotating these messages is often infeasible due to time and cost constraints. To address this challenge, we explore the use of semi-supervised learning, a technique that leverages both labeled and unlabeled data, to enhance neural models for disaster tweet classification. Specifically, we investigate state-of-the-art semi-supervised learning models and focus on co-training, a less-explored approach in recent years. Moreover, we propose a novel hybrid co-training architecture, Multihead Average Pseudo-Margin, which obtains state-of-the-art results on several classification tasks. Our approach extends the advantages of the voting mechanism from Multihead Co-Training by using the Average Pseudo-Margin (APM) score to improve the quality of the pseudo-labels and self-adaptive confidence thresholds for improving imbalanced classification. Our method achieves up to 7.98% accuracy improvement in low-data scenarios and 2.84% improvement when using the entire labeled dataset, reaching 89.55% accuracy on the Humanitarian task and 91.23% on the Informative task. These results demonstrate the potential of our approach in addressing the critical need for automated disaster tweet classification. We made our code publicly available for future research.<\/jats:p>","DOI":"10.3390\/info16060434","type":"journal-article","created":{"date-parts":[[2025,5,25]],"date-time":"2025-05-25T20:26:50Z","timestamp":1748204810000},"page":"434","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Multihead Average Pseudo-Margin Learning for Disaster Tweet Classification"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-4889-8379","authenticated-orcid":false,"given":"Iustin","family":"S\u00eerbu","sequence":"first","affiliation":[{"name":"Faculty of Automatic Control and Computer Science, National University of Science and Technology POLITEHNICA Bucharest, 060042 Bucharest, Romania"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robert-Adrian","family":"Popovici","sequence":"additional","affiliation":[{"name":"Faculty of Automatic Control and Computer Science, National University of Science and Technology POLITEHNICA Bucharest, 060042 Bucharest, Romania"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7255-5537","authenticated-orcid":false,"given":"Traian","family":"Rebedea","sequence":"additional","affiliation":[{"name":"Faculty of Automatic Control and Computer Science, National University of Science and Technology POLITEHNICA Bucharest, 060042 Bucharest, Romania"},{"name":"NVIDIA, Santa Clara, CA 95051, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8082-8497","authenticated-orcid":false,"given":"\u0218tefan","family":"Tr\u0103u\u0219an-Matu","sequence":"additional","affiliation":[{"name":"Faculty of Automatic Control and Computer Science, National University of Science and Technology POLITEHNICA Bucharest, 060042 Bucharest, Romania"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,5,24]]},"reference":[{"key":"ref_1","unstructured":"Centre for Research on the Epidemiology of Disasters (CRED) (2024). 2023 Disasters in Numbers: A Significant Year of Disaster Impact, Institute Health and Society\u2014UCLouvain. Technical Report."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Alam, F., Ofli, F., and Imran, M. (2018, January 25\u201328). CrisisMMD: Multimodal Twitter Datasets from Natural Disasters. Proceedings of the 12th International AAAI Conference on Web and Social Media (ICWSM), Palo Alto, CA, USA.","DOI":"10.1609\/icwsm.v12i1.14983"},{"key":"ref_3","unstructured":"Ashktorab, Z., Brown, C., Nandi, M., and Culotta, A. (2014, January 18\u201321). Tweedr: Mining twitter to inform disaster response. Proceedings of the 11th International ISCRAM Conference, University Park, PA, USA."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zou, H.P., Zhou, Y., Zhang, W., and Caragea, C. (2023). Decrisismb: Debiased semi-supervised learning for crisis tweet classification via memory bank. arXiv.","DOI":"10.18653\/v1\/2023.findings-emnlp.406"},{"key":"ref_5","unstructured":"Zou, H.P., Caragea, C., Zhou, Y., and Caragea, D. (2023, January 28\u201331). Semi-supervised few-shot learning for fine-grained disaster tweet classification. Proceedings of the 20th International ISCRAM Conference, ISCRAM 2023, Omaha, NE, USA."},{"key":"ref_6","unstructured":"Sirbu, I., Sosea, T., Caragea, C., Caragea, D., and Rebedea, T. (2022, January 12\u201317). Multimodal Semi-supervised Learning for Disaster Tweet Classification. Proceedings of the 29th International Conference on Computational Linguistics, Gyeongju, Republic of Korea."},{"key":"ref_7","unstructured":"Sohn, K., Berthelot, D., Li, C.L., Zhang, Z., Carlini, N., Cubuk, E.D., Kurakin, A., Zhang, H., and Raffel, C. (2020). Fixmatch: Simplifying semi-supervised learning with consistency and confidence. arXiv."},{"key":"ref_8","first-page":"18408","article-title":"FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling","volume":"34","author":"Zhang","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_9","unstructured":"Wang, Y., Chen, H., Heng, Q., Hou, W., Fan, Y., Wu, Z., Wang, J., Savvides, M., Shinozaki, T., and Raj, B. (2023). FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Sosea, T., and Caragea, C. (2023). MarginMatch: Improving Semi-Supervised Learning with Pseudo-Margins. arXiv.","DOI":"10.1109\/CVPR52729.2023.01514"},{"key":"ref_11","unstructured":"Chen, M., Du, Y., Zhang, Y., Qian, S., and Wang, C. (2021). Semi-Supervised Learning with Multi-Head Co-Training. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"8934","DOI":"10.1109\/TKDE.2022.3220219","article-title":"A Survey on Deep Semi-supervised Learning","volume":"35","author":"Yang","year":"2021","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_13","first-page":"1163","article-title":"Regularization with stochastic transformations and perturbations for deep semi-supervised learning","volume":"29","author":"Sajjadi","year":"2016","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_14","unstructured":"Laine, S., and Aila, T. (2016). Temporal ensembling for semi-supervised learning. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1080\/01621459.1975.10479874","article-title":"Iterative reclassification procedure for constructing an asymptotically optimal rule of allocation in discriminant analysis","volume":"70","author":"McLachlan","year":"1975","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhou, S., Tian, S., Yu, L., Wu, W., Zhang, D., Peng, Z., Zhou, Z., and Wang, J. (2023). FixMatch-LS: Semi-supervised skin lesion classification with label smoothing. Biomed. Signal Process. Control, 84.","DOI":"10.1016\/j.bspc.2023.104709"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhong, Y., Wang, F., Wang, C., and Han, B. (2024, January 27\u201330). Pixelfixmatch: A Semi-Supervised Image Segmentation Method Based on Fixmatch with Pixel Attention. Proceedings of the 2024 IEEE International Symposium on Biomedical Imaging (ISBI), Athens, Greece.","DOI":"10.1109\/ISBI56570.2024.10635605"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ihler, S., Kuhnke, F., Kuhlgatz, T., and Seel, T. (2024, January 16\u201322). Distribution-Aware Multi-Label FixMatch for Semi-Supervised Learning on CheXpert. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPRW63382.2024.00235"},{"key":"ref_19","first-page":"17044","article-title":"Identifying Mislabeled Data using the Area Under the Margin Ranking","volume":"33","author":"Pleiss","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Blum, A., and Mitchell, T. (1998, January 24\u201326). Combining labeled and unlabeled data with co-training. Proceedings of the Eleventh Annual Conference on Computational Learning Theory, Madison, WI, USA.","DOI":"10.1145\/279943.279962"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Qiao, S., Shen, W., Zhang, Z., Wang, B., and Yuille, A. (2018). Deep Co-Training for Semi-Supervised Image Recognition. arXiv.","DOI":"10.1007\/978-3-030-01267-0_9"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zou, H.P., and Caragea, C. (2023). JointMatch: A Unified Approach for Diverse and Collaborative Pseudo-Labeling to Semi-Supervised Text Classification. arXiv.","DOI":"10.18653\/v1\/2023.emnlp-main.451"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1109\/MIS.2012.6","article-title":"Using social media to enhance emergency situation awareness","volume":"27","author":"Yin","year":"2012","journal-title":"IEEE Intell. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"837","DOI":"10.1007\/s11069-014-1217-1","article-title":"Using social media data to understand and assess disasters","volume":"74","author":"Guan","year":"2014","journal-title":"Nat. Hazards"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"e1500779","DOI":"10.1126\/sciadv.1500779","article-title":"Rapid assessment of disaster damage using social media activity","volume":"2","author":"Kryvasheyeu","year":"2016","journal-title":"Sci. Adv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1111\/1468-5973.12194","article-title":"Disaster response aided by tweet classification with a domain adaptation approach","volume":"26","author":"Li","year":"2018","journal-title":"J. Contingencies Crisis Manag."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lagerstrom, R., Arzhaeva, Y., Szul, P., Obst, O., Power, R., Robinson, B., and Bednarz, T. (2016). Image Classification to Support Emergency Situation Awareness. Front. Robot. AI, 3.","DOI":"10.3389\/frobt.2016.00054"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Alam, F., Imran, M., and Ofli, F. (August, January 31). Image4act: Online social media image processing for disaster response. Proceedings of the 2017 IEEE\/ACM International Conference on Advances in Social Networks Analysis and Mining 2017, Sydney, Australia.","DOI":"10.1145\/3110025.3110164"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Nguyen, D.T., Ofli, F., Imran, M., and Mitra, P. (August, January 31). Damage assessment from social media imagery data during disasters. Proceedings of the 2017 IEEE\/ACM International Conference on Advances in Social Networks Analysis and Mining 2017, Sydney, Australia.","DOI":"10.1145\/3110025.3110109"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1007\/s13278-019-0588-4","article-title":"Localizing and quantifying infrastructure damage using class activation mapping approaches","volume":"9","author":"Li","year":"2019","journal-title":"Soc. Netw. Anal. Min."},{"key":"ref_31","unstructured":"Li, X., Caragea, D., Caragea, C., Imran, M., and Ofli, F. (2019, January 19\u201322). Identifying Disaster Damage Images Using a Domain Adaptation Approach. Proceedings of the 16th International Conference on Information Systems for Crisis Response and Management (ISCRAM 2019), Valencia, Spain."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Gautam, A.K., Misra, L., Kumar, A., Misra, K., Aggarwal, S., and Shah, R.R. (2019, January 11\u201313). Multimodal analysis of disaster tweets. Proceedings of the 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), Singapore.","DOI":"10.1109\/BigMM.2019.00-38"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Nalluru, G., Pandey, R., and Purohit, H. (2019, January 12\u201315). Relevancy classification of multimodal social media streams for emergency services. Proceedings of the 2019 IEEE International Conference on Smart Computing (SMARTCOMP), Washington, DC, USA.","DOI":"10.1109\/SMARTCOMP.2019.00040"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Agarwal, M., Leekha, M., Sawhney, R., and Shah, R.R. (2020, January 7\u201312). Crisis-DIAS: Towards Multimodal Damage Analysis-Deployment, Challenges and Assessment. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i01.5369"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Abavisani, M., Wu, L., Hu, S., Tetreault, J., and Jaimes, A. (2020, January 13\u201319). Multimodal Categorization of Crisis Events in Social Media. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01469"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"101760","DOI":"10.1016\/j.ijdrr.2020.101760","article-title":"Leveraging Multimodal Social Media Data for Rapid Disaster Damage Assessment","volume":"51","author":"Hao","year":"2020","journal-title":"Int. J. Disaster Risk Reduct."},{"key":"ref_37","unstructured":"Sosea, T., Sirbu, I., Caragea, C., Caragea, D., and Rebedea, T. (2021, January 23\u201326). Using the Image-Text Relationship to Improve Multimodal Disaster Tweet Classification. Proceedings of the 18th International Conference on Information Systems for Crisis Response and Management (ISCRAM 2021), Blacksburg, VA, USA."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Dinani, S.T., and Caragea, D. (2021, January 18\u201322). Disaster Image Classification Using Capsule Networks. Proceedings of the 2021 International Joint Conference on Neural Networks (IJCNN), Shenzhen, China.","DOI":"10.1109\/IJCNN52387.2021.9534448"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Lai, S., Xu, L., Liu, K., and Zhao, J. (2015, January 25\u201330). Recurrent convolutional neural networks for text classification. Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, USA.","DOI":"10.1609\/aaai.v29i1.9513"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_41","unstructured":"Hua, X.S., and Zhang, H.J. (December, January 30). An attention-based decision fusion scheme for multimedia information retrieval. Proceedings of the Pacific-Rim Conference on Multimedia, Tokyo, Japan."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_43","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"92889","DOI":"10.1109\/ACCESS.2022.3202976","article-title":"CAMM: Cross-attention multimodal classification of disaster-related tweets","volume":"10","author":"Khattar","year":"2022","journal-title":"IEEE Access"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_46","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1607","DOI":"10.1007\/s00521-022-07790-5","article-title":"Multimodal tweet classification in disaster response systems using transformer-based bidirectional attention model","volume":"35","author":"Koshy","year":"2023","journal-title":"Neural Comput. Appl."},{"key":"ref_48","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv."},{"key":"ref_49","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Alam, F., Joty, S., and Imran, M. (2018, January 25\u201328). Graph based semi-supervised learning with convolution neural networks to classify crisis related tweets. Proceedings of the International AAAI Conference on Web and Social Media, Palo Alto, CA, USA.","DOI":"10.1609\/icwsm.v12i1.15047"},{"key":"ref_51","unstructured":"Kiela, D., Bhooshan, S., Firooz, H., and Testuggine, D. (2019). Supervised multimodal bitransformers for classifying images and text. arXiv."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Wei, J., and Zou, K. (2019). EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks. arXiv.","DOI":"10.18653\/v1\/D19-1670"},{"key":"ref_53","unstructured":"Ofli, F., Alam, F., and Imran, M. (2020). Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response. arXiv."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Cubuk, E.D., Zoph, B., Shlens, J., and Le, Q.V. (2019). RandAugment: Practical automated data augmentation with a reduced search space. arXiv.","DOI":"10.1109\/CVPRW50498.2020.00359"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Edunov, S., Ott, M., Auli, M., and Grangier, D. (2018). Understanding Back-Translation at Scale. arXiv.","DOI":"10.18653\/v1\/D18-1045"},{"key":"ref_56","unstructured":"Wang, Y., Chen, H., Fan, Y., Sun, W., Tao, R., Hou, W., Wang, R., Yang, L., Zhou, Z., and Guo, L.Z. (2022). USB: A Unified Semi-supervised Learning Benchmark for Classification. arXiv."},{"key":"ref_57","unstructured":"Andreadis, S., Bozas, A., Gialampoukidis, I., Moumtzidou, A., Fiorin, R., Lombardo, F., Mavropoulos, T., Norbiato, D., Vrochidis, S., and Ferri, M. (2023, January 13\u201315). DisasterMM: Multimedia Analysis of Disaster-Related Social Media Data Task at MediaEval 2022. Proceedings of the MediaEval, Bergen, Norway."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/6\/434\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:39:49Z","timestamp":1760031589000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/6\/434"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,24]]},"references-count":57,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["info16060434"],"URL":"https:\/\/doi.org\/10.3390\/info16060434","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2025,5,24]]}}}