{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T01:41:28Z","timestamp":1772502088077,"version":"3.50.1"},"reference-count":69,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,2,11]],"date-time":"2025-02-11T00:00:00Z","timestamp":1739232000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2022YFC3005705"],"award-info":[{"award-number":["2022YFC3005705"]}]},{"name":"National Key Research and Development Program of China","award":["2024YFC3015603"],"award-info":[{"award-number":["2024YFC3015603"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJGI"],"abstract":"<jats:p>The extraction of disaster geospatial intelligence (DGI) from social media data with spatiotemporal attributes plays a crucial role in real-time disaster monitoring and emergency decision-making. However, conventional machine learning approaches struggle with semantic complexity and limited Chinese disaster corpus. Recent advancements in large language models (LLMs) offer new opportunities to overcome these challenges due to their enhanced semantic comprehension and multi-task learning capabilities. This study investigates the potential application of LLMs in disaster intelligence extraction and proposes an efficient, scalable method for multi-hazard DGI extraction. Building upon a unified ontological framework encompassing core natural disaster elements, this method employs parameter-efficient low-rank adaptation (LoRA) fine-tuning to optimize open-source Chinese LLMs using a meticulously curated instruction-tuning dataset. It achieves simultaneous identification of multi-hazard intelligence cues and extraction of disaster spatial entity attributes from unstructured Chinese social media texts through unified semantic parsing and structured knowledge mapping. Compared to pre-trained models such as BERT and ERNIE, the proposed method was shown to achieve state-of-the-art evaluation results, with the highest recognition accuracy (F1-score: 0.9714) and the best performance in structured information generation (BLEU-4 score: 92.9649). Furthermore, we developed and released DGI-Corpus, a Chinese instruction-tuning dataset covering various disaster types, to support the research and application of LLMs in this field. Lastly, the proposed method was applied to analyze the spatiotemporal evolution patterns of the Zhengzhou \u201c7.20\u201d flood disaster. This study enhances the efficiency of natural disaster monitoring and emergency management, offering technical support for disaster response and mitigation decision-making.<\/jats:p>","DOI":"10.3390\/ijgi14020079","type":"journal-article","created":{"date-parts":[[2025,2,12]],"date-time":"2025-02-12T11:08:51Z","timestamp":1739358531000},"page":"79","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Fine-Tuning LLM-Assisted Chinese Disaster Geospatial Intelligence Extraction and Case Studies"],"prefix":"10.3390","volume":"14","author":[{"given":"Yaoyao","family":"Han","sequence":"first","affiliation":[{"name":"Research Center of Geospatial Big Data Application, Chinese Academy of Surveying and Mapping, Beijing 100830, China"},{"name":"School of Spatial Informatics and Geomatics Engineering, Anhui University of Science and Technology, Huainan 232001, China"}]},{"given":"Jiping","family":"Liu","sequence":"additional","affiliation":[{"name":"Research Center of Geospatial Big Data Application, Chinese Academy of Surveying and Mapping, Beijing 100830, China"}]},{"given":"An","family":"Luo","sequence":"additional","affiliation":[{"name":"Research Center of Geospatial Big Data Application, Chinese Academy of Surveying and Mapping, Beijing 100830, China"}]},{"given":"Yong","family":"Wang","sequence":"additional","affiliation":[{"name":"Research Center of Geospatial Big Data Application, Chinese Academy of Surveying and Mapping, Beijing 100830, China"},{"name":"School of Spatial Informatics and Geomatics Engineering, Anhui University of Science and Technology, Huainan 232001, China"}]},{"given":"Shuai","family":"Bao","sequence":"additional","affiliation":[{"name":"Research Center of Geospatial Big Data Application, Chinese Academy of Surveying and Mapping, Beijing 100830, China"}]}],"member":"1968","published-online":{"date-parts":[[2025,2,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"101642","DOI":"10.1016\/j.ijdrr.2020.101642","article-title":"Multi-Hazard Disaster Studies: Monitoring, Detection, Recovery, and Management, Based on Emerging Technologies and Optimal Techniques","volume":"47","author":"Khan","year":"2020","journal-title":"Int. J. Disaster Risk Reduct."},{"key":"ref_2","first-page":"397","article-title":"Big Data and Emergency Management: Concepts, Methodologies, and Applications","volume":"8","author":"Song","year":"2020","journal-title":"IEEE Trans. Big Data"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1016\/j.ssci.2019.02.029","article-title":"Disaster Management 2.0: A Real-Time Disaster Damage Assessment Model Based on Mobile Social Media Data\u2014A Case Study of Weibo (Chinese Twitter)","volume":"115","author":"Shan","year":"2019","journal-title":"Saf. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Sit, M.A., Koylu, C., and Demir, I. (2020). Identifying Disaster-Related Tweets and Their Semantic, Spatial and Temporal Context Using Deep Learning, Natural Language Processing and Spatial Analysis: A Case Study of Hurricane Irma. Social Sensing and Big Data Computing for Disaster Management, Routledge.","DOI":"10.4324\/9781003106494-2"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1016\/j.ijinfomgt.2019.04.004","article-title":"Social Media for Intelligent Public Information and Warning in Disasters: An Interdisciplinary Review","volume":"49","author":"Zhang","year":"2019","journal-title":"Int. J. Inf. Manag."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yang, Y., Wu, Z., Yang, Y., Lian, S., Guo, F., and Wang, Z. (2022). A Survey of Information Extraction Based on Deep Learning. Appl. Sci., 12.","DOI":"10.3390\/app12199691"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1016\/j.neucom.2021.05.103","article-title":"An Introduction to Deep Learning in Natural Language Processing: Models, Techniques, and Tools","volume":"470","author":"Lauriola","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_8","first-page":"90","article-title":"Event Extraction via DMCNN in Open Domain Public Sentiment Information","volume":"Volume 1258","author":"Qin","year":"2020","journal-title":"Data Science"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ritter, A., Etzioni, O., and Clark, S. (2012, January 12\u201316). Open Domain Event Extraction from Twitter. Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Beijing, China.","DOI":"10.1145\/2339530.2339704"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Tekumalla, R., and Banda, J.M. (2022, January 17\u201320). TweetDIS: A Large Twitter Dataset for Natural Disasters Built Using Weak Supervision. Proceedings of the 2022 IEEE International Conference on Big Data (Big Data), Osaka, Japan.","DOI":"10.1109\/BigData55660.2022.10020214"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Alam, F., Qazi, U., Imran, M., and Ofli, F. (2021, January 7\u201310). Humaid: Human-Annotated Disaster Incidents Data from Twitter with Deep Learning Benchmarks. Proceedings of the International AAAI Conference on Web and Social Media, Online.","DOI":"10.1609\/icwsm.v15i1.18116"},{"key":"ref_12","unstructured":"McCreadie, R., and Buntain, C. (2023, January 28\u201331). CrisisFacts: Building and Evaluating Crisis Timelines. Proceedings of the 20th International ISCRAM Conference, Omaha, NE, USA."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"04020068","DOI":"10.1061\/(ASCE)ME.1943-5479.0000836","article-title":"Rapid Assessment of Disaster Impacts on Highways Using Social Media","volume":"36","author":"Chen","year":"2020","journal-title":"J. Manag. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"101760","DOI":"10.1016\/j.ijdrr.2020.101760","article-title":"Leveraging Multimodal Social Media Data for Rapid Disaster Damage Assessment","volume":"51","author":"Hao","year":"2020","journal-title":"Int. J. Disaster Risk Reduct."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1007\/s11069-020-04016-6","article-title":"Misinformation Debunking and Cross-Platform Information Sharing through Twitter during Hurricanes Harvey and Irma: A Case Study on Shelters and ID Checks","volume":"103","author":"Hunt","year":"2020","journal-title":"Nat. Hazards"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"101824","DOI":"10.1016\/j.compenvurbsys.2022.101824","article-title":"VictimFinder: Harvesting Rescue Requests in Disaster Response from Social Media with BERT","volume":"95","author":"Zhou","year":"2022","journal-title":"Comput. Environ. Urban Syst."},{"key":"ref_17","unstructured":"Minaee, S., Mikolov, T., Nikzad, N., Chenaghlu, M., Socher, R., Amatriain, X., and Gao, J. (2024). Large Language Models: A Survey. arXiv."},{"key":"ref_18","first-page":"101675","article-title":"Decoding ChatGPT: A Taxonomy of Existing Research, Current Challenges, and Possible Future Directions","volume":"35","author":"Sohail","year":"2023","journal-title":"J. King Saud Univ.-Comput. Inf. Sci."},{"key":"ref_19","unstructured":"Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., and Bhosale, S. (2023). Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Liu, Y., He, H., Han, T., Zhang, X., Liu, M., Tian, J., Zhang, Y., Wang, J., Gao, X., and Zhong, T. (2024). Understanding LLMs: A Comprehensive Overview from Training to Inference. arXiv.","DOI":"10.2139\/ssrn.4706201"},{"key":"ref_21","unstructured":"Mathav, R.J., Kushala, V.M., Warrier, H., and Gupta, Y. (2024). Fine Tuning LLM for Enterprise: Practical Guidelines and Recommendations. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3143","DOI":"10.1109\/TCSS.2022.3211552","article-title":"Application of Social Sensors in Natural Disasters Emergency Management: A Review","volume":"10","author":"Shi","year":"2023","journal-title":"IEEE Trans. Comput. Soc. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Owuor, I., and Hochmair, H.H. (2020). An Overview of Social Media Apps and Their Potential Role in Geospatial Research. ISPRS Int. J. Geo-Inf., 9.","DOI":"10.3390\/ijgi9090526"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1376","DOI":"10.1111\/tgis.12922","article-title":"Investigations of Disaster Information Representation from a Geospatial Perspective: Progress, Challenges and Recommendations","volume":"26","author":"Li","year":"2022","journal-title":"Trans. GIS"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1057\/s41599-022-01457-1","article-title":"Topical and Emotional Expressions Regarding Extreme Weather Disasters on Social Media: A Comparison of Posts from Official Media and the Public","volume":"9","author":"Han","year":"2022","journal-title":"Humanit. Soc. Sci. Commun."},{"key":"ref_26","first-page":"171","article-title":"Leveraging Social Media Sentiment Analysis for Enhanced Disaster Management: A Systematic Review and Future Research Agenda","volume":"15","author":"Hashim","year":"2025","journal-title":"J. Syst. Manag. Sci."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"He, W., Yuan, Q., and Li, N. (2023). Research on the Characteristics of Internet Public Opinion and Public Sentiment after the Sichuan Earthquake Based on the Perspective of Weibo. Appl. Sci., 13.","DOI":"10.3390\/app13031335"},{"key":"ref_28","first-page":"1","article-title":"Sentiment Analysis of Twitter Users over Time: The Case of the Boston Bombing Tragedy","volume":"Volume 258","author":"Sugumaran","year":"2016","journal-title":"E-Life: Web-Enabled Convergence of Commerce, Work, and Social Life"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Parsons, S., Atkinson, P.M., Simperl, E., and Weal, M. (2015, January 18\u201322). Thematically Analysing Social Network Content During Disasters Through the Lens of the Disaster Management Lifecycle. Proceedings of the 24th International Conference on World Wide Web, Florence, Italy.","DOI":"10.1145\/2740908.2741721"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1017\/S1351324918000025","article-title":"Topics and Topical Phases in German Social Media Communication during a Disaster","volume":"24","author":"Schlaf","year":"2018","journal-title":"Nat. Lang. Eng."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"851","DOI":"10.1007\/s11069-021-05081-1","article-title":"Early Detection of Emergency Events from Social Media: A New Text Clustering Approach","volume":"111","author":"Huang","year":"2022","journal-title":"Nat. Hazards"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"108401","DOI":"10.1016\/j.compeleceng.2022.108401","article-title":"Multi-Label Disaster Text Classification via Supervised Contrastive Learning for Social Media Data","volume":"104","author":"Xie","year":"2022","journal-title":"Comput. Electr. Eng."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"101313","DOI":"10.1016\/j.ijdrr.2019.101313","article-title":"Question-Answering Dialogue System for Emergency Operations","volume":"41","author":"Chan","year":"2019","journal-title":"Int. J. Disaster Risk Reduct."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Kemavuthanon, K., and Uchida, O. (2020). Integrated Question-Answering System for Natural Disaster Domains Based on Social Media Messages Posted at the Time of Disaster. Information, 11.","DOI":"10.3390\/info11090456"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"102482","DOI":"10.1016\/j.ijdrr.2021.102482","article-title":"Social Media Data-Based Typhoon Disaster Assessment","volume":"64","author":"Chen","year":"2021","journal-title":"Int. J. Disaster Risk Reduct."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"e1500779","DOI":"10.1126\/sciadv.1500779","article-title":"Rapid Assessment of Disaster Damage Using Social Media Activity","volume":"2","author":"Kryvasheyeu","year":"2016","journal-title":"Sci. Adv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1080\/23789689.2017.1294859","article-title":"Defining Resilience Analytics for Interdependent Cyber-Physical-Social Networks","volume":"2","author":"Barker","year":"2017","journal-title":"Sustain. Resilient Infrastruct."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Arachie, C., Gaur, M., Anzaroot, S., Groves, W., Zhang, K., and Jaimes, A. (2020, January 7\u201312). Unsupervised Detection of Sub-Events in Large Scale Disasters. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i01.5370"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"4215","DOI":"10.1007\/s10708-023-10858-x","article-title":"The Wisdom of Crowds for Improved Disaster Resilience: A near-Real-Time Analysis of Crowdsourced Social Media Data on the 2021 Flood in Germany","volume":"88","author":"Moghadas","year":"2023","journal-title":"GeoJournal"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Kumar, A., Jiang, M., and Fang, Y. (2014, January 6\u201311). Where Not to Go?: Detecting Road Hazards Using Twitter. Proceedings of the 37th international ACM SIGIR Conference on Research & Development in Information Retrieval, Gold Coast, Australia.","DOI":"10.1145\/2600428.2609550"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Hadi, M.U., Tashi, Q.A., Shah, A., Qureshi, R., Muneer, A., Irfan, M., Zafar, A., Shaikh, M.B., Akhtar, N., and Wu, J. (TechRxiv, 2023). Large Language Models: A Comprehensive Survey of Its Applications, Challenges, Limitations, and Future Prospects, TechRxiv.","DOI":"10.36227\/techrxiv.23589741.v3"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"2289","DOI":"10.1080\/13658816.2023.2266495","article-title":"Geo-Knowledge-Guided GPT Models Improve the Extraction of Location Descriptions from Disaster-Related Social Media Messages","volume":"37","author":"Hu","year":"2023","journal-title":"Int. J. Geogr. Inf. Sci."},{"key":"ref_43","unstructured":"Colverd, G., Darm, P., Silverberg, L., and Kasmanoff, N. (2023). FloodBrain: Flood Disaster Reporting by Web-Based Retrieval Augmented Generation with an LLM. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Xia, Y., Huang, Y., Qiu, Q., Zhang, X., Miao, L., and Chen, Y. (2024). A Question and Answering Service of Typhoon Disasters Based on the T5 Large Language Model. ISPRS Int. J. Geo-Inf., 13.","DOI":"10.3390\/ijgi13050165"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Hao, Y., Qi, J., Ma, X., Wu, S., Liu, R., and Zhang, X. (2024). An LLM-Based Inventory Construction Framework of Urban Ground Collapse Events with Spatiotemporal Locations. ISPRS Int. J. Geo-Inf., 13.","DOI":"10.3390\/ijgi13040133"},{"key":"ref_46","unstructured":"Yin, K., Liu, C., Mostafavi, A., and Hu, X. (2024). CrisisSense-LLM: Instruction Fine-Tuned Large Language Model for Multi-Label Social Media Text Classification in Disaster Informatics. arXiv."},{"key":"ref_47","first-page":"100647","article-title":"Crowdsourcing Intelligence for Improving Disaster Forecasts","volume":"5","author":"Wang","year":"2024","journal-title":"Innovation"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1016\/j.compenvurbsys.2005.04.002","article-title":"Ontology-Based Discovery of Geographic Information Services\u2014An Application in Disaster Management","volume":"30","author":"Klien","year":"2006","journal-title":"Comput. Environ. Urban Syst."},{"key":"ref_49","first-page":"1","article-title":"Theory and Practice on Disaster System Research in a Fourth Time","volume":"14","author":"Shi","year":"2005","journal-title":"J. Nat. Disasters"},{"key":"ref_50","unstructured":"Wang, X.H., Zhang, D.Q., Gu, T., and Pung, H.K. (2004, January 14\u201317). Ontology Based Context Modeling and Reasoning Using OWL. Proceedings of the IEEE Annual Conference on Pervasive Computing and Communications Workshops, Orlando, FL, USA. Proceedings of the second."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Ning, H., Li, Z., Akinboyewa, T., and Lessani, M.N. (2024). An Autonomous GIS Agent Framework for Geospatial Data Retrieval. arXiv.","DOI":"10.1080\/17538947.2025.2458688"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"e13282","DOI":"10.1111\/tgis.13282","article-title":"Advancing Large Language Models for Spatiotemporal and Semantic Association Mining of Similar Environmental Events","volume":"29","author":"Tian","year":"2025","journal-title":"Trans. GIS"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Alam, F., Ofli, F., and Imran, M. (2018, January 3\u20136). Crisismmd: Multimodal Twitter Datasets from Natural Disasters. Proceedings of the International AAAI Conference on Web and Social Media, Buffalo, NY, USA.","DOI":"10.1609\/icwsm.v12i1.14983"},{"key":"ref_54","unstructured":"Sun, Y., Wan, Z., Ueda, N., Yahata, S., Cheng, F., Chu, C., and Kurohashi, S. (2024). Rapidly Developing High-quality Instruction Data and Evaluation Benchmark for Large Language Models with Minimal Human Effort: A Case Study on Japanese. arXiv."},{"key":"ref_55","unstructured":"Peng, B., Li, C., He, P., Galley, M., and Gao, J. (2023). Instruction Tuning with GPT-4. arXiv."},{"key":"ref_56","unstructured":"(2018). Emergency Classification and Coding (Standard No. SAC\/TC307; GBT 28921\u20132012)."},{"key":"ref_57","unstructured":"(2016). Classification and Coding for Natural Disaster Exposure (Standard No. SAC\/TC307; GBT 32572-2016)."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Liu, H., Luo, N., and Zhao, Q. (2022). Research on the Construction of Typhoon Disaster Chain Based on Chinese Web Corpus. J. Mar. Sci. Eng., 10.","DOI":"10.3390\/jmse10010044"},{"key":"ref_59","first-page":"1950","article-title":"Few-Shot Parameter-Efficient Fine-Tuning Is Better and Cheaper than in-Context Learning","volume":"35","author":"Liu","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_60","unstructured":"Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., and Huang, F. (2024). Qwen2 Technical Report. arXiv."},{"key":"ref_61","unstructured":"Hu, S., Tu, Y., Han, X., He, C., Cui, G., Long, X., Zheng, Z., Fang, Y., Huang, Y., and Zhao, W. (2024). MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies. arXiv."},{"key":"ref_62","unstructured":"Xin, C., Lu, Y., Lin, H., Zhou, S., Zhu, H., Wang, W., Liu, Z., Han, X., and Sun, L. (2024, January 20\u201325). Beyond Full Fine-Tuning: Harnessing the Power of LoRA for Multi-Task Instruction Tuning. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, Torino, Italy."},{"key":"ref_63","unstructured":"Mai, G., Huang, W., Sun, J., Song, S., Mishra, D., Liu, N., Gao, S., Liu, T., Cong, G., and Hu, Y. (2023). On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligence. arXiv."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Huang, C., Chen, S., Li, Z., Qu, J., Xiao, Y., Liu, J., and Chen, Z. (2024, January 11\u201316). GeoAgent: To Empower LLMs Using Geospatial Tools for Address Standardization. Proceedings of the Findings of the Association for Computational Linguistics ACL, Bangkok, Thailand.","DOI":"10.18653\/v1\/2024.findings-acl.362"},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1016\/j.patcog.2019.02.023","article-title":"The Impact of Class Imbalance in Classification Performance Metrics Based on the Binary Confusion Matrix","volume":"91","author":"Luque","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_66","first-page":"100025","article-title":"Optimization of Paraphrase Generation and Identification Using Language Models in Natural Language Processing","volume":"1","author":"Palivela","year":"2021","journal-title":"Int. J. Inf. Manag. Data Insights"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Zheng, Y., Zhang, R., Zhang, J., Ye, Y., Luo, Z., Feng, Z., and Ma, Y. (2024). LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models. arXiv.","DOI":"10.18653\/v1\/2024.acl-demos.38"},{"key":"ref_68","doi-asserted-by":"crossref","unstructured":"Luceri, L., Boniardi, E., and Ferrara, E. (2024, January 13\u201317). Leveraging Large Language Models to Detect Influence Campaigns on Social Media. Proceedings of the Companion Proceedings of the ACM on Web Conference, Singapore.","DOI":"10.1145\/3589335.3651912"},{"key":"ref_69","unstructured":"National Development and Reform Commission (2024, November 02). Available online: https:\/\/www.ndrc.gov.cn\/fzggw\/jgsj\/yxj\/sjdt\/201504\/t20150410_987672.html."}],"container-title":["ISPRS International Journal of Geo-Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2220-9964\/14\/2\/79\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:31:34Z","timestamp":1760027494000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2220-9964\/14\/2\/79"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,11]]},"references-count":69,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,2]]}},"alternative-id":["ijgi14020079"],"URL":"https:\/\/doi.org\/10.3390\/ijgi14020079","relation":{},"ISSN":["2220-9964"],"issn-type":[{"value":"2220-9964","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,11]]}}}