{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,12]],"date-time":"2026-01-12T07:06:10Z","timestamp":1768201570766,"version":"3.49.0"},"reference-count":48,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2024,12,31]],"date-time":"2024-12-31T00:00:00Z","timestamp":1735603200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2022YFC3004402"],"award-info":[{"award-number":["2022YFC3004402"]}]},{"name":"National Key Research and Development Program of China","award":["221111321100"],"award-info":[{"award-number":["221111321100"]}]},{"name":"Henan provincial key research and development program","award":["2022YFC3004402"],"award-info":[{"award-number":["2022YFC3004402"]}]},{"name":"Henan provincial key research and development program","award":["221111321100"],"award-info":[{"award-number":["221111321100"]}]},{"name":"National Supercomputing Center in Zhengzhou","award":["2022YFC3004402"],"award-info":[{"award-number":["2022YFC3004402"]}]},{"name":"National Supercomputing Center in Zhengzhou","award":["221111321100"],"award-info":[{"award-number":["221111321100"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJGI"],"abstract":"<jats:p>As monitoring technologies and data collection methodologies advance, landslide disaster data reflects attributes such as diverse sources, heterogeneity, substantial volumes, and stringent real-time requirements. To bolster the data support capabilities for the monitoring, prevention, and management of landslide disasters, the efficient integration of multi-source heterogeneous data is of paramount importance. The present study proposes an innovative approach to integrate multi-source landslide disaster data by combining the Flink-oriented framework with load balancing task scheduling based on an improved particle swarm optimization (APSO) algorithm. It utilizes Flink\u2019s streaming processing capabilities to efficiently process and store multi-source landslide data. To tackle the issue of uneven cluster load distribution during the integration process, the APSO algorithm is proposed to facilitate cluster load balancing. The findings indicate the following: (1) The multi-source data integration method for landslide disaster based on Flink and APSO proposed in this article, combined with the structural characteristics of landslide disaster data, adopts different integration methods for data in different formats, which can effectively achieve the integration of multi-source landslide data. (2) A multi-source landslide data integration framework based on Flink has been established. Utilizing Kafka as a message queue, a real-time data pipeline was constructed, with Flink facilitating data processing and read\/write operations for the database. This implementation achieves efficient integration of multi-source landslide data. (3) Compared to Flink\u2019s default task scheduling strategy, the cluster load balancing strategy based on APSO demonstrated a reduction of approximately 4.7% in average task execution time and an improvement of approximately 5.4% in average system throughput during actual tests using landslide data sets. The research findings illustrate a significant improvement in the efficiency of data integration processing and system performance.<\/jats:p>","DOI":"10.3390\/ijgi14010012","type":"journal-article","created":{"date-parts":[[2024,12,31]],"date-time":"2024-12-31T10:17:40Z","timestamp":1735640260000},"page":"12","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Integration of Multi-Source Landslide Disaster Data Based on Flink Framework and APSO Load Balancing Task Scheduling"],"prefix":"10.3390","volume":"14","author":[{"given":"Zongmin","family":"Wang","sequence":"first","affiliation":[{"name":"School of Water Conservancy and Transportation, Zhengzhou University, Zhengzhou 450001, China"},{"name":"State Key Laboratory of Tunnel Boring Machine and Intelligent Operation and Maintenance, Zhengzhou 450001, China"}]},{"given":"Huangtaojun","family":"Liang","sequence":"additional","affiliation":[{"name":"School of Water Conservancy and Transportation, Zhengzhou University, Zhengzhou 450001, China"},{"name":"State Key Laboratory of Tunnel Boring Machine and Intelligent Operation and Maintenance, Zhengzhou 450001, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8794-1476","authenticated-orcid":false,"given":"Haibo","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Water Conservancy and Transportation, Zhengzhou University, Zhengzhou 450001, China"},{"name":"State Key Laboratory of Tunnel Boring Machine and Intelligent Operation and Maintenance, Zhengzhou 450001, China"}]},{"given":"Mengyu","family":"Li","sequence":"additional","affiliation":[{"name":"School of Water Conservancy and Transportation, Zhengzhou University, Zhengzhou 450001, China"},{"name":"State Key Laboratory of Tunnel Boring Machine and Intelligent Operation and Maintenance, Zhengzhou 450001, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1474-4222","authenticated-orcid":false,"given":"Yingchun","family":"Cai","sequence":"additional","affiliation":[{"name":"School of Water Conservancy and Transportation, Zhengzhou University, Zhengzhou 450001, China"},{"name":"State Key Laboratory of Tunnel Boring Machine and Intelligent Operation and Maintenance, Zhengzhou 450001, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,12,31]]},"reference":[{"key":"ref_1","first-page":"229","article-title":"Research on Comprehensive Disaster Prevention and Reduction Plan from the Perspective of Resilience","volume":"37","author":"Ge","year":"2022","journal-title":"J. Catastrophol."},{"key":"ref_2","first-page":"710","article-title":"Application of Big Data Techniques in Geological Disaster Analysis and Prevention: A Systematic Review","volume":"41","author":"Liu","year":"2021","journal-title":"J. Disaster Prev. Mitig. Eng."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1016\/j.compenvurbsys.2015.11.005","article-title":"Towards multi-agency sensor information integration for disaster management","volume":"56","author":"Farzad","year":"2016","journal-title":"Comput. Environ. Urban Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1007\/s12583-012-0245-6","article-title":"Comprehensive Study of Landslides Through the Integration of Multi Remote Sensing Techniques: Framework and Latest Advances","volume":"23","author":"Zhong","year":"2012","journal-title":"J. Earth Sci."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chen, Z., Song, J., and Yang, Y. (2018). An Approach to Measuring Semantic Relatedness of Geographic Terminologies Using a Thesaurus and Lexical Database Sources. ISPRS Int. J. Geo-Inf., 7.","DOI":"10.3390\/ijgi7030098"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1007\/s12517-020-05951-1","article-title":"Geological disaster monitoring and early warning system based on big data analysis","volume":"13","author":"Zhang","year":"2020","journal-title":"Arab. J. Geosci."},{"key":"ref_7","first-page":"94","article-title":"Automatic Integration and Analysis of Multi-Source Monitoring Data for Geo-Hazard Warning","volume":"13","author":"He","year":"2014","journal-title":"J. Eng. Geol."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Liu, L., Deng, J., and Tang, Y. (2023). A Dynamic Management and Integration Framework for Models in Landslide Early Warning System. ISPRS Int. J. Geo-Inf., 12.","DOI":"10.3390\/ijgi12050198"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"5317","DOI":"10.1109\/JSTARS.2022.3183684","article-title":"Review of Landslide Monitoring Techniques with IoT Integration Opportunities","volume":"15","author":"Thirugnanam","year":"2022","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"154300","DOI":"10.1109\/ACCESS.2019.2946884","article-title":"A Survey of Distributed Data Stream Processing Frameworks","volume":"7","author":"Isah","year":"2019","journal-title":"IEEE Access"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1007\/s13748-020-00210-6","article-title":"An analysis of technological frameworks for data streams","volume":"9","author":"Puentes","year":"2020","journal-title":"Prog. Artif. Intell."},{"key":"ref_12","first-page":"69","article-title":"Research on Interoperability Models Between Scientific Data Centers","volume":"4","author":"Lu","year":"2022","journal-title":"Front. Data Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"540","DOI":"10.1016\/j.future.2018.07.007","article-title":"SPARTAN: Semantic integration of big spatio-temporal data from streaming and archival sources","volume":"110","author":"Santipantakis","year":"2018","journal-title":"Future Gener. Comput. Syst.-Int. J. Escience"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"989","DOI":"10.1007\/s10462-019-09685-9","article-title":"The state of the art and taxonomy of big data analytics: View from new big data framework","volume":"53","author":"Mohamed","year":"2019","journal-title":"Artif. Intell. Rev."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1045","DOI":"10.1109\/TPDS.2018.2866581","article-title":"Towards Low-Latency Batched Stream Processing by Pre-Scheduling","volume":"30","author":"Jin","year":"2019","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_16","first-page":"205","article-title":"Research on Disaster Data Management Technology and Platform Progress and the Demand it Faces","volume":"34","author":"Wang","year":"2019","journal-title":"J. Catastrophol."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wu, Y., Niu, R., Wang, Y., and Chen, T. (2020). A Fast Deploying Monitoring and Real-Time Early Warning System for the Baige Landslide in Tibet, China. Sensors, 20.","DOI":"10.3390\/s20226619"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2409","DOI":"10.1007\/s10346-020-01402-w","article-title":"Application and verification of a multivariate real-time early warning method for rainfall-induced landslides: Implication for evolution of landslide-generated debris flows","volume":"17","author":"Yang","year":"2020","journal-title":"Landslides"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Du, Y., Xu, X., and He, X. (2024). Optimizing Geo-Hazard Response: LBE-YOLO\u2019s Innovative Lightweight Framework for Enhanced Real-Time Landslide Detection and Risk Mitigation. Remote Sens., 16.","DOI":"10.3390\/rs16030534"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Jin, B., Song, W., Zhao, K., Wei, X., Hu, F., and Jiang, Y. (2017). A High Performance, Spatiotemporal Statistical Analysis System Based on a Spatiotemporal Cloud Platform. ISPRS Int. J. Geo-Inf., 6.","DOI":"10.3390\/ijgi6060165"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.cageo.2019.06.003","article-title":"GeoBeam: A distributed computing framework for spatial data","volume":"131","author":"He","year":"2019","journal-title":"Comput. Geosci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1109\/JSTARS.2016.2547020","article-title":"In-Memory Parallel Processing of Massive Remotely Sensed Data Using an Apache Spark on Hadoop YARN Model","volume":"10","author":"Huang","year":"2017","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_23","first-page":"1141","article-title":"Research and implementation of a Flink-oriented load balancing task scheduling algorithm","volume":"44","author":"Li","year":"2022","journal-title":"Comput. Eng. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1086","DOI":"10.1109\/TBDATA.2022.3233031","article-title":"Cost-Efficient Scheduling of Streaming Applications in Apache Flink on Cloud","volume":"9","author":"Li","year":"2023","journal-title":"IEEE Trans. Big Data"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2703","DOI":"10.3233\/JIFS-222524","article-title":"A resource occupancy ratio-oriented load balancing task scheduling mechanism for Flink","volume":"44","author":"Dai","year":"2023","journal-title":"J. Intell. Fuzzy Syst."},{"key":"ref_26","first-page":"77","article-title":"Extraction Method of Landslide Comprehensive Monitoring Information Based on Data Fusion","volume":"27","author":"Chen","year":"2007","journal-title":"J. Geod. Geodyn."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1033","DOI":"10.1007\/s12303-017-0034-4","article-title":"Landslide prediction, monitoring and early warning: A concise review of state-of-the-art","volume":"21","author":"Chae","year":"2017","journal-title":"Geosci. J."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"951","DOI":"10.1007\/s10346-022-02007-1","article-title":"Landslide monitoring techniques in the Geological Surveys of Europe","volume":"20","author":"Herrera","year":"2023","journal-title":"Landslides"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Li, W., Wang, S., Chen, X., Tian, Y., Gu, Z., Lopez-Carr, A., Schroeder, A., Currier, K., Schildhauer, M., and Zhu, R. (2023). GeoGraphVis: A Knowledge Graph and Geovisualization Empowered Cyberinfrastructure to Support Disaster Response and Humanitarian Aid. ISPRS Int. J. Geo-Inf., 12.","DOI":"10.3390\/ijgi12030112"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"881","DOI":"10.1080\/19475705.2018.1478892","article-title":"Multi-sensor fusion of data for monitoring of Huangtupo landslide in the three Gorges Reservoir (China)","volume":"9","author":"Liu","year":"2018","journal-title":"Geomat. Nat. Hazards Risk"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1016\/j.enggeo.2011.05.003","article-title":"Characterization of the 3D geometry of flow-like landslides: A methodology based on the integration of heterogeneous multi-source data","volume":"128","author":"Travelletti","year":"2012","journal-title":"Eng. Geol."},{"key":"ref_32","first-page":"6","article-title":"Research on Recognition and Auto-correction of Elevation Errors for Contours at the Edge of Map Sheets Based on Hierarchical Grid Index","volume":"37","author":"Liu","year":"2021","journal-title":"Geogr. Geo-Inf. Sci."},{"key":"ref_33","unstructured":"(2019). Date and Time\u2014Representations for Information Interchange\u2014Part 1: Basic Rules (Standard No. Standard ISO 8601-1:2019). Available online: https:\/\/www.iso.org\/standard\/70907.html."},{"key":"ref_34","first-page":"1","article-title":"BIGQA: Declarative Big Data Quality Assessment","volume":"15","author":"Fadlallah","year":"2023","journal-title":"ACM J. Data Inf. Qual."},{"key":"ref_35","first-page":"1","article-title":"Editorial: Special Issue on Quality Assessment and Management in Big Data\u2014Part I","volume":"13","author":"Aljawarneh","year":"2021","journal-title":"ACM J. Data Inf. Qual."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"125","DOI":"10.3233\/SJI-230125","article-title":"Evaluating data quality for blended data using a data quality framework","volume":"40","author":"Parker","year":"2024","journal-title":"Stat. J. IAOS"},{"key":"ref_37","unstructured":"Xu, W. (2019). Flink Introduction and Practice, People\u2019s Post and Telecommunications Press."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1007\/s10707-022-00470-5","article-title":"joinTree: A novel join-oriented multivariate operator for spatio-temporal data management in Flink","volume":"27","author":"Ji","year":"2023","journal-title":"Geoinformatica"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Zhang, G., Li, X., Wang, X., Zhang, Z., Hu, G., Li, Y., and Zhang, R. (2022). Research on the Prediction Problem of Satellite Mission Schedulability Based on Bi-LSTM Model. Aerospace, 9.","DOI":"10.3390\/aerospace9110676"},{"key":"ref_40","first-page":"155","article-title":"A PSO-based task scheduling algorithm improved using a load-balancing technique for the cloud computing environment","volume":"30","author":"Ebadifard","year":"2017","journal-title":"Concurr. Comput. Pract. Exp."},{"key":"ref_41","first-page":"2370","article-title":"Heuristic initialization of PSO task scheduling algorithm in cloud computing","volume":"34","author":"Alsaidy","year":"2022","journal-title":"J. King Saud Univ.\u2014Comput. Inf. Sci."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1016\/j.future.2020.09.016","article-title":"A discrete PSO-based static load balancing algorithm for distributed simulations in a cloud environment","volume":"115","author":"Zhang","year":"2021","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"172628","DOI":"10.1109\/ACCESS.2019.2957024","article-title":"A Framework for the Data Integration of Earthquake Events","volume":"7","author":"Tian","year":"2019","journal-title":"IEEE Access"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"436","DOI":"10.4028\/www.scientific.net\/AMM.694.436","article-title":"The Research of Landslide Monitoring Data Integration Framework Based on Three-Dimensional WebGIS","volume":"694","author":"Zhang","year":"2014","journal-title":"Appl. Mech. Mater."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1016\/j.isprsjprs.2007.04.003","article-title":"Integration of heterogeneous geospatial data in a federated database","volume":"62","author":"Butenuth","year":"2007","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1016\/j.ins.2019.10.046","article-title":"The data richness estimation framework for federated data warehouse integration","volume":"513","author":"Kern","year":"2020","journal-title":"Inf. Sci."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1080\/19475705.2018.1513871","article-title":"Multi-sensor observation fusion scheme based on 3D variational assimilation for landslide monitoring","volume":"10","author":"Liu","year":"2019","journal-title":"Geomat. Nat. Hazards Risk"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Franceschini, R., Rosi, A., Soldato, M.D., Catani, F., and Casagli, N. (2022). Integrating multiple information sources for landslide hazard assessment: The case of Italy. Sci. Rep., 12.","DOI":"10.1038\/s41598-022-23577-z"}],"container-title":["ISPRS International Journal of Geo-Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2220-9964\/14\/1\/12\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:58:03Z","timestamp":1760115483000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2220-9964\/14\/1\/12"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,31]]},"references-count":48,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,1]]}},"alternative-id":["ijgi14010012"],"URL":"https:\/\/doi.org\/10.3390\/ijgi14010012","relation":{},"ISSN":["2220-9964"],"issn-type":[{"value":"2220-9964","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,31]]}}}