{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T19:37:30Z","timestamp":1767987450480,"version":"3.49.0"},"reference-count":34,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2021,11,2]],"date-time":"2021-11-02T00:00:00Z","timestamp":1635811200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U1711266 and No. 41925007"],"award-info":[{"award-number":["U1711266 and No. 41925007"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"the Strategic Priority Research Program of the Chinese Academy of Sciences","award":["XDA19090128"],"award-info":[{"award-number":["XDA19090128"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJGI"],"abstract":"<jats:p>Multi-source Internet of Things (IoT) data, archived in institutions\u2019 repositories, are becoming more and more widely open-sourced to make them publicly accessed by scientists, developers, and decision makers via web services to promote researches on geohazards prevention. In this paper, we design and implement a big data-turbocharged system for effective IoT data management following the data lake architecture. We first propose a multi-threading parallel data ingestion method to ingest IoT data from institutions\u2019 data repositories in parallel. Next, we design storage strategies for both ingested IoT data and processed IoT data to store them in a scalable, reliable storage environment. We also build a distributed cache layer to enable fast access to IoT data. Then, we provide users with a unified, SQL-based interactive environment to enable IoT data exploration by leveraging the processing ability of Apache Spark. In addition, we design a standard-based metadata model to describe ingested IoT data and thus support IoT dataset discovery. Finally, we implement a prototype system and conduct experiments on real IoT data repositories to evaluate the efficiency of the proposed system.<\/jats:p>","DOI":"10.3390\/ijgi10110743","type":"journal-article","created":{"date-parts":[[2021,11,2]],"date-time":"2021-11-02T12:39:42Z","timestamp":1635856782000},"page":"743","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Efficient IoT Data Management for Geological Disasters Based on Big Data-Turbocharged Data Lake Architecture"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0394-2357","authenticated-orcid":false,"given":"Xiaohui","family":"Huang","sequence":"first","affiliation":[{"name":"School of Computer Science, China University of Geosciences, Wuhan 430074, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Junqing","family":"Fan","sequence":"additional","affiliation":[{"name":"School of Computer Science, China University of Geosciences, Wuhan 430074, China"},{"name":"Hubei Key Laboratory of Intelligent Geo-Information Processing, China University of Geosciences, Wuhan 430074, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ze","family":"Deng","sequence":"additional","affiliation":[{"name":"School of Computer Science, China University of Geosciences, Wuhan 430074, China"},{"name":"Hubei Key Laboratory of Intelligent Geo-Information Processing, China University of Geosciences, Wuhan 430074, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jining","family":"Yan","sequence":"additional","affiliation":[{"name":"School of Computer Science, China University of Geosciences, Wuhan 430074, China"},{"name":"Hubei Key Laboratory of Intelligent Geo-Information Processing, China University of Geosciences, Wuhan 430074, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiabao","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science, China University of Geosciences, Wuhan 430074, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2766-0845","authenticated-orcid":false,"given":"Lizhe","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science, China University of Geosciences, Wuhan 430074, China"},{"name":"Hubei Key Laboratory of Intelligent Geo-Information Processing, China University of Geosciences, Wuhan 430074, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Yu, M., Yang, C., and Li, Y. (2018). Big Data in Natural Disaster Management: A Review. Geosciences, 8.","DOI":"10.3390\/geosciences8050165"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Zhao, C., and Lu, Z. (2018). Remote Sensing of Landslides\u2014A Review. Remote Sens., 10.","DOI":"10.3390\/rs10020279"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"939","DOI":"10.1007\/s10479-017-2584-2","article-title":"Big data and disaster management: A systematic review and agenda for future research","volume":"283","author":"Akter","year":"2019","journal-title":"Ann. Oper. Res."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"101642","DOI":"10.1016\/j.ijdrr.2020.101642","article-title":"Multi-hazard disaster studies: Monitoring, detection, recovery, and management, based on emerging technologies and optimal techniques","volume":"47","author":"Khan","year":"2020","journal-title":"Int. J. Disaster Risk Reduct."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4371","DOI":"10.1109\/JIOT.2019.2952593","article-title":"A Survey of Internet of Things (IoT) for Geohazard Prevention: Applications, Technologies, and Challenges","volume":"7","author":"Mei","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"4342","DOI":"10.1109\/JIOT.2020.2985598","article-title":"Data Science for the Internet of Things","volume":"7","author":"Piccialli","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_7","first-page":"74:1","article-title":"Analytics for the Internet of Things: A Survey","volume":"51","author":"Siow","year":"2018","journal-title":"ACM Comput. Surv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4581","DOI":"10.1007\/s11071-021-06532-x","article-title":"Synchronization of chaotic artificial neurons and its application to secure image transmission under MQTT for IoT protocol","volume":"104","author":"Maritza","year":"2021","journal-title":"Nonlinear Dyn."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1186\/s13638-021-02033-y","article-title":"5G IoT-based geohazard monitoring and early warning system and its application","volume":"2021","author":"Li","year":"2021","journal-title":"EURASIP J. Wirel. Commun. Netw."},{"key":"ref_10","first-page":"1","article-title":"On rapid multidisciplinary response aspects for Samos 2020 M7.0 earthquake","volume":"69","author":"Foumelis","year":"2021","journal-title":"Acta Geophys."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"9278","DOI":"10.1109\/JIOT.2021.3056586","article-title":"A Self-Powered, Real-Time, LoRaWAN IoT-Based Soil Health Monitoring System","volume":"8","author":"Ramson","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"131:1","DOI":"10.1145\/3419634","article-title":"A Survey on IoT Big Data: Current Status, 13 V\u2019s Challenges, and Future Directions","volume":"53","author":"Bansal","year":"2021","journal-title":"ACM Comput. Surv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Nikoui, T.S., Rahmani, A.M., Balador, A., and Javadi, H.H.S. (2021). Internet of Things architecture challenges: A systematic review. Int. J. Commun. Syst., 34.","DOI":"10.1002\/dac.4678"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2901","DOI":"10.14778\/3415478.3415504","article-title":"Apache IoTDB: Time-Series Database for Internet of Things","volume":"13","author":"Wang","year":"2020","journal-title":"Proc. VLDB Endow."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1109\/JIOT.2017.2722378","article-title":"An Ingestion and Analytics Architecture for IoT Applied to Smart City Use Cases","volume":"5","author":"Akbar","year":"2018","journal-title":"IEEE Internet Things J."},{"key":"ref_16","unstructured":"Wu, X., Jermaine, C., Xiong, L., Hu, X., Kotevska, O., Lu, S., Xu, W., Aluru, S., Zhai, C., and Al-Masri, E. (2020). Developing an Architecture for IoT Interoperability in Healthcare: A Case Study of Real-time SpO2 Signal Monitoring and Analysis. Proceedings of the IEEE International Conference on Big Data, Big Data 2020, Atlanta, GA, USA, 10\u201313 December 2020, IEEE."},{"key":"ref_17","unstructured":"Chbeir, R., Agrawal, R., and Biskri, I. (2016, January 1\u20134). The next information architecture evolution: The data lake wave. Proceedings of the 8th International Conference on Management of Digital EcoSystems, MEDES 2016, Biarritz, France."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Skluzacek, T.J., Chard, K., and Foster, I.T. (2016, January 14). Klimatic: A Virtual Data Lake for Harvesting and Distribution of Geospatial Data. Proceedings of the 1st Joint International Workshop on Parallel Data Storage and data Intensive Scalable Computing Systems, PDSW-DISCS@SC 2016, Salt Lake, UT, USA.","DOI":"10.1109\/PDSW-DISCS.2016.010"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Mehmood, H., Gilman, E., Cort\u00e9s, M., Kostakos, P., Byrne, A., Valta, K., Tekes, S., and Riekki, J. (2019, January 8\u201312). Implementing Big Data Lake for Heterogeneous Data Sources. Proceedings of the 35th IEEE International Conference on Data Engineering Workshops, ICDE Workshops 2019, Macao, China.","DOI":"10.1109\/ICDEW.2019.00-37"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1986","DOI":"10.14778\/3352063.3352116","article-title":"Data Lake Management: Challenges and Opportunities","volume":"12","author":"Nargesian","year":"2019","journal-title":"Proc. VLDB Endow."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Cuzzocrea, A. (2021, January 17\u201320). Big Data Lakes: Models, Frameworks, and Techniques. Proceedings of the IEEE International Conference on Big Data and Smart Computing, BigComp 2021, Jeju Island, Korea.","DOI":"10.1109\/BigComp51126.2021.00010"},{"key":"ref_22","unstructured":"Bershad, B.N., and Mogul, J.C. (2006, January 6\u20138). Ceph: A Scalable, High-Performance Distributed File System. Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI \u201906), Seattle, WA, USA."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Vohra, D. (2016). Apache parquet. Practical Hadoop Ecosystem, Springer.","DOI":"10.1007\/978-1-4842-2199-0"},{"key":"ref_24","unstructured":"Li, H. (2018). Alluxio: A Virtual Distributed File System. [Ph.D. Thesis, University of California]."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1145\/2934664","article-title":"Apache Spark: A unified engine for big data processing","volume":"59","author":"Zaharia","year":"2016","journal-title":"Commun. ACM"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"13147","DOI":"10.1007\/s00521-019-04678-9","article-title":"High-performance IoT streaming data prediction system using Spark: A case study of air pollution","volume":"32","author":"Jin","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"ref_27","unstructured":"Sellis, T.K., Davidson, S.B., and Ives, Z.G. (June, January 31). Spark SQL: Relational Data Processing in Spark. Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, Melbourne, VIC, Australia."},{"key":"ref_28","unstructured":"Aguiar, A., Chiba, S., and Boix, E.G. (2020, January 23\u201326). Towards dynamic SQL compilation in Apache Spark. Proceedings of the Programming\u201920: 4th International Conference on the Art, Science, and Engineering of Programming, Porto, Portugal."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1902","DOI":"10.14778\/3352063.3352095","article-title":"Juneau: Data Lake Management for Jupyter","volume":"12","author":"Zhang","year":"2019","journal-title":"Proc. VLDB Endow."},{"key":"ref_30","unstructured":"Sanielevici, S. (2018, January 22\u201326). Building Big Data Processing and Visualization Pipeline through Apache Zeppelin. Proceedings of the 23\u201326 Practice and Experience on Advanced Research Computing, PEARC 2018, Pittsburgh, PA, USA."},{"key":"ref_31","unstructured":"\u00d6zcan, F., Koutrika, G., and Madden, S. (July, January 26). Constance: An Intelligent Data Lake System. Proceedings of the 2016 International Conference on Management of Data, SIGMOD Conference 2016, San Francisco, CA, USA."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2732","DOI":"10.1109\/TFUZZ.2018.2812157","article-title":"Soft and Declarative Fishing of Information in Big Data Lake","volume":"26","author":"Stabla","year":"2018","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Brodeur, J., Coetzee, S., Danko, D.M., Garcia, S., and Hjelmager, J. (2019). Geographic Information Metadata\u2014An Outlook from the International Standardization Perspective. ISPRS Int. J. Geo-Inf., 8.","DOI":"10.3390\/ijgi8060280"},{"key":"ref_34","unstructured":"Lim, E., Winslett, M., Sanderson, M., Fu, A.W., Sun, J., Culpepper, J.S., Lo, E., Ho, J.C., Donato, D., and Agrawal, R. (2017, January 6\u201310). CoreDB: A Data Lake Service. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM 2017, Singapore."}],"container-title":["ISPRS International Journal of Geo-Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2220-9964\/10\/11\/743\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:24:34Z","timestamp":1760167474000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2220-9964\/10\/11\/743"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,2]]},"references-count":34,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["ijgi10110743"],"URL":"https:\/\/doi.org\/10.3390\/ijgi10110743","relation":{},"ISSN":["2220-9964"],"issn-type":[{"value":"2220-9964","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,2]]}}}