{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T00:03:07Z","timestamp":1772064187909,"version":"3.50.1"},"reference-count":34,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2018,10,6]],"date-time":"2018-10-06T00:00:00Z","timestamp":1538784000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003565","name":"Ministry of Land, Infrastructure and Transport","doi-asserted-by":"publisher","award":["18NSIP-B081011-05"],"award-info":[{"award-number":["18NSIP-B081011-05"]}],"id":[{"id":"10.13039\/501100003565","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJGI"],"abstract":"<jats:p>With the rapid development of Internet of Things (IoT) technologies, the increasing volume and diversity of sources of geospatial big data have created challenges in storing, managing, and processing data. In addition to the general characteristics of big data, the unique properties of spatial data make the handling of geospatial big data even more complicated. To facilitate users implementing geospatial big data applications in a MapReduce framework, several big data processing systems have extended the original Hadoop to support spatial properties. Most of those platforms, however, have included spatial functionalities by embedding them as a form of plug-in. Although offering a convenient way to add new features to an existing system, the plug-in has several limitations. In particular, while executing spatial and nonspatial operations by alternating between the existing system and the plug-in, additional read and write overheads have to be added to the workflow, significantly reducing performance efficiency. To address this issue, we have developed Marmot, a high-performance, geospatial big data processing system based on MapReduce. Marmot extends Hadoop at a low level to support seamless integration between spatial and nonspatial operations of a solid framework, allowing improved performance of geoprocessing workflow. This paper explains the overall architecture and data model of Marmot as well as the main algorithm for automatic construction of MapReduce jobs from a given spatial analysis task. To illustrate how Marmot transforms a sequence of operators for spatial analysis to map and reduce functions in a way to achieve better performance, this paper presents an example of spatial analysis retrieving the number of subway stations per city in Korea. This paper also experimentally demonstrates that Marmot generally outperforms SpatialHadoop, one of the top plug-in based spatial big data frameworks, particularly in dealing with complex and time-intensive queries involving spatial index.<\/jats:p>","DOI":"10.3390\/ijgi7100399","type":"journal-article","created":{"date-parts":[[2018,10,8]],"date-time":"2018-10-08T10:44:53Z","timestamp":1538995493000},"page":"399","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["High-Performance Geospatial Big Data Processing System Based on MapReduce"],"prefix":"10.3390","volume":"7","author":[{"given":"Junghee","family":"Jo","sequence":"first","affiliation":[{"name":"IoT Research Division, Electronics and Telecommunications Research Institute (ETRI), 218 Gajeong-ro, Yuseong-gu, Daejeon 34129, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kang-Woo","family":"Lee","sequence":"additional","affiliation":[{"name":"IoT Research Division, Electronics and Telecommunications Research Institute (ETRI), 218 Gajeong-ro, Yuseong-gu, Daejeon 34129, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,10,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1109\/TPDS.2013.2295810","article-title":"A time efficient approach for detecting errors in big sensor data on cloud","volume":"26","author":"Yang","year":"2015","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1259","DOI":"10.1109\/JIOT.2017.2695535","article-title":"Big sensor data systems for smart cities","volume":"4","author":"Ang","year":"2017","journal-title":"IEEE Internet Things J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.isprsjprs.2015.10.012","article-title":"Geospatial big data handling theory and methods: A review and research challenges","volume":"115","author":"Li","year":"2016","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.bdr.2015.01.003","article-title":"Geospatial big data: Challenges and opportunities","volume":"2","author":"Lee","year":"2015","journal-title":"Big Data Res."},{"key":"ref_5","unstructured":"Amirian, P., Van Loggerenberg, F., Lang, T., and Varga, M. (2015). Geospatial big data for finding useful insights from machine data. GISResearch UK, University of Leeds."},{"key":"ref_6","unstructured":"Morais, C.D. (2018, April 04). Where is the Phrase \u201c80% of Data is Geographic\u201d From?. Available online: http:\/\/www.gislounge.com\/80-percent-data-is-geographic."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Jeansoulin, R. (2016). Review of forty years of technological changes in geomatics toward the big data paradigm. ISPRS Int. J. Geo-Inf., 5.","DOI":"10.3390\/ijgi5090155"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"He, Z., Liu, Q., Deng, M., and Xu, F. (2017, January 10\u201312). March. Handling multiple testing in local statistics of spatial association by controlling the false discovery rate: A comparative analysis. Proceedings of the 2017 IEEE 2nd International Conference Big Data Analysis (ICBDA), Beijing, China.","DOI":"10.1109\/ICBDA.2017.8078722"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Seethapathy, B.K., and Parvathi, R. (2017). A review on spatial big data analytics and visualization. Modern Technologies for Big Data Classification and Clustering, IGI Global.","DOI":"10.4018\/978-1-5225-2805-0.ch007"},{"key":"ref_10","unstructured":"Carpenter, J., and Snell, J. (2018, April 04). Future Trends in Geospatial Information Management: The Five to Ten Year Vision. United Nations Initiative on Global Geospatial Information Management. Available online: http:\/\/ggim.un.org\/ggim_20171012\/docs\/meetings\/GGIM5\/Future%20Trends%20in%20Geospatial%20Information%20Management%20%20the%20five%20to%20ten%20year%20vision.pdf."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Vatsavai, R.R., Ganguly, A., Chandola, V., Stefanidis, A., Klasky, S., and Shekhar, S. (2012, January 7\u20139). Spatiotemporal data mining in the era of big spatial data: Algorithms and applications. Proceedings of the 1st ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data, Redondo Beach, CA, USA.","DOI":"10.1145\/2447481.2447482"},{"key":"ref_12","unstructured":"Dasgupta, A. (2018, April 20). Big Data: The Future is in Analytics. Available online: https:\/\/www.geospatialworld.net\/article\/big-data-the-future-is-in-analytics."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/S0198-9715(04)00045-6","article-title":"The emergence of geoportals and their role in spatial data infrastructures","volume":"29","author":"Maguire","year":"2005","journal-title":"Comput. Environ. Urban Syst."},{"key":"ref_14","unstructured":"Annoni, A., Bernard, L., Fullerton, K., de Groof, H., Kanellopoulos, I., Millot, M., Peedell, S., Rase, D., Smits, P., and Vanderhaegen, M. (2004, January 2\u20136). Towards a European spatial data infrastructure: The INSPIRE initiative. Proceedings of the 7th International Global Spatial Data Infrastructure Conference, Bangalore, India."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/S0198-9715(04)00049-3","article-title":"The European geoportal\u2014One step towards the establishment of a European Spatial Data Infrastructure","volume":"29","author":"Bernard","year":"2005","journal-title":"Comput. Environ. Urban Syst."},{"key":"ref_16","unstructured":"Busby, J.R., and Kelly, P. (2004, January 2\u20136). Australian spatial data infrastructures. Proceedings of the 7th International Global Spatial Data Infrastructure Conference, Bangalore, India."},{"key":"ref_17","unstructured":"Sivakumar, R., Rao, M., and Dasgupta, A.R. (2004, January 2\u20136). Perspectives of India\u2019s national spatial data infrastructure. Proceedings of the 7th International Global Spatial Data Infrastructure Conference, Bangalore, India."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1145\/129888.129894","article-title":"Parallel database system: The future of high performance database systems","volume":"35","author":"Dewitt","year":"1992","journal-title":"Commun. ACM"},{"key":"ref_19","unstructured":"White, T. (2012). Hadoop: The Definitive Guide, O\u2019Reilly Media. [3rd ed.]."},{"key":"ref_20","unstructured":"(2018, May 10). GitHub. Available online: http:\/\/esri.github.io\/gis-tools-for-hadoop."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Aji, A., Wang, F., Vo, H., Lee, R., Liu, Q., Zhang, X., and Saltz, J. (2013, January 26\u201330). Hadoop-GIS: A high performance spatial data warehousing system over MapReduce. Proceedings of the VLDB Endowment 2013, Trento, Italy.","DOI":"10.14778\/2536222.2536227"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1016\/j.compenvurbsys.2014.02.004","article-title":"Constructing gazetteers from volunteered big geo-data based on Hadoop","volume":"61","author":"Gao","year":"2017","journal-title":"Comput. Environ. Urban Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Eldawy, A. (2014, January 22\u201327). SpatialHadoop: Towards flexible and scalable spatial processing using mapreduce. Proceedings of the SIGMOD PhD Symposium 2014, Snowbird, UT, USA.","DOI":"10.1145\/2602622.2602625"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Eldawy, A., and Mokbel, M.F. (2015, January 13\u201317). SpatialHadoop: A MapReduce framework for spatial data. Proceedings of the 2015 IEEE 31st International Conference Data Engineering (ICDE), Seoul, South Korea.","DOI":"10.1109\/ICDE.2015.7113382"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1145\/1327452.1327492","article-title":"MapReduce: Simplified data processing on large clusters","volume":"51","author":"Dean","year":"2008","journal-title":"Commun. ACM"},{"key":"ref_26","unstructured":"Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., and Stoica, I. (2010, January 21\u201325). Spark: Cluster computing with working sets. Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, Boston, MA, USA."},{"key":"ref_27","unstructured":"Agarwal, S., and Rajan, K.S. (2017, January 14\u201319). Analyzing the performance of NoSQL vs. SQL databases for Spatial and Aggregate queries. Proceedings of the Free and Open Source Software for Geospatial (FOSS4G) Conference 2017, Boston, MA, USA."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1186\/s40537-016-0045-4","article-title":"Spatial data extension for Cassandra NoSQL database","volume":"3","author":"Brahim","year":"2016","journal-title":"J. Big Data"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Vasavi, S., Priya, M.P., and Gokhale, A.A. (2018). Framework for Geospatial Query Processing by Integrating Cassandra with Hadoop. Knowl. Comput. Appl., 131\u2013160.","DOI":"10.1007\/978-981-10-6680-1_7"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1145\/2766196.2766198","article-title":"The ecosystem of SpatialHadoop","volume":"6","author":"Eldawy","year":"2015","journal-title":"SIGSPATIAL Spec."},{"key":"ref_31","unstructured":"Manoochehri, M. (2013). Data Just Right: Introduction to Large-Scale Data & Analytics, Addison-Wesley."},{"key":"ref_32","unstructured":"(2018, September 29). Marmot from GitHub. Available online: https:\/\/github.com\/kwlee0220\/marmot.server.dist."},{"key":"ref_33","unstructured":"(2018, September 15). Ministry of Land, Infrastructure and Transport in Korea, V-World. Available online: http:\/\/eng.vworld.kr\/eng\/em_main.do."},{"key":"ref_34","first-page":"3","article-title":"Marmot: A Hadoop-based high performance data storage management system for processing geospatial or geo-spatial big data","volume":"26","author":"Jo","year":"2018","journal-title":"Korean Soc. Geospat. Inf. Sci."}],"container-title":["ISPRS International Journal of Geo-Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2220-9964\/7\/10\/399\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:24:12Z","timestamp":1760196252000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2220-9964\/7\/10\/399"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,10,6]]},"references-count":34,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2018,10]]}},"alternative-id":["ijgi7100399"],"URL":"https:\/\/doi.org\/10.3390\/ijgi7100399","relation":{},"ISSN":["2220-9964"],"issn-type":[{"value":"2220-9964","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,10,6]]}}}