{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T10:58:26Z","timestamp":1777546706456,"version":"3.51.4"},"reference-count":23,"publisher":"Walter de Gruyter GmbH","issue":"1","license":[{"start":{"date-parts":[[2022,1,1]],"date-time":"2022-01-01T00:00:00Z","timestamp":1640995200000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,7,26]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>MapReduce (MR) is a technique used to improve distributed data processing vastly and can massively speed up computation. Hadoop and MR rely on memory-intensive JVM and Java. A MR framework based on High-Performance Computing (HPC) could be used, which is both memory-efficient and faster than standard MR. This article explores a C++-based approach to MR and its feasibility on multiple factors like developer friendliness, deployment interface, efficiency, and scalability. This article also introduces Eager Reduction and Delayed Reduction techniques to speed up MR.<\/jats:p>","DOI":"10.1515\/comp-2022-0246","type":"journal-article","created":{"date-parts":[[2022,7,26]],"date-time":"2022-07-26T16:31:09Z","timestamp":1658853069000},"page":"238-247","source":"Crossref","is-referenced-by-count":4,"title":["An alternative C++-based HPC system for Hadoop MapReduce"],"prefix":"10.1515","volume":"12","author":[{"given":"Vignesh","family":"Srinivasakumar","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering (SCOPE), Vellore Institute of Technology , Chennai , Tamil Nadu , India"}]},{"given":"Muthumanikandan","family":"Vanamoorthy","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering (SCOPE), Vellore Institute of Technology , Chennai , Tamil Nadu , India"}]},{"given":"Siddarth","family":"Sairaj","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering (SCOPE), Vellore Institute of Technology , Chennai , Tamil Nadu , India"}]},{"given":"Sainath","family":"Ganesh","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering (SCOPE), Vellore Institute of Technology , Chennai , Tamil Nadu , India"}]}],"member":"374","published-online":{"date-parts":[[2022,7,26]]},"reference":[{"key":"2022081707553241786_j_comp-2022-0246_ref_001","doi-asserted-by":"crossref","unstructured":"V. Kalavri and V. Vlassov, \u201cMapReduce: Limitations, optimizations and open issues,\u201d 2013 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, 2013. 10.1109\/trustcom.2013.126.","DOI":"10.1109\/TrustCom.2013.126"},{"key":"2022081707553241786_j_comp-2022-0246_ref_002","unstructured":"\u201c13 Big Limitations of Hadoop & Solution To Hadoop Drawbacks,\u201d By DataFlair Team. Accessed on: [Online], March 2019. https:\/\/data-flair.training\/blogs\/13-limitations-of-hadoop\/."},{"key":"2022081707553241786_j_comp-2022-0246_ref_003","doi-asserted-by":"crossref","unstructured":"J.-F. Weets, M. K. Kakhani, and A. Kumar, \u201cLimitations and challenges of HDFS and MapReduce,\u201d 2015 International Conference on Green Computing and Internet of Things (ICGCIoT), 2015. 10.1109\/icgciot.2015.7380524.","DOI":"10.1109\/ICGCIoT.2015.7380524"},{"key":"2022081707553241786_j_comp-2022-0246_ref_004","doi-asserted-by":"crossref","unstructured":"D. Jeffrey and S. Ghemawat, \u201cMapReduce: simplified data processing on large clusters,\u201d Commun. ACM51, vol. 1, no. January 2008, pp. 107\u201313, 2008. 10.1145\/1327452.1327492.","DOI":"10.1145\/1327452.1327492"},{"key":"2022081707553241786_j_comp-2022-0246_ref_005","doi-asserted-by":"crossref","unstructured":"F. Chesani, A. Ciampolini, D. Loreti, and P. Mello, \u201cMapReduce autoscaling over the cloud with process mining monitoring,\u201d International Conference on Cloud Computing and Services Science 2016 Apr 23, Cham, Springer, 2017, pp. 109\u201330. 10.1007\/978-3-319-62594-2_6.","DOI":"10.1007\/978-3-319-62594-2_6"},{"key":"2022081707553241786_j_comp-2022-0246_ref_006","doi-asserted-by":"crossref","unstructured":"K. Chen, J. Powers, S. Guo, and F. Tian, \u201cCRESP: Towards optimal resource provisioning for MapReduce computing in public clouds. parallel and distributed systems,\u201d IEEE Tran. on, vol. 25, pp. 1403\u201312, 2014. 10.1109\/TPDS.2013.297.","DOI":"10.1109\/TPDS.2013.297"},{"key":"2022081707553241786_j_comp-2022-0246_ref_007","doi-asserted-by":"crossref","unstructured":"V. Muthumanikandan and C. Valliyammai, \u201cLink failure recovery using shortest path fast rerouting technique in SDN,\u201d Wireless Pers. Commun., vol. 97, pp. 2475\u201395, 2017. 10.1007\/s11277-017-4618-0.","DOI":"10.1007\/s11277-017-4618-0"},{"key":"2022081707553241786_j_comp-2022-0246_ref_008","doi-asserted-by":"crossref","unstructured":"S. Plimpton and K. Devine, \u201cMapReduce in MPI for large-scale graph algorithms,\u201d Parallel Comp., vol. 37, no. 610, pp. 632, 2011. 10.1016\/j.parco.2011.02.004.","DOI":"10.1016\/j.parco.2011.02.004"},{"key":"2022081707553241786_j_comp-2022-0246_ref_009","doi-asserted-by":"crossref","unstructured":"T. Gao, Y. Guo, B. Zhang, P. Cicotti, Y. Lu, P. Balaji, et al., \u201cMimir: memory-efficient and scalable MapReduce for large supercomputing systems,\u201d 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2017. 10.1109\/ipdps.2017.31.","DOI":"10.1109\/IPDPS.2017.31"},{"key":"2022081707553241786_j_comp-2022-0246_ref_010","unstructured":"\u201cBlaze: Simplified High Performance Cluster Computing by Junhao Li, Hang Zhang.\u201d arXiv:1902.01437 [cs.DC]."},{"key":"2022081707553241786_j_comp-2022-0246_ref_011","doi-asserted-by":"crossref","unstructured":"Z. Fadika, E. Dede, M. Govindaraju, and L. Ramakrishnan, \u201cMARIANE: MapReduce implementation adapted for HPC environments,\u201d 2011 IEEE\/ACM 12th International Conference on Grid Computing, Lyon, 2011, pp. 82\u20139.","DOI":"10.1109\/Grid.2011.20"},{"key":"2022081707553241786_j_comp-2022-0246_ref_012","doi-asserted-by":"crossref","unstructured":"S. Kang, S. Lee, and K. M. Lee, \u201cPerformance comparison of OpenMP, MPI, and MapReduce in practical problems,\u201d Adv. Multimedia, vol. 2015, pp. 1\u20139, 2015. 10.1155\/2015\/575687.","DOI":"10.1155\/2015\/575687"},{"key":"2022081707553241786_j_comp-2022-0246_ref_013","unstructured":"\u201cMicrosoft Research. \u201c[Online]. http:\/\/www.microsoft.com\/windows azure\/."},{"key":"2022081707553241786_j_comp-2022-0246_ref_014","unstructured":"Amazon, \u201cAmazon Elastic Compute Cloud.\u201d [Online]. http:\/\/aws.amazon.com\/ec2."},{"key":"2022081707553241786_j_comp-2022-0246_ref_015","doi-asserted-by":"crossref","unstructured":"V. Muthumanikandan, P. Singh, R. Chithreddy, \u201cCloud-based face and face mask detection system. In: Intelligent Systems and Sustainable Computing. Smart Innovation, Systems and Technologies, V. S. Reddy, V. K. Prasad, D. N. Mallikarjuna Rao, S. C. Satapathy, Eds, vol. 289. Springer, Singapore. 2022. 10.1007\/978-981-19-0011-2_38.","DOI":"10.1007\/978-981-19-0011-2_38"},{"key":"2022081707553241786_j_comp-2022-0246_ref_016","doi-asserted-by":"crossref","unstructured":"Y. Wang, G. Agrawal, T. Bicer, and W. Jiang, \u201cSmart: A MapReduce-like framework for in-situ scientific analytics,\u201d Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2015.","DOI":"10.1145\/2807591.2807650"},{"key":"2022081707553241786_j_comp-2022-0246_ref_017","doi-asserted-by":"crossref","unstructured":"N. Hu, Z. Chen, Y. Du, and Y. Lu, \u201cMimir+: An optimized framework of MapReduce on heterogeneous high-performance computing system,\u201d Network and Parallel Computing. NPC 2018. Lecture Notes in Computer Science, F. Zhang, J. Zhai, M. Snir, H. Jin, H. Kasahara, M. Valero, (eds), vol. 11276, Cham, Springer, 2018.","DOI":"10.1007\/978-3-030-05677-3_18"},{"key":"2022081707553241786_j_comp-2022-0246_ref_018","doi-asserted-by":"crossref","unstructured":"N. Nguyen and D. Bein, \u201cDistributed MPI cluster with Docker Swarm mode,\u201d 2017 IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA, 2017, pp. 1\u20137.","DOI":"10.1109\/CCWC.2017.7868429"},{"key":"2022081707553241786_j_comp-2022-0246_ref_019","doi-asserted-by":"crossref","unstructured":"K. Doucet and J. Zhang, \u201cThe creation of a low-cost Raspberry pi cluster for teaching,\u201d WCCCE \u201819: Proceedings of the Western Canadian Conference on Computing Education, 2019, pp. 1\u20135. 10.1145\/3314994.3325088.","DOI":"10.1145\/3314994.3325088"},{"key":"2022081707553241786_j_comp-2022-0246_ref_020","doi-asserted-by":"crossref","unstructured":"W. Zhao, H. Ma, and Q. He, Parallel k-means clustering based on MapReduce, Cloud Computing. CloudCom 2009. Lecture Notes in Computer Science, M. G. Jaatun, G. Zhao, C. Rong, (eds), vol. 5931, Berlin, Heidelberg, Springer, 2009.","DOI":"10.1007\/978-3-642-10665-1_71"},{"key":"2022081707553241786_j_comp-2022-0246_ref_021","doi-asserted-by":"crossref","unstructured":"Z. Fadika and M. Govindaraju, \u201cDelma: Dynamically elastic mapreduce framework for CPU-intensive applications,\u201d CCGRID, 2011, pp. 454\u201363.","DOI":"10.1109\/CCGrid.2011.71"},{"key":"2022081707553241786_j_comp-2022-0246_ref_022","unstructured":"\u201cAlpine Linux with MPICH for developing and deploying distributed MPI programs,\u201d Accessed on: Jan, 2020 [Online]. https:\/\/hub.docker.com\/r\/nlknguyen\/alpine-mpich."},{"key":"2022081707553241786_j_comp-2022-0246_ref_023","unstructured":"\u201cHigh Performance MapReduce-First Cluster Computing,\u201d Junhao12131. Accessed on: Feb 2020. https:\/\/github.com\/junhao12131\/blaze."}],"container-title":["Open Computer Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.degruyter.com\/document\/doi\/10.1515\/comp-2022-0246\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.degruyter.com\/document\/doi\/10.1515\/comp-2022-0246\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,17]],"date-time":"2022-08-17T08:01:20Z","timestamp":1660723280000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.degruyter.com\/document\/doi\/10.1515\/comp-2022-0246\/html"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,1]]},"references-count":23,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,3,16]]},"published-print":{"date-parts":[[2022,3,16]]}},"alternative-id":["10.1515\/comp-2022-0246"],"URL":"https:\/\/doi.org\/10.1515\/comp-2022-0246","relation":{},"ISSN":["2299-1093"],"issn-type":[{"value":"2299-1093","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,1]]}}}