{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T12:12:09Z","timestamp":1763467929135,"version":"3.41.0"},"reference-count":36,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2009,4,21]],"date-time":"2009-04-21T00:00:00Z","timestamp":1240272000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGOPS Oper. Syst. Rev."],"published-print":{"date-parts":[[2009,4,21]]},"abstract":"<jats:p>Asymmetric multi-core processors (AMPs) with general-purpose and specialized cores packaged on the same chip, are emerging as a leading paradigm for high-end computing. A large body of existing research explores the use of standalone AMPs in computationally challenging and data-intensive applications. AMPs are rapidly deployed as high-performance accelerators on clusters. In these settings, scheduling, communication and I\/O are managed by generalpurpose processors (GPPs), while computation is off-loaded to AMPs. Design space exploration for the configuration and software stack of hybrid clusters of AMPs and GPPs is an open problem. In this paper, we explore this design space in an implementation of the popular MapReduce programming model. Our contributions are: An exploration of various design alternatives for hybrid asymmetric clusters of AMPs and GPPs; the adoption of a streaming approach to supporting MapReduce computations on clusters with asymmetric components; and adaptive schedulers that take into account individual component capabilities in asymmetric clusters. Throughout our design, we remove I\/O bottlenecks, using double-buffering and asynchronous I\/O. We present an evaluation of the design choices through experiments on a real cluster with MapReduce workloads of varying degrees of computation intensity. We find that in a cluster with resource-constrained and well-provisioned AMP accelerators, a streaming approach achieves 50.5% and 73.1% better performance compared to the non-streaming approach, respectively, and scales almost linearly with increasing number of compute nodes.We also show that our dynamic scheduling mechanisms adapt effectively the parameters of the scheduling policies between applications with different computation density.<\/jats:p>","DOI":"10.1145\/1531793.1531800","type":"journal-article","created":{"date-parts":[[2009,4,28]],"date-time":"2009-04-28T14:58:07Z","timestamp":1240930687000},"page":"25-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":37,"title":["Supporting MapReduce on large-scale asymmetric multi-core clusters"],"prefix":"10.1145","volume":"43","author":[{"given":"M. Mustafa","family":"Rafique","sequence":"first","affiliation":[{"name":"Virginia Tech, Blacksburg, Virginia, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Benjamin","family":"Rose","sequence":"additional","affiliation":[{"name":"Virginia Tech, Blacksburg, Virginia, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ali R.","family":"Butt","sequence":"additional","affiliation":[{"name":"Virginia Tech, Blacksburg, Virginia, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dimitrios S.","family":"Nikolopoulos","sequence":"additional","affiliation":[{"name":"FORTH, Heraklion, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2009,4,21]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Astrophysicist Replaces Supercomputer with Eight PlayStation 3s. http:\/\/www.wired.com\/techbiz\/it\/news\/2007\/10\/ps3_supercomputer. Astrophysicist Replaces Supercomputer with Eight PlayStation 3s. http:\/\/www.wired.com\/techbiz\/it\/news\/2007\/10\/ps3_supercomputer.","DOI":"10.1063\/pt.5.021583"},{"key":"e_1_2_1_2_1","unstructured":"Adam Pisoni. Skynet Apr. 2008. http:\/\/skynet.rubyforge.org. Adam Pisoni. Skynet Apr. 2008. http:\/\/skynet.rubyforge.org."},{"key":"e_1_2_1_3_1","unstructured":"Amazon. Amazon Elastic Compute Cloud (EC2). http:\/\/www.amazon.com\/b?ie=UTF8&node=201590011. Amazon. Amazon Elastic Compute Cloud (EC2). http:\/\/www.amazon.com\/b?ie=UTF8&node=201590011."},{"key":"e_1_2_1_4_1","unstructured":"AMD. The Industry-Changing Impact of Accelerated Computing. 2008. AMD. The Industry-Changing Impact of Accelerated Computing. 2008."},{"key":"e_1_2_1_5_1","unstructured":"Apache Software Foundation. Hadoop May 2007. http:\/\/hadoop.apache.org\/core\/. Apache Software Foundation. Hadoop May 2007. http:\/\/hadoop.apache.org\/core\/."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.5555\/1782174.1782198"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2005.51"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/MM.2003.1196112"},{"key":"e_1_2_1_9_1","volume-title":"Proc. IPDPS.","author":"Blagojevic F.","year":"2007","unstructured":"F. Blagojevic , A. Stamatakis , C. Antonopoulos , and D. Nikolopoulos . RAxML-CELL: Parallel Phylogenetic Tree Construction on the Cell Broadband Engine . In Proc. IPDPS. , 2007 . F. Blagojevic, A. Stamatakis, C. Antonopoulos, and D. Nikolopoulos. RAxML-CELL: Parallel Phylogenetic Tree Construction on the Cell Broadband Engine. In Proc. IPDPS., 2007."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1366230.1366234"},{"key":"e_1_2_1_13_1","volume-title":"Proc. USENIX OSDI.","author":"Dean J.","year":"2004","unstructured":"J. Dean and S. Ghemawat . MapReduce: Simplified Data Processing on Large Clusters . In Proc. USENIX OSDI. , 2004 . J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In Proc. USENIX OSDI., 2004."},{"key":"e_1_2_1_14_1","volume-title":"Proc. VLDB.","author":"Gedik B.","year":"2007","unstructured":"B. Gedik , R. Bordawekar , and P.S. Yu . CellSort: High Performance Sorting on the Cell Processor . In Proc. VLDB. , 2007 . B. Gedik, R. Bordawekar, and P.S. Yu. CellSort: High Performance Sorting on the Cell Processor. In Proc. VLDB., 2007."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.parco.2007.09.002"},{"key":"e_1_2_1_16_1","unstructured":"GraphStream Inc. GraphStream Scalable Computing Platform (SCP). 2006. http:\/\/www.graphstream.com. GraphStream Inc. GraphStream Scalable Computing Platform (SCP). 2006. http:\/\/www.graphstream.com."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1454115.1454152"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1363189.1363195"},{"key":"e_1_2_1_19_1","volume-title":"Cell Broadband Engine Architecture (Version 1.02)","author":"IBM Corp.","year":"2007","unstructured":"IBM Corp. Cell Broadband Engine Architecture (Version 1.02) . 2007 . IBM Corp. Cell Broadband Engine Architecture (Version 1.02). 2007."},{"key":"e_1_2_1_20_1","volume-title":"Oct","author":"Cross Jason","year":"2007","unstructured":"Jason Cross . A Dramatic Leap Forward-GeForce 8800 GT , Oct 2007 . http:\/\/www.extremetech.com\/article2\/0,1697, 2209197,00.asp. Jason Cross. A Dramatic Leap Forward-GeForce 8800 GT, Oct 2007. http:\/\/www.extremetech.com\/article2\/0,1697, 2209197,00.asp."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/1148882.1148891"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCSE.2008.85"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1346281.1346318"},{"issue":"1","key":"e_1_2_1_25_1","first-page":"299","article-title":"A Message Passing Interface Standard","volume":"12","author":"Interface Forum Message Passing","year":"1998","unstructured":"Message Passing Interface Forum . MPI2 : A Message Passing Interface Standard . Int. J. of High Performance Computing Applications , 12 ( 1-2 ): 299 , 1998 . Message Passing Interface Forum. MPI2: A Message Passing Interface Standard. Int. J. of High Performance Computing Applications, 12(1-2):299, 1998.","journal-title":"Int. J. of High Performance Computing Applications"},{"key":"e_1_2_1_26_1","unstructured":"Mueller. NC State Engineer Creates First Academic Playstation 3 Computing Cluster. http:\/\/moss.csc.ncsu.edu\/~mueller\/cluster\/ps3\/coe.html. Mueller. NC State Engineer Creates First Academic Playstation 3 Computing Cluster. http:\/\/moss.csc.ncsu.edu\/~mueller\/cluster\/ps3\/coe.html."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1147\/rd.515.0573"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/1299042.1299107"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2007.370252"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.5555\/998680.1006707"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1366230.1366236"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2007.346181"},{"key":"e_1_2_1_33_1","volume-title":"Proc. Summer USENIX","author":"Sandberg R.","year":"1985","unstructured":"R. Sandberg , D. Goldberg , S. Kleiman , D. Walsh , and B. Lyon . Design and Implementation of the Sun Network Filesystem . In Proc. Summer USENIX , 1985 . R. Sandberg, D. Goldberg, S. Kleiman, D.Walsh, and B. Lyon. Design and Implementation of the Sun Network Filesystem. In Proc. Summer USENIX, 1985."},{"key":"e_1_2_1_34_1","volume-title":"Proc. Ottawa Linux Symposium","author":"Schwan P.","year":"2003","unstructured":"P. Schwan . Lustre : Building a File System for 1,000-node Clusters . In Proc. Ottawa Linux Symposium , 2003 . P. Schwan. Lustre: Building a File System for 1,000-node Clusters. In Proc. Ottawa Linux Symposium, 2003."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1147\/rd.515.0559"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.v17:2\/4"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.5555\/1413370.1413372"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1454115.1454125"},{"key":"e_1_2_1_39_1","volume-title":"Proc. USENIX OSDI.","author":"Zaharia M.","year":"2008","unstructured":"M. Zaharia , A. Konwinski , and A.D. Joseph . Improving MapReduce Performance in Heterogeneous Environments . In Proc. USENIX OSDI. , 2008 . M. Zaharia, A. Konwinski, and A.D. Joseph. Improving MapReduce Performance in Heterogeneous Environments. In Proc. USENIX OSDI., 2008."}],"container-title":["ACM SIGOPS Operating Systems Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1531793.1531800","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1531793.1531800","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T13:29:28Z","timestamp":1750253368000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1531793.1531800"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2009,4,21]]},"references-count":36,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2009,4,21]]}},"alternative-id":["10.1145\/1531793.1531800"],"URL":"https:\/\/doi.org\/10.1145\/1531793.1531800","relation":{},"ISSN":["0163-5980"],"issn-type":[{"type":"print","value":"0163-5980"}],"subject":[],"published":{"date-parts":[[2009,4,21]]},"assertion":[{"value":"2009-04-21","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}