{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,5]],"date-time":"2026-09-05T01:10:19Z","timestamp":1788570619239,"version":"build-2803163510"},"reference-count":33,"publisher":"Association for Computing Machinery (ACM)","issue":"7","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2014,3]]},"abstract":"<jats:p>SystemML aims at declarative, large-scale machine learning (ML) on top of MapReduce, where high-level ML scripts with R-like syntax are compiled to programs of MR jobs. The declarative specification of ML algorithms enables---in contrast to existing large-scale machine learning libraries---automatic optimization. SystemML's primary focus is on data parallelism but many ML algorithms inherently exhibit opportunities for task parallelism as well. A major challenge is how to efficiently combine both types of parallelism for arbitrary ML scripts and workloads. In this paper, we present a systematic approach for combining task and data parallelism for large-scale machine learning on top of MapReduce. We employ a generic Parallel FOR construct (ParFOR) as known from high performance computing (HPC). Our core contributions are (1) complementary parallelization strategies for exploiting multi-core and cluster parallelism, as well as (2) a novel cost-based optimization framework for automatically creating optimal parallel execution plans. Experiments on a variety of use cases showed that this achieves both efficiency and scalability due to automatic adaptation to ad-hoc workloads and unknown data characteristics.<\/jats:p>","DOI":"10.14778\/2732286.2732292","type":"journal-article","created":{"date-parts":[[2015,5,12]],"date-time":"2015-05-12T15:37:52Z","timestamp":1431445072000},"page":"553-564","source":"Crossref","is-referenced-by-count":68,"title":["Hybrid parallelization strategies for large-scale machine learning in SystemML"],"prefix":"10.14778","volume":"7","author":[{"given":"Matthias","family":"Boehm","sequence":"first","affiliation":[{"name":"IBM Research, Almaden, San Jose, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shirish","family":"Tatikonda","sequence":"additional","affiliation":[{"name":"IBM Research, Almaden, San Jose, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Berthold","family":"Reinwald","sequence":"additional","affiliation":[{"name":"IBM Research, Almaden, San Jose, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Prithviraj","family":"Sen","sequence":"additional","affiliation":[{"name":"IBM Research, Almaden, San Jose, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuanyuan","family":"Tian","sequence":"additional","affiliation":[{"name":"IBM Research, Almaden, San Jose, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Douglas R.","family":"Burdick","sequence":"additional","affiliation":[{"name":"IBM Research, Almaden, San Jose, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shivakumar","family":"Vaithyanathan","sequence":"additional","affiliation":[{"name":"IBM Research, Almaden, San Jose, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2014,3]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"NSDI","author":"Agarwal S.","year":"2012","unstructured":"S. Agarwal , S. Kandula , N. Bruno , M.-C. Wu , I. Stoica , and J. Zhou . Re-optimizing Data-Parallel Computing . In NSDI , 2012 . S. Agarwal, S. Kandula, N. Bruno, M.-C. Wu, I. Stoica, and J. Zhou. Re-optimizing Data-Parallel Computing. In NSDI, 2012."},{"key":"e_1_2_1_2_1","unstructured":"Apache. Mahout. mahout.apache.org.  Apache. Mahout. mahout.apache.org."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.14778\/3402755.3402761"},{"issue":"2","key":"e_1_2_1_4_1","first-page":"24","volume":"35","author":"Borkar V. R.","year":"2012","unstructured":"V. R. Borkar , Y. Bu , M. J. Carey , J. Rosen , N. Polyzotis , T. Condie , M. Weimer , and R. Ramakrishnan . Declarative Systems for Large-Scale Machine Learning. IEEE Data Eng. Bull. , 35 ( 2 ): 24 -- 32 , 2012 . V. R. Borkar, Y. Bu, M. J. Carey, J. Rosen, N. Polyzotis, T. Condie, M. Weimer, and R. Ramakrishnan. Declarative Systems for Large-Scale Machine Learning. IEEE Data Eng. Bull., 35(2):24--32, 2012.","journal-title":"Declarative Systems for Large-Scale Machine Learning. IEEE Data Eng. Bull."},{"key":"e_1_2_1_5_1","volume-title":"NIPS","author":"Chu C.-T.","year":"2006","unstructured":"C.-T. Chu , S. K. Kim , Y.-A. Lin , Y. Yu , G. R. Bradski , A. Y. Ng , and K. Olukotun . Map-Reduce for Machine Learning on Multicore . In NIPS , 2006 . C.-T. Chu, S. K. Kim, Y.-A. Lin, Y. Yu, G. R. Bradski, A. Y. Ng, and K. Olukotun. Map-Reduce for Machine Learning on Multicore. In NIPS, 2006."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687553.1687576"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807167.1807275"},{"key":"e_1_2_1_8_1","volume-title":"OSDI","author":"Dean J.","year":"2004","unstructured":"J. Dean and S. Ghemawat . MapReduce: Simplified Data Processing on Large Clusters . In OSDI , 2004 . J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In OSDI, 2004."},{"key":"e_1_2_1_9_1","volume-title":"CRAN Task View: High-Performance and Parallel Computing with R. R Project","author":"Eddelbuettel Dirk","year":"2013","unstructured":"Dirk Eddelbuettel . CRAN Task View: High-Performance and Parallel Computing with R. R Project , 2013 . cran.r-project.org\/web\/views\/HighPerformanceComputing.html. Dirk Eddelbuettel. CRAN Task View: High-Performance and Parallel Computing with R. R Project, 2013. cran.r-project.org\/web\/views\/HighPerformanceComputing.html."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2011.5767930"},{"key":"e_1_2_1_12_1","volume-title":"NIPS","author":"Graf H. P.","year":"2004","unstructured":"H. P. Graf , E. Cosatto , L. Bottou , I. Durdanovic , and V. Vapnik . Parallel Support Vector Machines: The Cascade SVM . In NIPS , 2004 . H. P. Graf, E. Cosatto, L. Bottou, I. Durdanovic, and V. Vapnik. Parallel Support Vector Machines: The Cascade SVM. In NIPS, 2004."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.14778\/3402707.3402746"},{"key":"e_1_2_1_14_1","volume-title":"CIDR","author":"Herodotou H.","year":"2011","unstructured":"H. Herodotou , H. Lim , G. Luo , N. Borisov , L. Dong , F. B. Cetin , and S. Babu . Starfish: A Self-tuning System for Big Data Analytics . In CIDR , 2011 . H. Herodotou, H. Lim, G. Luo, N. Borisov, L. Dong, F. B. Cetin, and S. Babu. Starfish: A Self-tuning System for Big Data Analytics. In CIDR, 2011."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.5555\/382009.383686"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2465273"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.14778\/2350229.2350244"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/125826.126137"},{"key":"e_1_2_1_19_1","volume-title":"Optimizing Compilers for Modern Architectures: A Dependence-Based Approach","author":"Kennedy K.","year":"2002","unstructured":"K. Kennedy and J. R. Allen . Optimizing Compilers for Modern Architectures: A Dependence-Based Approach . Morgan Kaufmann Publishers Inc ., 2002 . K. Kennedy and J. R. Allen. Optimizing Compilers for Modern Architectures: A Dependence-Based Approach. Morgan Kaufmann Publishers Inc., 2002."},{"key":"e_1_2_1_20_1","volume-title":"CIDR","author":"Kraska T.","year":"2013","unstructured":"T. Kraska , A. Talwalkar , J. Duchi , R. Griffith , M. J. Franklin , and M. Jordan . MLbase: A Distributed Machine-learning System . In CIDR , 2013 . T. Kraska, A. Talwalkar, J. Duchi, R. Griffith, M. J. Franklin, and M. Jordan. MLbase: A Distributed Machine-learning System. In CIDR, 2013."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.14778\/2350229.2350239"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.5555\/1390681.1390703"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2213836.2213958"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.14778\/2212351.2212354"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920906"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1376616.1376726"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/TC.1987.5009495"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10766-008-0082-5"},{"key":"e_1_2_1_29_1","unstructured":"The MADlib Analytics Library. madlib.net.  The MADlib Analytics Library. madlib.net."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687553.1687609"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2012.12"},{"key":"e_1_2_1_32_1","volume-title":"NSDI","author":"Zaharia M.","year":"2012","unstructured":"M. Zaharia , M. Chowdhury , T. Das , A. Dave , J. Ma , M. McCauley , M. J. Franklin , S. Shenker , and I. Stoica . Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing . In NSDI , 2012 . M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley, M. J. Franklin, S. Shenker, and I. Stoica. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In NSDI, 2012."},{"key":"e_1_2_1_33_1","volume-title":"CIDR","author":"Zhang Y.","year":"2009","unstructured":"Y. Zhang , H. Herodotou , and J. Yang . RIOT: I\/O-Efficient Numerical Computing without SQL . In CIDR , 2009 . Y. Zhang, H. Herodotou, and J. Yang. RIOT: I\/O-Efficient Numerical Computing without SQL. In CIDR, 2009."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.14778\/2212351.2212358"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/2732286.2732292","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T11:01:01Z","timestamp":1672225261000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/2732286.2732292"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,3]]},"references-count":33,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2014,3]]}},"alternative-id":["10.14778\/2732286.2732292"],"URL":"https:\/\/doi.org\/10.14778\/2732286.2732292","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2014,3]]}}}