{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T04:55:35Z","timestamp":1755838535665},"reference-count":31,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2016,8]]},"abstract":"<jats:p>\n            Nowadays, there is a rising trend of building array-based distributed computing frameworks, which are suitable for implementing many machine learning and data mining algorithms. However, most of these frameworks only execute each primitive in an isolated manner and in the exact order defined by programmers, which implies a huge space for optimization. In this paper, we propose a novel array-based programming model, named K\n            <jats:sc>asen<\/jats:sc>\n            , which distinguishes itself from models in the existing literature by defining a strict computation and communication model. This model makes it easy to analyze programs' behavior and measure their performance, with which we design a corresponding optimizer that can automatically apply high-level optimizations to the original programs written by programmers. According to our evaluation, the optimizer of K\n            <jats:sc>asen<\/jats:sc>\n            can achieve a significant reduction on memory read\/write, buffer allocation and network traffic, which leads to a speedup up to 5.82x.\n          <\/jats:p>","DOI":"10.14778\/2994509.2994511","type":"journal-article","created":{"date-parts":[[2016,9,6]],"date-time":"2016-09-06T15:27:03Z","timestamp":1473175623000},"page":"912-923","source":"Crossref","is-referenced-by-count":11,"title":["Measuring and optimizing distributed array programs"],"prefix":"10.14778","volume":"9","author":[{"given":"Mingxing","family":"Zhang","sequence":"first","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yongwei","family":"Wu","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kang","family":"Chen","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Teng","family":"Ma","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weimin","family":"Zheng","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2016,8]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"http:\/\/snap.stanford.edu\/data.  http:\/\/snap.stanford.edu\/data."},{"key":"e_1_2_1_2_1","volume-title":"et al. Tensorflow: Large-scale machine learning on heterogeneous systems","author":"Abadi M.","year":"2015","unstructured":"M. Abadi , A. Agarwal , P. Barham , E. Brevdo , Z. Chen , C. Citro , G. S. Corrado , A. Davis , J. Dean , M. Devin , et al. Tensorflow: Large-scale machine learning on heterogeneous systems , 2015 . Software available from tensorflow. org. M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, et al. Tensorflow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow. org."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1038\/43601"},{"key":"e_1_2_1_4_1","first-page":"18","article-title":"The use and limitations of static-analysis tools to improve software quality","volume":"21","author":"Anderson P.","year":"2008","unstructured":"P. Anderson . The use and limitations of static-analysis tools to improve software quality . CrossTalk: The Journal of Defense Software Engineering , 21 : 18 -- 21 , 2008 . P. Anderson. The use and limitations of static-analysis tools to improve software quality. CrossTalk: The Journal of Defense Software Engineering, 21:18--21, 2008.","journal-title":"CrossTalk: The Journal of Defense Software Engineering"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1654059.1654119"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1248377.1248391"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2523616.2525965"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807167.1807271"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1177\/1094342011403516"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2741948.2741970"},{"key":"e_1_2_1_11_1","first-page":"1332","volume-title":"NIPS '14","author":"Chen W.","unstructured":"W. Chen , Z. Wang , and J. Zhou . Large-scale L-BFGS using MapReduce . NIPS '14 , pages 1332 -- 1340 . W. Chen, Z. Wang, and J. Zhou. Large-scale L-BFGS using MapReduce. NIPS '14, pages 1332--1340."},{"key":"e_1_2_1_12_1","first-page":"269","volume-title":"LCPC '95","author":"De Rose L.","unstructured":"L. De Rose , K. Gallivan , E. Gallopoulos , B. Marsolf , and D. Padua . FALCON: A MATLAB interactive restructuring compiler . LCPC '95 , pages 269 -- 288 . L. De Rose, K. Gallivan, E. Gallopoulos, B. Marsolf, and D. Padua. FALCON: A MATLAB interactive restructuring compiler. LCPC '95, pages 269--288."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.5555\/898758"},{"key":"e_1_2_1_14_1","first-page":"17","volume-title":"PowerGraph: Distributed Graph-parallel Computation on Natural Graphs. OSDI'12","author":"Gonzalez J. E.","unstructured":"J. E. Gonzalez , Y. Low , H. Gu , D. Bickson , and C. Guestrin . PowerGraph: Distributed Graph-parallel Computation on Natural Graphs. OSDI'12 , pages 17 -- 30 . J. E. Gonzalez, Y. Low, H. Gu, D. Bickson, and C. Guestrin. PowerGraph: Distributed Graph-parallel Computation on Natural Graphs. OSDI'12, pages 17--30."},{"key":"e_1_2_1_15_1","first-page":"1","volume-title":"USENIX ATC '15","author":"Huang C.-C.","unstructured":"C.-C. Huang , Q. Chen , Z. Wang , R. Power , J. Ortiz , J. Li , and Z. Xiao . Spartan: A distributed array framework with smart tiling . USENIX ATC '15 , pages 1 -- 15 . C.-C. Huang, Q. Chen, Z. Wang, R. Power, J. Ortiz, J. Li, and Z. Xiao. Spartan: A distributed array framework with smart tiling. USENIX ATC '15, pages 1--15."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/503272.503298"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772756"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1080\/15427951.2009.10129177"},{"key":"e_1_2_1_19_1","first-page":"503","volume-title":"On the limited memory BFGS method for large scale optimization. Mathematical programming","author":"Liu D. C.","year":"1989","unstructured":"D. C. Liu and J. Nocedal . On the limited memory BFGS method for large scale optimization. Mathematical programming , pages 503 -- 528 , 1989 . D. C. Liu and J. Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical programming, pages 503--528, 1989."},{"key":"e_1_2_1_20_1","first-page":"930","volume-title":"SDM '12","author":"Lugowski A.","unstructured":"A. Lugowski , D. Alber , A. Bulu\u00e7 , J. R. Gilbert , S. Reinhardt , Y. Teng , and A. Waranis . A flexible open-source toolbox for scalable complex graph analysis . SDM '12 , pages 930 -- 941 . A. Lugowski, D. Alber, A. Bulu\u00e7, J. R. Gilbert, S. Reinhardt, Y. Teng, and A. Waranis. A flexible open-source toolbox for scalable complex graph analysis. SDM '12, pages 930--941."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/305138.305230"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.5555\/243179.243182"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2427023.2427030"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2168836.2168857"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2588555.2610518"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CloudCom.2010.17"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/79173.79181"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2465351.2465371"},{"key":"e_1_2_1_29_1","volume-title":"Numerical optimization","author":"Wright S. J.","year":"1999","unstructured":"S. J. Wright and J. Nocedal . Numerical optimization , volume 2 . Springer New York , 1999 . S. J. Wright and J. Nocedal. Numerical optimization, volume 2. Springer New York, 1999."},{"key":"e_1_2_1_30_1","first-page":"2","volume-title":"Resilient Distributed Datasets: A Fault-tolerant Abstraction for In-memory Cluster Computing. NSDI'12","author":"Zaharia M.","unstructured":"M. Zaharia , M. Chowdhury , T. Das , A. Dave , J. Ma , M. McCauley , M. J. Franklin , S. Shenker , and I. Stoica . Resilient Distributed Datasets: A Fault-tolerant Abstraction for In-memory Cluster Computing. NSDI'12 , pages 2 -- 2 . M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley, M. J. Franklin, S. Shenker, and I. Stoica. Resilient Distributed Datasets: A Fault-tolerant Abstraction for In-memory Cluster Computing. NSDI'12, pages 2--2."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.14778\/2212351.2212358"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/2994509.2994511","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T10:51:29Z","timestamp":1672224689000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/2994509.2994511"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,8]]},"references-count":31,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2016,8]]}},"alternative-id":["10.14778\/2994509.2994511"],"URL":"https:\/\/doi.org\/10.14778\/2994509.2994511","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2016,8]]}}}