{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T15:47:52Z","timestamp":1778860072141,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":61,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,10,2]],"date-time":"2021-10-02T00:00:00Z","timestamp":1633132800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF (National Science Foundation)","doi-asserted-by":"publisher","award":["CNS grants 1750109, 1730128, and 1717588"],"award-info":[{"award-number":["CNS grants 1750109, 1730128, and 1717588"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,11,22]]},"DOI":"10.1145\/3464298.3476132","type":"proceedings-article","created":{"date-parts":[[2021,10,3]],"date-time":"2021-10-03T06:07:43Z","timestamp":1633241263000},"page":"39-51","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Towards optimal placement and scheduling of DNN operations with Pesto"],"prefix":"10.1145","author":[{"given":"Ubaid Ullah","family":"Hafeez","sequence":"first","affiliation":[{"name":"Stony Brook University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiao","family":"Sun","sequence":"additional","affiliation":[{"name":"Stony Brook University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anshul","family":"Gandhi","sequence":"additional","affiliation":[{"name":"Stony Brook University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhenhua","family":"Liu","sequence":"additional","affiliation":[{"name":"Stony Brook University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,10,2]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"[n.d.]. Amazon EC2 Instance Types. https:\/\/aws.amazon.com\/ec2\/instance-types. Accessed: 2020-11-28. [n.d.]. Amazon EC2 Instance Types. https:\/\/aws.amazon.com\/ec2\/instance-types. Accessed: 2020-11-28."},{"key":"e_1_3_2_1_2_1","volume-title":"Baechi: Fase Device Placement on Machine Learning Graphs (SoCC","year":"2020","unstructured":"[n.d.]. Baechi: Fase Device Placement on Machine Learning Graphs (SoCC 2020 ). https:\/\/github.com\/beomyeol\/baechi. Accessed : 2021-03-28. [n.d.]. Baechi: Fase Device Placement on Machine Learning Graphs (SoCC 2020). https:\/\/github.com\/beomyeol\/baechi. Accessed: 2021-03-28."},{"key":"e_1_3_2_1_3_1","unstructured":"[n.d.]. NVIDIA V100 TENSOR CORE GPUs. https:\/\/www.nvidia.com\/en-us\/data-center\/v100\/. Accessed: 2020-11-28. [n.d.]. NVIDIA V100 TENSOR CORE GPUs. https:\/\/www.nvidia.com\/en-us\/data-center\/v100\/. Accessed: 2020-11-28."},{"key":"e_1_3_2_1_4_1","unstructured":"[n.d.]. NVLINK FABRIC: A FASTER MORE SCALABLE INTERCONNECT. https:\/\/www.nvidia.com\/en-us\/data-center\/nvlink\/. [n.d.]. NVLINK FABRIC: A FASTER MORE SCALABLE INTERCONNECT. https:\/\/www.nvidia.com\/en-us\/data-center\/nvlink\/."},{"key":"e_1_3_2_1_5_1","unstructured":"[n.d.]. Pesto Source-code. https:\/\/github.com\/PACELab\/TF-Pesto. Accessed: 2020-11-28. [n.d.]. Pesto Source-code. https:\/\/github.com\/PACELab\/TF-Pesto. Accessed: 2020-11-28."},{"key":"e_1_3_2_1_6_1","unstructured":"[n.d.]. TensorFlow NMT GitHub. https:\/\/github.com\/tensorflow\/nmt. Accessed: 2020-11-28. [n.d.]. TensorFlow NMT GitHub. https:\/\/github.com\/tensorflow\/nmt. Accessed: 2020-11-28."},{"key":"e_1_3_2_1_7_1","volume-title":"WMT 2016","year":"2020","unstructured":"[n.d.]. WMT 2016 . http:\/\/www.statmt.org\/wmt16\/. Accessed : 2020 -11-28. [n.d.]. WMT 2016. http:\/\/www.statmt.org\/wmt16\/. Accessed: 2020-11-28."},{"key":"e_1_3_2_1_8_1","unstructured":"Martín Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado Andy Davis Jeffrey Dean Matthieu Devin Sanjay Ghemawat Ian Goodfellow Andrew Harp Geoffrey Irving Michael Isard Yangqing Jia Rafal Jozefowicz Lukasz Kaiser Manjunath Kudlur Josh Levenberg Dan Mané Rajat Monga Sherry Moore Derek Murray Chris Olah Mike Schuster Jonathon Shlens Benoit Steiner Ilya Sutskever Kunal Talwar Paul Tucker Vincent Vanhoucke Vijay Vasudevan Fernanda Viégas Oriol Vinyals Pete Warden Martin Wattenberg Martin Wicke Yuan Yu and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. http:\/\/tensorflow.org\/ Software available from tensorflow.org. Martín Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado Andy Davis Jeffrey Dean Matthieu Devin Sanjay Ghemawat Ian Goodfellow Andrew Harp Geoffrey Irving Michael Isard Yangqing Jia Rafal Jozefowicz Lukasz Kaiser Manjunath Kudlur Josh Levenberg Dan Mané Rajat Monga Sherry Moore Derek Murray Chris Olah Mike Schuster Jonathon Shlens Benoit Steiner Ilya Sutskever Kunal Talwar Paul Tucker Vincent Vanhoucke Vijay Vasudevan Fernanda Viégas Oriol Vinyals Pete Warden Martin Wattenberg Martin Wicke Yuan Yu and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. http:\/\/tensorflow.org\/ Software available from tensorflow.org."},{"key":"e_1_3_2_1_9_1","volume-title":"Tensorflow: A system for large-scale machine learning. In <i>12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16)<\/i>. 265–283.","author":"Barham Paul","year":"2016","unstructured":"Martín Abadi, Paul Barham , Jianmin Chen , Zhifeng Chen , Andy Davis , Jeffrey Dean , Matthieu Devin , Sanjay Ghemawat , Geoffrey Irving , Michael Isard , 2016 . Tensorflow: A system for large-scale machine learning. In <i>12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16)<\/i>. 265–283. Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. Tensorflow: A system for large-scale machine learning. In <i>12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16)<\/i>. 265–283."},{"key":"e_1_3_2_1_10_1","volume-title":"Shreyan Gupta, Hongzi Mao, and Mohammad Alizadeh.","author":"Addanki Ravichandra","year":"2019","unstructured":"Ravichandra Addanki , Shaileshh Bojja Venkatakrishnan , Shreyan Gupta, Hongzi Mao, and Mohammad Alizadeh. 2019 . Placeto : Learning generalizable device placement algorithms for distributed machine learning. In <i>Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS) <\/i>. 3983–3993. Ravichandra Addanki, Shaileshh Bojja Venkatakrishnan, Shreyan Gupta, Hongzi Mao, and Mohammad Alizadeh. 2019. Placeto: Learning generalizable device placement algorithms for distributed machine learning. In <i>Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS)<\/i>. 3983–3993."},{"key":"e_1_3_2_1_11_1","volume-title":"ICLR 2015<\/i>.","author":"Bahdanau Dzmitry","year":"2015","unstructured":"Dzmitry Bahdanau , Kyung Hyun Cho , and Yoshua Bengio . 2015 . Neural machine translation by jointly learning to align and translate. In <i>3rd International Conference on Learning Representations , ICLR 2015<\/i>. Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In <i>3rd International Conference on Learning Representations, ICLR 2015<\/i>."},{"key":"e_1_3_2_1_12_1","volume-title":"Findings of the 2014 Workshop on Statistical Machine Translation. In <i>Proceedings of the Ninth Workshop on Statistical Machine Translation<\/i>. Association for Computational Linguistics","author":"Bojar Ondrej","year":"2014","unstructured":"Ondrej Bojar , Christian Buck , Christian Federmann , Barry Haddow , Philipp Koehn , Johannes Leveling , Christof Monz , Pavel Pecina , Matt Post , Herve Saint-Amand , Radu Soricut , Lucia Specia , and Aleš Tamchyna. 2014 . Findings of the 2014 Workshop on Statistical Machine Translation. In <i>Proceedings of the Ninth Workshop on Statistical Machine Translation<\/i>. Association for Computational Linguistics , Baltimore, Maryland, USA, 12–58. http:\/\/www.aclweb.org\/anthology\/W\/W14\/W14-3302 Ondrej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Specia, and Aleš Tamchyna. 2014. Findings of the 2014 Workshop on Statistical Machine Translation. In <i>Proceedings of the Ninth Workshop on Statistical Machine Translation<\/i>. Association for Computational Linguistics, Baltimore, Maryland, USA, 12–58. http:\/\/www.aclweb.org\/anthology\/W\/W14\/W14-3302"},{"key":"e_1_3_2_1_13_1","volume-title":"Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. <i>arXiv preprint arXiv:1512.01274<\/i&gt","author":"Chen Tianqi","year":"2015","unstructured":"Tianqi Chen , Mu Li , Yutian Li , Min Lin , Naiyan Wang , Minjie Wang , Tianjun Xiao , Bing Xu , Chiyuan Zhang , and Zheng Zhang . 2015 . Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. <i>arXiv preprint arXiv:1512.01274<\/i&gt ; (2015). Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. 2015. Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. <i>arXiv preprint arXiv:1512.01274<\/i> (2015)."},{"key":"e_1_3_2_1_14_1","volume-title":"PT-Scotch: A tool for efficient parallel graph ordering. <i>Parallel computing<\/i&gt","author":"Fran Chevalier","year":"2008","unstructured":"Cédric Chevalier and Fran çois Pellegrini. 2008. PT-Scotch: A tool for efficient parallel graph ordering. <i>Parallel computing<\/i&gt ; 34, 6-8 ( 2008 ), 318–331. Cédric Chevalier and François Pellegrini. 2008. PT-Scotch: A tool for efficient parallel graph ordering. <i>Parallel computing<\/i> 34, 6-8 (2008), 318–331."},{"key":"e_1_3_2_1_15_1","unstructured":"Trishul Chilimbi Yutaka Suzue Johnson Apacible and Karthik Kalyanaraman. 2014. Project adam: Building an efficient and scalable deep learning training system. In <i>11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14)<\/i>. 571–582. Trishul Chilimbi Yutaka Suzue Johnson Apacible and Karthik Kalyanaraman. 2014. Project adam: Building an efficient and scalable deep learning training system. In <i>11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14)<\/i>. 571–582."},{"key":"e_1_3_2_1_16_1","unstructured":"Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Marc'aurelio Ranzato Andrew Senior Paul Tucker Ke Yang etal 2012a. Large scale distributed deep networks. <i>Advances in neural information processing systems<\/i> 25 (2012) 1223–1231. Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Marc'aurelio Ranzato Andrew Senior Paul Tucker Ke Yang et al. 2012a. Large scale distributed deep networks. <i>Advances in neural information processing systems<\/i> 25 (2012) 1223–1231."},{"key":"e_1_3_2_1_17_1","unstructured":"Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Andrew Senior Paul Tucker Ke Yang Quoc V Le etal 2012b. Large scale distributed deep networks. In <i>Advances in neural information processing systems<\/i>. 1223–1231. Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Andrew Senior Paul Tucker Ke Yang Quoc V Le et al. 2012b. Large scale distributed deep networks. In <i>Advances in neural information processing systems<\/i>. 1223–1231."},{"key":"e_1_3_2_1_18_1","volume-title":"Imagenet: A large-scale hierarchical image database. In <i>2009 IEEE conference on computer vision and pattern recognition<\/i>. Ieee, 248–255.","author":"Deng Jia","year":"2009","unstructured":"Jia Deng , Wei Dong , Richard Socher , Li-Jia Li , Kai Li , and Li Fei-Fei . 2009 . Imagenet: A large-scale hierarchical image database. In <i>2009 IEEE conference on computer vision and pattern recognition<\/i>. Ieee, 248–255. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In <i>2009 IEEE conference on computer vision and pattern recognition<\/i>. Ieee, 248–255."},{"key":"e_1_3_2_1_19_1","volume-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In <i>Proceedings of the 2019 Conference of the North American","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In <i>Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) <\/i>. 4171–4186. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In <i>Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)<\/i>. 4171–4186."},{"key":"e_1_3_2_1_20_1","volume-title":"Lift and project algorithms for precedence constrained scheduling to minimize completion time. In <i>Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms<\/i>","author":"Garg Shashwat","unstructured":"Shashwat Garg , Janardhan Kulkarni , and Shi Li. 2019. Lift and project algorithms for precedence constrained scheduling to minimize completion time. In <i>Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms<\/i> . SIAM , 1570–1584. Shashwat Garg, Janardhan Kulkarni, and Shi Li. 2019. Lift and project algorithms for precedence constrained scheduling to minimize completion time. In <i>Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms<\/i>. SIAM, 1570–1584."},{"key":"e_1_3_2_1_21_1","volume-title":"Bounds on multiprocessing timing anomalies. <i>SIAM journal on Applied Mathematics<\/i&gt","author":"Graham Ronald L.","year":"1969","unstructured":"Ronald L. Graham . 1969. Bounds on multiprocessing timing anomalies. <i>SIAM journal on Applied Mathematics<\/i&gt ; 17, 2 ( 1969 ), 416–429. Ronald L. Graham. 1969. Bounds on multiprocessing timing anomalies. <i>SIAM journal on Applied Mathematics<\/i> 17, 2 (1969), 416–429."},{"key":"e_1_3_2_1_22_1","volume-title":"Empirical Analysis and Modeling of Compute Times of CNN Operations on AWS Cloud. In <i>2020 IEEE International Symposium on Workload Characterization (IISWC)<\/i>","author":"Hafeez Ubaid Ullah","unstructured":"Ubaid Ullah Hafeez and Anshul Gandhi . 2020. Empirical Analysis and Modeling of Compute Times of CNN Operations on AWS Cloud. In <i>2020 IEEE International Symposium on Workload Characterization (IISWC)<\/i> . IEEE , 181–192. Ubaid Ullah Hafeez and Anshul Gandhi. 2020. Empirical Analysis and Modeling of Compute Times of CNN Operations on AWS Cloud. In <i>2020 IEEE International Symposium on Workload Characterization (IISWC)<\/i>. IEEE, 181–192."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"crossref","unstructured":"Claire Hanen and Alix Munier. 1995. An approximation algorithm for scheduling dependent tasks on m processors with small communication delays. In <i>Proceedings 1995 INRIA\/IEEE Symposium on Emerging Technologies and Factory Automation. ETFA'95<\/i> Vol. 1. IEEE 167–189. Claire Hanen and Alix Munier. 1995. An approximation algorithm for scheduling dependent tasks on m processors with small communication delays. In <i>Proceedings 1995 INRIA\/IEEE Symposium on Emerging Technologies and Factory Automation. ETFA'95<\/i> Vol. 1. IEEE 167–189.","DOI":"10.1109\/ETFA.1995.496773"},{"key":"e_1_3_2_1_24_1","volume-title":"Performance of Coffman-Graham schedules in the presence of unit communication delays. <i>Discrete applied mathematics<\/i&gt","author":"Hanen Claire","year":"1998","unstructured":"Claire Hanen and Alix Munier . 1998. Performance of Coffman-Graham schedules in the presence of unit communication delays. <i>Discrete applied mathematics<\/i&gt ; 81, 1-3 ( 1998 ), 93–108. Claire Hanen and Alix Munier. 1998. Performance of Coffman-Graham schedules in the presence of unit communication delays. <i>Discrete applied mathematics<\/i> 81, 1-3 (1998), 93–108."},{"key":"e_1_3_2_1_25_1","volume-title":"Acyclic partitioning of large directed acyclic graphs. In <i>2017 17th IEEE\/ACM international symposium on cluster, cloud and grid computing (CCGRID)<\/i>","author":"Herrmann Julien","unstructured":"Julien Herrmann , Jonathan Kho , Bora Uçar, Kamer Kaya , and Ümit V Çatalyürek. 2017. Acyclic partitioning of large directed acyclic graphs. In <i>2017 17th IEEE\/ACM international symposium on cluster, cloud and grid computing (CCGRID)<\/i> . IEEE , 371–380. Julien Herrmann, Jonathan Kho, Bora Uçar, Kamer Kaya, and Ümit V Çatalyürek. 2017. Acyclic partitioning of large directed acyclic graphs. In <i>2017 17th IEEE\/ACM international symposium on cluster, cloud and grid computing (CCGRID)<\/i>. IEEE, 371–380."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"crossref","unstructured":"Geoffrey Hinton Li Deng Dong Yu George Dahl Abdel-rahman Mohamed Navdeep Jaitly Andrew Senior Vincent Vanhoucke Patrick Nguyen Brian Kingsbury etal 2012. Deep neural networks for acoustic modeling in speech recognition. <i>IEEE Signal processing magazine<\/i> 29 (2012). Geoffrey Hinton Li Deng Dong Yu George Dahl Abdel-rahman Mohamed Navdeep Jaitly Andrew Senior Vincent Vanhoucke Patrick Nguyen Brian Kingsbury et al. 2012. Deep neural networks for acoustic modeling in speech recognition. <i>IEEE Signal processing magazine<\/i> 29 (2012).","DOI":"10.1109\/MSP.2012.2205597"},{"key":"e_1_3_2_1_27_1","volume-title":"Long short-term memory. <i>Neural computation<\/i&gt","author":"Hochreiter Sepp","year":"1997","unstructured":"Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. <i>Neural computation<\/i&gt ; 9, 8 ( 1997 ), 1735–1780. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. <i>Neural computation<\/i> 9, 8 (1997), 1735–1780."},{"key":"e_1_3_2_1_28_1","volume-title":"Jan Karel Lenstra, and Bart Veltman","author":"Hoogeveen JA","year":"1994","unstructured":"JA Hoogeveen , Jan Karel Lenstra, and Bart Veltman . 1994 . Three, four, five, six, or the complexity of scheduling with communication delays. <i>Operations Research Letters <\/i> 16, 3 (1994), 129–137. JA Hoogeveen, Jan Karel Lenstra, and Bart Veltman. 1994. Three, four, five, six, or the complexity of scheduling with communication delays. <i>Operations Research Letters<\/i> 16, 3 (1994), 129–137."},{"key":"e_1_3_2_1_29_1","volume-title":"GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism. <i>arXiv preprint arXiv:1811.06965<\/i&gt","author":"Huang Yanping","year":"2018","unstructured":"Yanping Huang , Yonglong Cheng , Dehao Chen , HyoukJoong Lee , Jiquan Ngiam , Quoc V Le , and Zhifeng Chen . 2018. GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism. <i>arXiv preprint arXiv:1811.06965<\/i&gt ; ( 2018 ). Yanping Huang, Yonglong Cheng, Dehao Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, and Zhifeng Chen. 2018. GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism. <i>arXiv preprint arXiv:1811.06965<\/i> (2018)."},{"key":"e_1_3_2_1_30_1","unstructured":"IBM. [n.d.]. <i>CPLEX Optimizer<\/i>. https:\/\/www.ibm.com\/analytics\/cplex-optimizer IBM. [n.d.]. <i>CPLEX Optimizer<\/i>. https:\/\/www.ibm.com\/analytics\/cplex-optimizer"},{"key":"e_1_3_2_1_31_1","volume-title":"Baechi: fast device placement of machine learning graphs. In <i>Proceedings of the 11th ACM Symposium on Cloud Computing<\/i>. 416–430","author":"Jeon Beomyeol","unstructured":"Beomyeol Jeon , Linda Cai , Pallavi Srivastava , Jintao Jiang , Xiaolan Ke , Yitao Meng , Cong Xie , and Indranil Gupta . 2020a. Baechi: fast device placement of machine learning graphs. In <i>Proceedings of the 11th ACM Symposium on Cloud Computing<\/i>. 416–430 . Beomyeol Jeon, Linda Cai, Pallavi Srivastava, Jintao Jiang, Xiaolan Ke, Yitao Meng, Cong Xie, and Indranil Gupta. 2020a. Baechi: fast device placement of machine learning graphs. In <i>Proceedings of the 11th ACM Symposium on Cloud Computing<\/i>. 416–430."},{"key":"e_1_3_2_1_32_1","volume-title":"Baechi: fast device placement of machine learning graphs. In <i>Proceedings of the 11th ACM Symposium on Cloud Computing<\/i>. 416–430","author":"Jeon Beomyeol","unstructured":"Beomyeol Jeon , Linda Cai , Pallavi Srivastava , Jintao Jiang , Xiaolan Ke , Yitao Meng , Cong Xie , and Indranil Gupta . 2020b. Baechi: fast device placement of machine learning graphs. In <i>Proceedings of the 11th ACM Symposium on Cloud Computing<\/i>. 416–430 . Beomyeol Jeon, Linda Cai, Pallavi Srivastava, Jintao Jiang, Xiaolan Ke, Yitao Meng, Cong Xie, and Indranil Gupta. 2020b. Baechi: fast device placement of machine learning graphs. In <i>Proceedings of the 11th ACM Symposium on Cloud Computing<\/i>. 416–430."},{"key":"e_1_3_2_1_33_1","volume-title":"Beyond data and model parallelism for deep neural networks. <i>SysML 2019<\/i&gt","author":"Jia Zhihao","year":"2019","unstructured":"Zhihao Jia , Matei Zaharia , and Alex Aiken . 2019. Beyond data and model parallelism for deep neural networks. <i>SysML 2019<\/i&gt ; ( 2019 ). Zhihao Jia, Matei Zaharia, and Alex Aiken. 2019. Beyond data and model parallelism for deep neural networks. <i>SysML 2019<\/i> (2019)."},{"key":"e_1_3_2_1_34_1","volume-title":"Exploring the limits of language modeling. <i>arXiv preprint arXiv:1602.02410<\/i&gt","author":"Jozefowicz Rafal","year":"2016","unstructured":"Rafal Jozefowicz , Oriol Vinyals , Mike Schuster , Noam Shazeer , and Yonghui Wu. 2016. Exploring the limits of language modeling. <i>arXiv preprint arXiv:1602.02410<\/i&gt ; ( 2016 ). Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu. 2016. Exploring the limits of language modeling. <i>arXiv preprint arXiv:1602.02410<\/i> (2016)."},{"key":"e_1_3_2_1_35_1","volume-title":"Topological sorting of large networks. <i>Commun. ACM<\/i&gt","author":"Kahn Arthur B","year":"1962","unstructured":"Arthur B Kahn . 1962. Topological sorting of large networks. <i>Commun. ACM<\/i&gt ; 5, 11 ( 1962 ), 558–562. Arthur B Kahn. 1962. Topological sorting of large networks. <i>Commun. ACM<\/i> 5, 11 (1962), 558–562."},{"key":"e_1_3_2_1_36_1","volume-title":"A convolutional neural network for modelling sentences. In <i>52nd Annual Meeting of the Association for Computational Linguistics<\/i>","author":"Kalchbrenner N","unstructured":"N Kalchbrenner , E Grefenstette , and Philip Blunsom . 2014. A convolutional neural network for modelling sentences. In <i>52nd Annual Meeting of the Association for Computational Linguistics<\/i> . Association for Computational Linguistics . N Kalchbrenner, E Grefenstette, and Philip Blunsom. 2014. A convolutional neural network for modelling sentences. In <i>52nd Annual Meeting of the Association for Computational Linguistics<\/i>. Association for Computational Linguistics."},{"key":"e_1_3_2_1_37_1","unstructured":"Jin Kyu Kim Qirong Ho Seunghak Lee Xun Zheng Wei Dai Garth A Gibson and Eric P Xing. 2016. STRADS: a distributed framework for scheduled model parallel machine learning. In <i>Proceedings of the Eleventh European Conference on Computer Systems<\/i>. 1–16. Jin Kyu Kim Qirong Ho Seunghak Lee Xun Zheng Wei Dai Garth A Gibson and Eric P Xing. 2016. STRADS: a distributed framework for scheduled model parallel machine learning. In <i>Proceedings of the Eleventh European Conference on Computer Systems<\/i>. 1–16."},{"key":"e_1_3_2_1_38_1","volume-title":"One weird trick for parallelizing convolutional neural networks. <i>arXiv preprint arXiv:1404.5997<\/i&gt","author":"Krizhevsky Alex","year":"2014","unstructured":"Alex Krizhevsky . 2014. One weird trick for parallelizing convolutional neural networks. <i>arXiv preprint arXiv:1404.5997<\/i&gt ; ( 2014 ). Alex Krizhevsky. 2014. One weird trick for parallelizing convolutional neural networks. <i>arXiv preprint arXiv:1404.5997<\/i> (2014)."},{"key":"e_1_3_2_1_39_1","volume-title":"Hierarchy-based algorithms for minimizing makespan under precedence and communication constraints. In <i>Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms<\/i>","author":"Kulkarni Janardhan","unstructured":"Janardhan Kulkarni , Shi Li , Jakub Tarnawski , and Minwei Ye. 2020. Hierarchy-based algorithms for minimizing makespan under precedence and communication constraints. In <i>Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms<\/i> . SIAM , 2770–2789. Janardhan Kulkarni, Shi Li, Jakub Tarnawski, and Minwei Ye. 2020. Hierarchy-based algorithms for minimizing makespan under precedence and communication constraints. In <i>Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms<\/i>. SIAM, 2770–2789."},{"key":"e_1_3_2_1_40_1","volume-title":"Building high-level features using large scale unsupervised learning. In <i>2013 IEEE international conference on acoustics, speech and signal processing<\/i>","author":"Quoc V Le.","unstructured":"Quoc V Le. 2013. Building high-level features using large scale unsupervised learning. In <i>2013 IEEE international conference on acoustics, speech and signal processing<\/i> . IEEE , 8595–8598. Quoc V Le. 2013. Building high-level features using large scale unsupervised learning. In <i>2013 IEEE international conference on acoustics, speech and signal processing<\/i>. IEEE, 8595–8598."},{"key":"e_1_3_2_1_41_1","volume-title":"STOC16–201","author":"Levey Elaine","year":"2019","unstructured":"Elaine Levey and Thomas Rothvoss . 2019. A (1+ epsilon)-approximation for makespan scheduling with precedence constraints using LP hierarchies. <i>SIAM J. Comput .<\/i> 0 ( 2019 ), STOC16–201 . Elaine Levey and Thomas Rothvoss. 2019. A (1+ epsilon)-approximation for makespan scheduling with precedence constraints using LP hierarchies. <i>SIAM J. Comput.<\/i> 0 (2019), STOC16–201."},{"key":"e_1_3_2_1_42_1","volume":"201","author":"Li Ang","unstructured":"Ang Li , Shuaiwen Leon Song , Jieyang Chen , Jiajia Li , Xu Liu , Nathan R Tallent , and Kevin J Barker. 201 9. Evaluating modern GPU interconnect: Pcie, nvlink, nv-sli, nvswitch and gpudirect. <i>IEEE Transactions on Parallel and Distributed Systems<\/i> 31, 1 (2019), 94–110. Ang Li, Shuaiwen Leon Song, Jieyang Chen, Jiajia Li, Xu Liu, Nathan R Tallent, and Kevin J Barker. 2019. Evaluating modern GPU interconnect: Pcie, nvlink, nv-sli, nvswitch and gpudirect. <i>IEEE Transactions on Parallel and Distributed Systems<\/i> 31, 1 (2019), 94–110.","journal-title":"Kevin J Barker."},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"crossref","unstructured":"Mitchell Marcus Beatrice Santorini and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. (1993). Mitchell Marcus Beatrice Santorini and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. (1993).","DOI":"10.21236\/ADA273556"},{"key":"e_1_3_2_1_44_1","unstructured":"Azalia Mirhoseini Anna Goldie Hieu Pham Benoit Steiner Quoc V Le and Jeff Dean. 2018. A Hierarchical Model for Device Placement. In <i>International Conference on Learning Representations (ICLR)<\/i>. Azalia Mirhoseini Anna Goldie Hieu Pham Benoit Steiner Quoc V Le and Jeff Dean. 2018. A Hierarchical Model for Device Placement. In <i>International Conference on Learning Representations (ICLR)<\/i>."},{"key":"e_1_3_2_1_45_1","volume-title":"Device placement optimization with reinforcement learning. In <i>Proceedings of the 34th International Conference on Machine Learning-Volume 70<\/i>. JMLR. org, 2430–2439","author":"Mirhoseini Azalia","unstructured":"Azalia Mirhoseini , Hieu Pham , Quoc V Le , Benoit Steiner , Rasmus Larsen , Yuefeng Zhou , Naveen Kumar , Mohammad Norouzi , Samy Bengio , and Jeff Dean . 2017. Device placement optimization with reinforcement learning. In <i>Proceedings of the 34th International Conference on Machine Learning-Volume 70<\/i>. JMLR. org, 2430–2439 . Azalia Mirhoseini, Hieu Pham, Quoc V Le, Benoit Steiner, Rasmus Larsen, Yuefeng Zhou, Naveen Kumar, Mohammad Norouzi, Samy Bengio, and Jeff Dean. 2017. Device placement optimization with reinforcement learning. In <i>Proceedings of the 34th International Conference on Machine Learning-Volume 70<\/i>. JMLR. org, 2430–2439."},{"key":"e_1_3_2_1_46_1","volume-title":"PipeDream: generalized pipeline parallelism for DNN training. In <i>Proceedings of the 27th ACM Symposium on Operating Systems Principles<\/i>. 1–15","author":"Narayanan Deepak","unstructured":"Deepak Narayanan , Aaron Harlap , Amar Phanishayee , Vivek Seshadri , Nikhil R Devanur , Gregory R Ganger , Phillip B Gibbons , and Matei Zaharia . 2019. PipeDream: generalized pipeline parallelism for DNN training. In <i>Proceedings of the 27th ACM Symposium on Operating Systems Principles<\/i>. 1–15 . Deepak Narayanan, Aaron Harlap, Amar Phanishayee, Vivek Seshadri, Nikhil R Devanur, Gregory R Ganger, Phillip B Gibbons, and Matei Zaharia. 2019. PipeDream: generalized pipeline parallelism for DNN training. In <i>Proceedings of the 27th ACM Symposium on Operating Systems Principles<\/i>. 1–15."},{"key":"e_1_3_2_1_47_1","volume-title":"Pytorch: An imperative style, high-performance deep learning library. In <i>Advances in neural information processing systems<\/i>. 8026–8037.","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke , Sam Gross , Francisco Massa , Adam Lerer , James Bradbury , Gregory Chanan , Trevor Killeen , Zeming Lin , Natalia Gimelshein , Luca Antiga , 2019 . Pytorch: An imperative style, high-performance deep learning library. In <i>Advances in neural information processing systems<\/i>. 8026–8037. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. In <i>Advances in neural information processing systems<\/i>. 8026–8037."},{"key":"e_1_3_2_1_48_1","volume-title":"Zero: Memory optimizations toward training trillion parameter models. In <i>SC20: International Conference for High Performance Computing, Networking, Storage and Analysis<\/i>","author":"Rajbhandari Samyam","year":"2020","unstructured":"Samyam Rajbhandari , Jeff Rasley , Olatunji Ruwase , and Yuxiong He . 2020 . Zero: Memory optimizations toward training trillion parameter models. In <i>SC20: International Conference for High Performance Computing, Networking, Storage and Analysis<\/i> . IEEE , 1–16. Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. Zero: Memory optimizations toward training trillion parameter models. In <i>SC20: International Conference for High Performance Computing, Networking, Storage and Analysis<\/i>. IEEE, 1–16."},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"crossref","unstructured":"Jeff Rasley Samyam Rajbhandari Olatunji Ruwase and Yuxiong He. 2020. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In <i>Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining<\/i>. 3505–3506. Jeff Rasley Samyam Rajbhandari Olatunji Ruwase and Yuxiong He. 2020. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In <i>Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining<\/i>. 3505–3506.","DOI":"10.1145\/3394486.3406703"},{"key":"e_1_3_2_1_50_1","volume-title":"Regularized evolution for image classifier architecture search. In <i>Proceedings of the aaai conference on artificial intelligence<\/i>","author":"Real Esteban","unstructured":"Esteban Real , Alok Aggarwal , Yanping Huang , and Quoc V Le. 2019. Regularized evolution for image classifier architecture search. In <i>Proceedings of the aaai conference on artificial intelligence<\/i> , Vol. 33 . 4780–4789. Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le. 2019. Regularized evolution for image classifier architecture search. In <i>Proceedings of the aaai conference on artificial intelligence<\/i>, Vol. 33. 4780–4789."},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"crossref","unstructured":"Yassir Samadi Mostapha Zbakh and Claude Tadonki. 2018. E-HEFT: enhancement heterogeneous earliest finish time algorithm for task scheduling based on load balancing in cloud computing. In <i>2018 International Conference on High Performance Computing & Simulation (HPCS)<\/i>. IEEE 601–609. Yassir Samadi Mostapha Zbakh and Claude Tadonki. 2018. E-HEFT: enhancement heterogeneous earliest finish time algorithm for task scheduling based on load balancing in cloud computing. In <i>2018 International Conference on High Performance Computing & Simulation (HPCS)<\/i>. IEEE 601–609.","DOI":"10.1109\/HPCS.2018.00100"},{"key":"e_1_3_2_1_52_1","unstructured":"Noam Shazeer Youlong Cheng Niki Parmar Dustin Tran Ashish Vaswani Penporn Koanantakool Peter Hawkins HyoukJoong Lee Mingsheng Hong Cliff Young etal 2018. Mesh-TensorFlow: deep learning for supercomputers. In <i>Proceedings of the 32nd International Conference on Neural Information Processing Systems<\/i>. 10435–10444. Noam Shazeer Youlong Cheng Niki Parmar Dustin Tran Ashish Vaswani Penporn Koanantakool Peter Hawkins HyoukJoong Lee Mingsheng Hong Cliff Young et al. 2018. Mesh-TensorFlow: deep learning for supercomputers. In <i>Proceedings of the 32nd International Conference on Neural Information Processing Systems<\/i>. 10435–10444."},{"key":"e_1_3_2_1_53_1","unstructured":"Kenneth W Stufflebeam Jr. 2006. Configurable PCI express switch which allows multiple CPUs to be connected to multiple I\/O devices. US Patent 7 058 738. Kenneth W Stufflebeam Jr. 2006. Configurable PCI express switch which allows multiple CPUs to be connected to multiple I\/O devices. US Patent 7 058 738."},{"key":"e_1_3_2_1_54_1","doi-asserted-by":"crossref","unstructured":"Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jon Shlens and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In <i>Proceedings of the IEEE conference on computer vision and pattern recognition<\/i>. 2818–2826. Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jon Shlens and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In <i>Proceedings of the IEEE conference on computer vision and pattern recognition<\/i>. 2818–2826.","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_3_2_1_55_1","volume-title":"Performance-effective and low-complexity task scheduling for heterogeneous computing. <i&gt","author":"Topcuoglu Haluk","year":"2002","unstructured":"Haluk Topcuoglu , Salim Hariri , and Min-you Wu. 2002. Performance-effective and low-complexity task scheduling for heterogeneous computing. <i&gt ; IEEE transactions on parallel and distributed systems<\/i> 13, 3 ( 2002 ), 260–274. Haluk Topcuoglu, Salim Hariri, and Min-you Wu. 2002. Performance-effective and low-complexity task scheduling for heterogeneous computing. <i>IEEE transactions on parallel and distributed systems<\/i> 13, 3 (2002), 260–274."},{"key":"e_1_3_2_1_56_1","volume-title":"NP-complete scheduling problems. <i>J. Comput. System Sci.<\/i&gt","author":"Ullman J.D.","year":"1975","unstructured":"J.D. Ullman . 1975. NP-complete scheduling problems. <i>J. Comput. System Sci.<\/i&gt ; 10 ( 1975 ), 384–393. J.D. Ullman. 1975. NP-complete scheduling problems. <i>J. Comput. System Sci.<\/i> 10 (1975), 384–393."},{"key":"e_1_3_2_1_57_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In <i>Advances in neural information processing systems<\/i>. 5998–6008. Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In <i>Advances in neural information processing systems<\/i>. 5998–6008."},{"key":"e_1_3_2_1_58_1","unstructured":"Yonghui Wu Mike Schuster Zhifeng Chen Quoc V Le Mohammad Norouzi Wolfgang Macherey Maxim Krikun Yuan Cao Qin Gao Klaus Macherey etal 2016. Google's neural machine translation system: Bridging the gap between human and machine translation. <i>arXiv preprint arXiv:1609.08144<\/i> (2016). Yonghui Wu Mike Schuster Zhifeng Chen Quoc V Le Mohammad Norouzi Wolfgang Macherey Maxim Krikun Yuan Cao Qin Gao Klaus Macherey et al. 2016. Google's neural machine translation system: Bridging the gap between human and machine translation. <i>arXiv preprint arXiv:1609.08144<\/i> (2016)."},{"key":"e_1_3_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.5555\/902507"},{"key":"e_1_3_2_1_60_1","volume-title":"Recurrent neural network regularization. <i>arXiv preprint arXiv:1409.2329<\/i&gt","author":"Zaremba Wojciech","year":"2014","unstructured":"Wojciech Zaremba , Ilya Sutskever , and Oriol Vinyals . 2014. Recurrent neural network regularization. <i>arXiv preprint arXiv:1409.2329<\/i&gt ; ( 2014 ). Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent neural network regularization. <i>arXiv preprint arXiv:1409.2329<\/i> (2014)."},{"key":"e_1_3_2_1_61_1","doi-asserted-by":"crossref","unstructured":"Barret Zoph Vijay Vasudevan Jonathon Shlens and Quoc V Le. 2018. Learning transferable architectures for scalable image recognition. In <i>Proceedings of the IEEE conference on computer vision and pattern recognition<\/i>. 8697–8710. Barret Zoph Vijay Vasudevan Jonathon Shlens and Quoc V Le. 2018. Learning transferable architectures for scalable image recognition. In <i>Proceedings of the IEEE conference on computer vision and pattern recognition<\/i>. 8697–8710.","DOI":"10.1109\/CVPR.2018.00907"}],"event":{"name":"Middleware '21: 22nd International Middleware Conference","location":"Qu\u00e9bec city Canada","acronym":"Middleware '21","sponsor":["ACM Association for Computing Machinery","USENIX Assoc USENIX Assoc","IFIP"]},"container-title":["Proceedings of the 22nd International Middleware Conference"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3464298.3476132","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/abs\/10.1145\/3464298.3476132","content-type":"text\/html","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3464298.3476132","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3464298.3476132","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:12:15Z","timestamp":1750191135000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3464298.3476132"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,2]]},"references-count":61,"alternative-id":["10.1145\/3464298.3476132","10.1145\/3464298"],"URL":"https:\/\/doi.org\/10.1145\/3464298.3476132","relation":{},"subject":[],"published":{"date-parts":[[2021,10,2]]},"assertion":[{"value":"2021-10-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}