{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,11]],"date-time":"2026-03-11T01:44:29Z","timestamp":1773193469293,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":64,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,6,3]],"date-time":"2021-06-03T00:00:00Z","timestamp":1622678400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,6,3]]},"DOI":"10.1145\/3447818.3460372","type":"proceedings-article","created":{"date-parts":[[2021,6,4]],"date-time":"2021-06-04T15:09:36Z","timestamp":1622819376000},"page":"254-265","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Partitioning sparse deep neural networks for scalable training and inference"],"prefix":"10.1145","author":[{"given":"Gunduz Vehbi","family":"Demirci","sequence":"first","affiliation":[{"name":"University of Warwick"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hakan","family":"Ferhatosmanoglu","sequence":"additional","affiliation":[{"name":"University of Warwick"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,6,4]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Sparse communication for distributed gradient descent. arXiv preprint arXiv:1704.05021","author":"Aji Alham Fikri","year":"2017","unstructured":"Alham Fikri Aji and Kenneth Heafield . 2017. Sparse communication for distributed gradient descent. arXiv preprint arXiv:1704.05021 ( 2017 ). Alham Fikri Aji and Kenneth Heafield. 2017. Sparse communication for distributed gradient descent. arXiv preprint arXiv:1704.05021 (2017)."},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1137\/100813956"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3018743.3018769"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPEC.2019.8916223"},{"key":"e_1_3_2_1_5_1","unstructured":"Tom B Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell etal 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020).  Tom B Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)."},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3343211.3343218"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/71.780863"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3369583.3392686"},{"key":"e_1_3_2_1_9_1","volume-title":"11th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 14). 571--582.","author":"Chilimbi Trishul","unstructured":"Trishul Chilimbi , Yutaka Suzue , Johnson Apacible , and Karthik Kalyanaraman . 2014. Project adam: Building an efficient and scalable deep learning training system . In 11th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 14). 571--582. Trishul Chilimbi, Yutaka Suzue, Johnson Apacible, and Karthik Kalyanaraman. 2014. Project adam: Building an efficient and scalable deep learning training system. In 11th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 14). 571--582."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3392717.3392771"},{"key":"e_1_3_2_1_11_1","volume-title":"International conference on machine learning. 1337--1345","author":"Coates Adam","year":"2013","unstructured":"Adam Coates , Brody Huval , Tao Wang , David Wu , Bryan Catanzaro , and Ng Andrew . 2013 . Deep learning with COTS HPC systems . In International conference on machine learning. 1337--1345 . Adam Coates, Brody Huval, Tao Wang, David Wu, Bryan Catanzaro, and Ng Andrew. 2013. Deep learning with COTS HPC systems. In International conference on machine learning. 1337--1345."},{"key":"e_1_3_2_1_12_1","first-page":"2493","article-title":"Natural language processing (almost) from scratch","author":"Collobert Ronan","year":"2011","unstructured":"Ronan Collobert , Jason Weston , L\u00e9on Bottou , Michael Karlen , Koray Kavukcuoglu , and Pavel Kuksa . 2011 . Natural language processing (almost) from scratch . Journal of machine learning research 12 , Aug (2011), 2493 -- 2537 . Ronan Collobert, Jason Weston, L\u00e9on Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of machine learning research 12, Aug (2011), 2493--2537.","journal-title":"Journal of machine learning research 12"},{"key":"e_1_3_2_1_13_1","volume-title":"Distributed deep learning using synchronous stochastic gradient descent. arXiv preprint arXiv:1602.06709","author":"Das Dipankar","year":"2016","unstructured":"Dipankar Das , Sasikanth Avancha , Dheevatsa Mudigere , Karthikeyan Vaidynathan , Srinivas Sridharan , Dhiraj Kalamkar , Bharat Kaul , and Pradeep Dubey . 2016. Distributed deep learning using synchronous stochastic gradient descent. arXiv preprint arXiv:1602.06709 ( 2016 ). Dipankar Das, Sasikanth Avancha, Dheevatsa Mudigere, Karthikeyan Vaidynathan, Srinivas Sridharan, Dhiraj Kalamkar, Bharat Kaul, and Pradeep Dubey. 2016. Distributed deep learning using synchronous stochastic gradient descent. arXiv preprint arXiv:1602.06709 (2016)."},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3322125"},{"key":"e_1_3_2_1_15_1","volume-title":"Write quick, run fast: Sparse deep neural network in 20 minutes of development time via SuiteSparse: GraphBLAS. In 2019 IEEE High Performance extreme Computing Conference (HPEC)","author":"Davis Timothy A","unstructured":"Timothy A Davis , Mohsen Aznaveh , and Scott Kolodziej . 2019. Write quick, run fast: Sparse deep neural network in 20 minutes of development time via SuiteSparse: GraphBLAS. In 2019 IEEE High Performance extreme Computing Conference (HPEC) . IEEE , 1--6. Timothy A Davis, Mohsen Aznaveh, and Scott Kolodziej. 2019. Write quick, run fast: Sparse deep neural network in 20 minutes of development time via SuiteSparse: GraphBLAS. In 2019 IEEE High Performance extreme Computing Conference (HPEC). IEEE, 1--6."},{"key":"e_1_3_2_1_16_1","unstructured":"Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Marc'aurelio Ranzato Andrew Senior Paul Tucker Ke Yang etal 2012. Large scale distributed deep networks. In Advances in neural information processing systems. 1223--1231.  Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Marc'aurelio Ranzato Andrew Senior Paul Tucker Ke Yang et al. 2012. Large scale distributed deep networks. In Advances in neural information processing systems . 1223--1231."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_2_1_18_1","volume-title":"The state of sparsity in deep neural networks. arXiv preprint arXiv:1902.09574","author":"Gale Trevor","year":"2019","unstructured":"Trevor Gale , Erich Elsen , and Sara Hooker . 2019. The state of sparsity in deep neural networks. arXiv preprint arXiv:1902.09574 ( 2019 ). Trevor Gale, Erich Elsen, and Sara Hooker. 2019. The state of sparsity in deep neural networks. arXiv preprint arXiv:1902.09574 (2019)."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3330345.3330386"},{"key":"e_1_3_2_1_20_1","volume-title":"large minibatch sgd: Training imagenet in 1 hour. arXiv preprint arXiv:1706.02677","author":"Goyal Priya","year":"2017","unstructured":"Priya Goyal , Piotr Doll\u00e1r , Ross Girshick , Pieter Noordhuis , Lukasz Wesolowski , Aapo Kyrola , Andrew Tulloch , Yangqing Jia , and Kaiming He. 2017. Accurate , large minibatch sgd: Training imagenet in 1 hour. arXiv preprint arXiv:1706.02677 ( 2017 ). Priya Goyal, Piotr Doll\u00e1r, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. 2017. Accurate, large minibatch sgd: Training imagenet in 1 hour. arXiv preprint arXiv:1706.02677 (2017)."},{"key":"e_1_3_2_1_21_1","volume-title":"Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural networks 18, 5-6","author":"Graves Alex","year":"2005","unstructured":"Alex Graves and J\u00fcrgen Schmidhuber . 2005. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural networks 18, 5-6 ( 2005 ), 602--610. Alex Graves and J\u00fcrgen Schmidhuber. 2005. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural networks 18, 5-6 (2005), 602--610."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"crossref","unstructured":"Robert M Gray. 2006. Toeplitz and circulant matrices: A review. now publishers inc.  Robert M Gray. 2006. Toeplitz and circulant matrices: A review . now publishers inc.","DOI":"10.1561\/9781933019680"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/3433701.3433722"},{"key":"e_1_3_2_1_24_1","unstructured":"Babak Hassibi and David G Stork. 1993. Second order derivatives for network pruning: Optimal brain surgeon. In Advances in neural information processing systems. 164--171.  Babak Hassibi and David G Stork. 1993. Second order derivatives for network pruning: Optimal brain surgeon. In Advances in neural information processing systems . 164--171."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_1_26_1","unstructured":"B Hendrickson and TG Kolda. [n.d.]. Partitioning Rectangular and Structurally Nonsymmetric Sparse Matrices for Parallel Processing submitted to SIAM Journal of Scientific Computing.  B Hendrickson and TG Kolda. [n.d.]. Partitioning Rectangular and Structurally Nonsymmetric Sparse Matrices for Parallel Processing submitted to SIAM Journal of Scientific Computing."},{"key":"e_1_3_2_1_27_1","volume-title":"At-Scale Sparse Deep Neural Network Inference With Efficient GPU Implementation. In 2020 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 1--7.","author":"Hidayeto\u011flu Mert","year":"2020","unstructured":"Mert Hidayeto\u011flu , Carl Pearson , Vikram Sharma Mailthody , Eiman Ebrahimi , Jinjun Xiong , Rakesh Nagi , and Wen-mei Hwu. 2020 . At-Scale Sparse Deep Neural Network Inference With Efficient GPU Implementation. In 2020 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 1--7. Mert Hidayeto\u011flu, Carl Pearson, Vikram Sharma Mailthody, Eiman Ebrahimi, Jinjun Xiong, Rakesh Nagi, and Wen-mei Hwu. 2020. At-Scale Sparse Deep Neural Network Inference With Efficient GPU Implementation. In 2020 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 1--7."},{"key":"e_1_3_2_1_28_1","volume-title":"Selective Brain Damage: Measuring the Disparate Impact of Model Pruning. arXiv preprint arXiv:1911.05248","author":"Hooker Sara","year":"2019","unstructured":"Sara Hooker , Aaron Courville , Yann Dauphin , and Andrea Frome . 2019. Selective Brain Damage: Measuring the Disparate Impact of Model Pruning. arXiv preprint arXiv:1911.05248 ( 2019 ). Sara Hooker, Aaron Courville, Yann Dauphin, and Andrea Frome. 2019. Selective Brain Damage: Measuring the Disparate Impact of Model Pruning. arXiv preprint arXiv:1911.05248 (2019)."},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.284"},{"key":"e_1_3_2_1_30_1","volume-title":"Exploring hidden dimensions in parallelizing convolutional neural networks. arXiv preprint arXiv:1802.04924","author":"Jia Zhihao","year":"2018","unstructured":"Zhihao Jia , Sina Lin , Charles R Qi , and Alex Aiken . 2018. Exploring hidden dimensions in parallelizing convolutional neural networks. arXiv preprint arXiv:1802.04924 ( 2018 ). Zhihao Jia, Sina Lin, Charles R Qi, and Alex Aiken. 2018. Exploring hidden dimensions in parallelizing convolutional neural networks. arXiv preprint arXiv:1802.04924 (2018)."},{"key":"e_1_3_2_1_31_1","volume-title":"Beyond Data and Model Parallelism for Deep Neural Networks. SysML 2019","author":"Jia Zhihao","year":"2019","unstructured":"Zhihao Jia , Matei Zaharia , and Alex Aiken . 2019. Beyond Data and Model Parallelism for Deep Neural Networks. SysML 2019 ( 2019 ). Zhihao Jia, Matei Zaharia, and Alex Aiken. 2019. Beyond Data and Model Parallelism for Deep Neural Networks. SysML 2019 (2019)."},{"key":"e_1_3_2_1_32_1","volume-title":"How to scale distributed deep learning? arXiv preprint arXiv:1611.04581","author":"Jin Peter H","year":"2016","unstructured":"Peter H Jin , Qiaochu Yuan , Forrest Iandola , and Kurt Keutzer . 2016. How to scale distributed deep learning? arXiv preprint arXiv:1611.04581 ( 2016 ). Peter H Jin, Qiaochu Yuan, Forrest Iandola, and Kurt Keutzer. 2016. How to scale distributed deep learning? arXiv preprint arXiv:1611.04581 (2016)."},{"key":"e_1_3_2_1_33_1","unstructured":"George Karypis. 1998. hMETIS 1.5: A hypergraph partitioning package. http:\/\/www. cs. umn. edu\/\u0303 metis (1998).  George Karypis. 1998. hMETIS 1.5: A hypergraph partitioning package. http:\/\/www. cs. umn. edu\/\u0303 metis (1998)."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2807591.2807624"},{"key":"e_1_3_2_1_35_1","volume-title":"org Sparse Deep Neural Network Performance. arXiv preprint arXiv:2004.01181","author":"Kepner Jeremy","year":"2020","unstructured":"Jeremy Kepner , Simon Alford , Vijay Gadepally , Michael Jones , Lauren Milechin , Albert Reuther , Ryan Robinett , and Sid Samsi . 2020. GraphChallenge. org Sparse Deep Neural Network Performance. arXiv preprint arXiv:2004.01181 ( 2020 ). Jeremy Kepner, Simon Alford, Vijay Gadepally, Michael Jones, Lauren Milechin, Albert Reuther, Ryan Robinett, and Sid Samsi. 2020. GraphChallenge. org Sparse Deep Neural Network Performance. arXiv preprint arXiv:2004.01181 (2020)."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPEC.2019.8916336"},{"key":"e_1_3_2_1_37_1","volume-title":"RadiX-Net: Structured Sparse Matrices for Deep Neural Networks. In 2019 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, 268--274","author":"Kepner Jeremy","year":"2019","unstructured":"Jeremy Kepner and Ryan Robinett . 2019 . RadiX-Net: Structured Sparse Matrices for Deep Neural Networks. In 2019 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, 268--274 . Jeremy Kepner and Ryan Robinett. 2019. RadiX-Net: Structured Sparse Matrices for Deep Neural Networks. In 2019 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, 268--274."},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/BFb0018528"},{"key":"e_1_3_2_1_39_1","volume-title":"Ananda Theertha Suresh, and Dave Bacon","author":"Kone\u010dn\u1ef3 Jakub","year":"2016","unstructured":"Jakub Kone\u010dn\u1ef3 , H Brendan McMahan , Felix X Yu , Peter Richt\u00e1rik , Ananda Theertha Suresh, and Dave Bacon . 2016 . Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492 (2016). Jakub Kone\u010dn\u1ef3, H Brendan McMahan, Felix X Yu, Peter Richt\u00e1rik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492 (2016)."},{"key":"e_1_3_2_1_40_1","unstructured":"Alex Krizhevsky Ilya Sutskever and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.  Alex Krizhevsky Ilya Sutskever and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems . 1097--1105."},{"key":"e_1_3_2_1_41_1","unstructured":"Yann LeCun. 1998. The MNIST database of handwritten digits. http:\/\/yann. lecun. com\/exdb\/mnist\/ (1998).  Yann LeCun. 1998. The MNIST database of handwritten digits. http:\/\/yann. lecun. com\/exdb\/mnist\/ (1998)."},{"key":"e_1_3_2_1_42_1","unstructured":"Yann LeCun John S Denker and Sara A Solla. 1990. Optimal brain damage. In Advances in neural information processing systems. 598--605.  Yann LeCun John S Denker and Sara A Solla. 1990. Optimal brain damage. In Advances in neural information processing systems . 598--605."},{"key":"e_1_3_2_1_43_1","volume-title":"Alexander J Smola, Amr Ahmed, Vanja Josifovski, James Long, Eugene J Shekita, and Bor-Yiing Su.","author":"Li Mu","year":"2014","unstructured":"Mu Li , David G Andersen , Jun Woo Park , Alexander J Smola, Amr Ahmed, Vanja Josifovski, James Long, Eugene J Shekita, and Bor-Yiing Su. 2014 . Scaling distributed machine learning with the parameter server. In 11th {USENIX} Symposium on Operating Systems Design and Implementation ( {OSDI} 14). 583--598. Mu Li, David G Andersen, Jun Woo Park, Alexander J Smola, Amr Ahmed, Vanja Josifovski, James Long, Eugene J Shekita, and Bor-Yiing Su. 2014. Scaling distributed machine learning with the parameter server. In 11th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 14). 583--598."},{"key":"e_1_3_2_1_44_1","volume-title":"PyTorch Distributed: Experiences on Accelerating Data Parallel Training. Proceedings of the VLDB Endowment 13","author":"Li Shen","unstructured":"Shen Li , Yanli Zhao , Rohan Varma , Omkar Salpekar , Pieter Noordhuis , Teng Li , Adam Paszke , Jeff Smith , Brian Vaughan , Pritam Damania , PyTorch Distributed: Experiences on Accelerating Data Parallel Training. Proceedings of the VLDB Endowment 13 , 12 ([n. d.]). Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, et al. [n.d.]. PyTorch Distributed: Experiences on Accelerating Data Parallel Training. Proceedings of the VLDB Endowment 13, 12 ([n. d.])."},{"key":"e_1_3_2_1_45_1","volume-title":"Deep gradient compression: Reducing the communication bandwidth for distributed training. arXiv preprint arXiv:1712.01887","author":"Lin Yujun","year":"2017","unstructured":"Yujun Lin , Song Han , Huizi Mao , Yu Wang , and William J Dally . 2017. Deep gradient compression: Reducing the communication bandwidth for distributed training. arXiv preprint arXiv:1712.01887 ( 2017 ). Yujun Lin, Song Han, Huizi Mao, Yu Wang, and William J Dally. 2017. Deep gradient compression: Reducing the communication bandwidth for distributed training. arXiv preprint arXiv:1712.01887 (2017)."},{"key":"e_1_3_2_1_46_1","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition. 806--814","author":"Liu Baoyuan","year":"2015","unstructured":"Baoyuan Liu , Min Wang , Hassan Foroosh , Marshall Tappen , and Marianna Pensky . 2015 . Sparse convolutional neural networks . In Proceedings of the IEEE conference on computer vision and pattern recognition. 806--814 . Baoyuan Liu, Min Wang, Hassan Foroosh, Marshall Tappen, and Marianna Pensky. 2015. Sparse convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 806--814."},{"key":"e_1_3_2_1_47_1","volume-title":"Learning Sparse Neural Networks through L0 Regularization. arXiv preprint arXiv:1712.01312","author":"Louizos Christos","year":"2017","unstructured":"Christos Louizos , Max Welling , and Diederik P Kingma . 2017. Learning Sparse Neural Networks through L0 Regularization. arXiv preprint arXiv:1712.01312 ( 2017 ). Christos Louizos, Max Welling, and Diederik P Kingma. 2017. Learning Sparse Neural Networks through L0 Regularization. arXiv preprint arXiv:1712.01312 (2017)."},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPEC.2019.8916494"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPEC43674.2020.9286195"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3330345.3330384"},{"key":"e_1_3_2_1_51_1","volume-title":"Combinatorial Tiling for Sparse Neural Networks. In 2020 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 1--7.","author":"Paw\u0142owski Filip","year":"2020","unstructured":"Filip Paw\u0142owski , Rob H Bisseling , Bora U\u00e7ar , and AN Yzelman . 2020 . Combinatorial Tiling for Sparse Neural Networks. In 2020 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 1--7. Filip Paw\u0142owski, Rob H Bisseling, Bora U\u00e7ar, and AN Yzelman. 2020. Combinatorial Tiling for Sparse Neural Networks. In 2020 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 1--7."},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01261-8_2"},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2011.332"},{"key":"e_1_3_2_1_54_1","volume-title":"Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman . 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 ( 2014 ). Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)."},{"key":"e_1_3_2_1_55_1","volume-title":"Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1","author":"Srivastava Nitish","year":"2014","unstructured":"Nitish Srivastava , Geoffrey Hinton , Alex Krizhevsky , Ilya Sutskever , and Ruslan Salakhutdinov . 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1 ( 2014 ), 1929--1958. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1 (2014), 1929--1958."},{"key":"e_1_3_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.220"},{"key":"e_1_3_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3369583.3392681"},{"key":"e_1_3_2_1_58_1","volume-title":"Accelerating DNN Inference with GraphBLAS and the GPU. In 2019 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 1--6.","author":"Wang Xiaoyun","year":"2019","unstructured":"Xiaoyun Wang , Zhongyi Lin , Carl Yang , and John D Owens . 2019 . Accelerating DNN Inference with GraphBLAS and the GPU. In 2019 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 1--6. Xiaoyun Wang, Zhongyi Lin, Carl Yang, and John D Owens. 2019. Accelerating DNN Inference with GraphBLAS and the GPU. In 2019 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 1--6."},{"key":"e_1_3_2_1_59_1","unstructured":"Jianqiao Wangni Jialei Wang Ji Liu and Tong Zhang. 2018. Gradient sparsification for communication-efficient distributed optimization. In Advances in Neural Information Processing Systems. 1299--1309.  Jianqiao Wangni Jialei Wang Ji Liu and Tong Zhang. 2018. Gradient sparsification for communication-efficient distributed optimization. In Advances in Neural Information Processing Systems . 1299--1309."},{"key":"e_1_3_2_1_60_1","volume-title":"Fast sparse matrix-vector multiplication on GPUs: implications for graph mining. arXiv preprint arXiv:1103.2405","author":"Yang Xintian","year":"2011","unstructured":"Xintian Yang , Srinivasan Parthasarathy , and Ponnuswamy Sadayappan . 2011. Fast sparse matrix-vector multiplication on GPUs: implications for graph mining. arXiv preprint arXiv:1103.2405 ( 2011 ). Xintian Yang, Srinivasan Parthasarathy, and Ponnuswamy Sadayappan. 2011. Fast sparse matrix-vector multiplication on GPUs: implications for graph mining. arXiv preprint arXiv:1103.2405 (2011)."},{"key":"e_1_3_2_1_61_1","volume-title":"Scaling sgd batch size to 32k for imagenet training. arXiv preprint arXiv:1708.03888 6","author":"You Yang","year":"2017","unstructured":"Yang You , Igor Gitman , and Boris Ginsburg . 2017. Scaling sgd batch size to 32k for imagenet training. arXiv preprint arXiv:1708.03888 6 ( 2017 ). Yang You, Igor Gitman, and Boris Ginsburg. 2017. Scaling sgd batch size to 32k for imagenet training. arXiv preprint arXiv:1708.03888 6 (2017)."},{"key":"e_1_3_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2019.2913833"},{"key":"e_1_3_2_1_63_1","unstructured":"Sixin Zhang Anna E Choromanska and Yann LeCun. 2015. Deep learning with elastic averaging SGD. In Advances in neural information processing systems. 685--693.  Sixin Zhang Anna E Choromanska and Yann LeCun. 2015. Deep learning with elastic averaging SGD. In Advances in neural information processing systems . 685--693."},{"key":"e_1_3_2_1_64_1","volume-title":"To prune, or not to prune: exploring the efficacy of pruning for model compression. arXiv preprint arXiv:1710.01878","author":"Zhu Michael","year":"2017","unstructured":"Michael Zhu and Suyog Gupta . 2017. To prune, or not to prune: exploring the efficacy of pruning for model compression. arXiv preprint arXiv:1710.01878 ( 2017 ). Michael Zhu and Suyog Gupta. 2017. To prune, or not to prune: exploring the efficacy of pruning for model compression. arXiv preprint arXiv:1710.01878 (2017)."}],"event":{"name":"ICS '21: 2021 International Conference on Supercomputing","location":"Virtual Event USA","acronym":"ICS '21","sponsor":["SIGARCH ACM Special Interest Group on Computer Architecture"]},"container-title":["Proceedings of the ACM International Conference on Supercomputing"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447818.3460372","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3447818.3460372","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T17:49:27Z","timestamp":1750268967000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447818.3460372"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,3]]},"references-count":64,"alternative-id":["10.1145\/3447818.3460372","10.1145\/3447818"],"URL":"https:\/\/doi.org\/10.1145\/3447818.3460372","relation":{},"subject":[],"published":{"date-parts":[[2021,6,3]]},"assertion":[{"value":"2021-06-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}