{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:57:25Z","timestamp":1760245045640,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":26,"publisher":"ACM","license":[{"start":{"date-parts":[[2017,10,23]],"date-time":"2017-10-23T00:00:00Z","timestamp":1508716800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2017,10,23]]},"DOI":"10.1145\/3126686.3126749","type":"proceedings-article","created":{"date-parts":[[2017,10,23]],"date-time":"2017-10-23T19:20:32Z","timestamp":1508786432000},"page":"110-116","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Efficient Communications in Training Large Scale Neural Networks"],"prefix":"10.1145","author":[{"given":"Yiyang","family":"Zhao","sequence":"first","affiliation":[{"name":"University of Electronic Science and Technology of China, Chengdu, China"}]},{"given":"Linnan","family":"Wang","sequence":"additional","affiliation":[{"name":"Brown University, Rhode Island, USA"}]},{"given":"Wei","family":"Wu","sequence":"additional","affiliation":[{"name":"University of Tennessee, Knoxville, TN, USA"}]},{"given":"George","family":"Bosilca","sequence":"additional","affiliation":[{"name":"University of Tennessee, Knoxville, TN, USA"}]},{"given":"Richard","family":"Vuduc","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}]},{"given":"Jinmian","family":"Ye","sequence":"additional","affiliation":[{"name":"University of Electronic Science and Technology of China, Chengdu, China"}]},{"given":"Wenqi","family":"Tang","sequence":"additional","affiliation":[{"name":"University of Electronic Science and Technology of China, Chengdu, China"}]},{"given":"Zenglin","family":"Xu","sequence":"additional","affiliation":[{"name":"University of Electronic Science and Technology of China, Chengdu, China"}]}],"member":"320","published-online":{"date-parts":[[2017,10,23]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.5555\/2627435.2638571"},{"key":"e_1_3_2_1_2_1","volume-title":"Duchi","author":"Agarwal Alekh","year":"2011","unstructured":"Alekh Agarwal and John C . Duchi . 2011 . Distributed delayed stochastic optimization. In Advances in Neural Information Processing Systems . 873--881. Alekh Agarwal and John C. Duchi. 2011. Distributed delayed stochastic optimization. In Advances in Neural Information Processing Systems. 873--881."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1088149.1088183"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.v19:13"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2352\/ISSN.2470-1173.2016.16.HVEI-131","article-title":"From Vision Science to Data Science: Applying Perception to Problems in Big Data","volume":"2016","author":"Chang Remco","year":"2016","unstructured":"Remco Chang , Fumeng Yang , and Marianne Procopio . 2016 . From Vision Science to Data Science: Applying Perception to Problems in Big Data . Electronic Imaging , Vol. 2016 , 16 (2016), 1 -- 7 . Remco Chang, Fumeng Yang, and Marianne Procopio. 2016. From Vision Science to Data Science: Applying Perception to Problems in Big Data. Electronic Imaging, Vol. 2016, 16 (2016), 1--7.","journal-title":"Electronic Imaging"},{"key":"e_1_3_2_1_6_1","volume-title":"Proceedings of the 30th international conference on machine learning. 1337--1345","author":"Coates Adam","year":"2013","unstructured":"Adam Coates , Brody Huval , Tao Wang , David Wu , Bryan Catanzaro , and Ng Andrew . 2013 . Deep learning with COTS HPC systems . In Proceedings of the 30th international conference on machine learning. 1337--1345 . Adam Coates, Brody Huval, Tao Wang, David Wu, Bryan Catanzaro, and Ng Andrew. 2013. Deep learning with COTS HPC systems. In Proceedings of the 30th international conference on machine learning. 1337--1345."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2901318.2901323"},{"key":"e_1_3_2_1_8_1","unstructured":"Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Andrew Senior Paul Tucker Ke Yang Quoc V. Le etal 2012. Large scale distributed deep networks. In Advances in Neural Information Processing Systems. 1223--1231. Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Andrew Senior Paul Tucker Ke Yang Quoc V. Le et al. 2012. Large scale distributed deep networks. In Advances in Neural Information Processing Systems. 1223--1231."},{"key":"e_1_3_2_1_9_1","first-page":"165","article-title":"Optimal distributed online prediction using mini-batches","volume":"13","author":"Dekel Ofer","year":"2012","unstructured":"Ofer Dekel , Ran Gilad-Bachrach , Ohad Shamir , and Lin Xiao . 2012 . Optimal distributed online prediction using mini-batches . The Journal of Machine Learning Research Vol. 13 , 1 (2012), 165 -- 202 . Ofer Dekel, Ran Gilad-Bachrach, Ohad Shamir, and Lin Xiao. 2012. Optimal distributed online prediction using mini-batches. The Journal of Machine Learning Research Vol. 13, 1 (2012), 165--202.","journal-title":"The Journal of Machine Learning Research"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2021068"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"crossref","unstructured":"Edgar Gabriel Graham E Fagg George Bosilca Thara Angskun Jack J. Dongarra Jeffrey M. Squyres Vishal Sahay Prabhanjan Kambadur Brian Barrett Andrew Lumsdaine etal 2004. Open MPI: Goals concept and design of a next generation MPI implementation European Parallel Virtual Machine\/Message Passing Interface Users' Group Meeting. Springer 97--104. Edgar Gabriel Graham E Fagg George Bosilca Thara Angskun Jack J. Dongarra Jeffrey M. Squyres Vishal Sahay Prabhanjan Kambadur Brian Barrett Andrew Lumsdaine et al. 2004. Open MPI: Goals concept and design of a next generation MPI implementation European Parallel Virtual Machine\/Message Passing Interface Users' Group Meeting. Springer 97--104.","DOI":"10.1007\/978-3-540-30218-6_19"},{"key":"e_1_3_2_1_12_1","volume-title":"Phillip B. Gibbons, Garth A. Gibson, Greg Ganger, and Eric P. Xing.","author":"Ho Qirong","year":"2013","unstructured":"Qirong Ho , James Cipar , Henggang Cui , Seunghak Lee , Jin Kyu Kim , Phillip B. Gibbons, Garth A. Gibson, Greg Ganger, and Eric P. Xing. 2013 . More effective distributed ml via a stale synchronous parallel parameter server Advances in neural information processing systems. 1223--1231. Qirong Ho, James Cipar, Henggang Cui, Seunghak Lee, Jin Kyu Kim, Phillip B. Gibbons, Garth A. Gibson, Greg Ganger, and Eric P. Xing. 2013. More effective distributed ml via a stale synchronous parallel parameter server Advances in neural information processing systems. 1223--1231."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2654889"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"crossref","unstructured":"Mu Li David G. Andersen Alex J. Smola and Kai Yu. 2014. Communication efficient distributed machine learning with the parameter server Advances in Neural Information Processing Systems. 19--27. Mu Li David G. Andersen Alex J. Smola and Kai Yu. 2014. Communication efficient distributed machine learning with the parameter server Advances in Neural Information Processing Systems. 19--27.","DOI":"10.1145\/2640087.2644155"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2774993.2774997"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2774993.2774994"},{"key":"e_1_3_2_1_17_1","volume-title":"Hogwild: A lock-free approach to parallelizing stochastic gradient descent Advances in Neural Information Processing Systems. 693--701.","author":"Recht Benjamin","year":"2011","unstructured":"Benjamin Recht , Christopher Re , Stephen Wright , and Feng Niu . 2011 . Hogwild: A lock-free approach to parallelizing stochastic gradient descent Advances in Neural Information Processing Systems. 693--701. Benjamin Recht, Christopher Re, Stephen Wright, and Feng Niu. 2011. Hogwild: A lock-free approach to parallelizing stochastic gradient descent Advances in Neural Information Processing Systems. 693--701."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"crossref","unstructured":"Frank Seide Hao Fu Jasha Droppo Gang Li and Dong Yu. 2014. 1-bit stochastic gradient descent and its application to data-parallel distributed training of speech DNNs. In INTERSPEECH. 1058--1062. Frank Seide Hao Fu Jasha Droppo Gang Li and Dong Yu. 2014. 1-bit stochastic gradient descent and its application to data-parallel distributed training of speech DNNs. In INTERSPEECH. 1058--1062.","DOI":"10.21437\/Interspeech.2014-274"},{"key":"e_1_3_2_1_19_1","unstructured":"Ohad Shamir. 2014. Fundamental limits of online and distributed algorithms for statistical learning and estimation. In Advances in Neural Information Processing Systems. 163--171. Ohad Shamir. 2014. Fundamental limits of online and distributed algorithms for statistical learning and estimation. In Advances in Neural Information Processing Systems. 163--171."},{"volume-title":"Proceedings 20th IEEE International Parallel & Distributed Processing Symposium. IEEE, 10--pp.","author":"Shipman Galen M.","key":"e_1_3_2_1_20_1","unstructured":"Galen M. Shipman , Timothy S. Woodall , Richard L. Graham , Arthur B. Maccabe , and Patrick G. Bridges . 2006. Infiniband scalability in Open MPI . In Proceedings 20th IEEE International Parallel & Distributed Processing Symposium. IEEE, 10--pp. Galen M. Shipman, Timothy S. Woodall, Richard L. Graham, Arthur B. Maccabe, and Patrick G. Bridges. 2006. Infiniband scalability in Open MPI. In Proceedings 20th IEEE International Parallel & Distributed Processing Symposium. IEEE, 10--pp."},{"key":"e_1_3_2_1_21_1","volume-title":"Gropp","author":"Thakur Rajeev","year":"2003","unstructured":"Rajeev Thakur and William D . Gropp . 2003 . Improving the performance of collective operations in MPICH. Recent Advances in Parallel Virtual Machine and Message Passing Interface. Springer , 257--267. Rajeev Thakur and William D. Gropp. 2003. Improving the performance of collective operations in MPICH. Recent Advances in Parallel Virtual Machine and Message Passing Interface. Springer, 257--267."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1177\/1094342005051521"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2925426.2926256"},{"key":"e_1_3_2_1_24_1","volume-title":"Martin Renqiang Min, and Srimat Chakradhar","author":"Wang Linnan","year":"2016","unstructured":"Linnan Wang , Yi Yang , Martin Renqiang Min, and Srimat Chakradhar . 2016 . Accelerating Deep Neural Network Training with Inconsistent Stochastic Gradient Descent . arXiv preprint arXiv:1603.05544 (2016). Linnan Wang, Yi Yang, Martin Renqiang Min, and Srimat Chakradhar. 2016. Accelerating Deep Neural Network Training with Inconsistent Stochastic Gradient Descent. arXiv preprint arXiv:1603.05544 (2016)."},{"key":"e_1_3_2_1_25_1","volume-title":"LCN'03","author":"Worringen Joachim","year":"2003","unstructured":"Joachim Worringen . 2003 . Pipelining and overlapping for MPI collective operations Local Computer Networks, 2003 . LCN'03 . Proceedings. 28th Annual IEEE International Conference on. IEEE, 548--557. Joachim Worringen. 2003. Pipelining and overlapping for MPI collective operations Local Computer Networks, 2003. LCN'03. Proceedings. 28th Annual IEEE International Conference on. IEEE, 548--557."},{"key":"e_1_3_2_1_26_1","volume-title":"Smola","author":"Zinkevich Martin","year":"2010","unstructured":"Martin Zinkevich , Markus Weimer , Lihong Li , and Alex J . Smola . 2010 . Parallelized stochastic gradient descent. In Advances in neural information processing systems. 2595--2603. Martin Zinkevich, Markus Weimer, Lihong Li, and Alex J. Smola. 2010. Parallelized stochastic gradient descent. In Advances in neural information processing systems. 2595--2603."}],"event":{"name":"MM '17: ACM Multimedia Conference","sponsor":["SIGMM ACM Special Interest Group on Multimedia"],"location":"Mountain View California USA","acronym":"MM '17"},"container-title":["Proceedings of the on Thematic Workshops of ACM Multimedia 2017"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3126686.3126749","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3126686.3126749","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:10:54Z","timestamp":1750212654000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3126686.3126749"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,10,23]]},"references-count":26,"alternative-id":["10.1145\/3126686.3126749","10.1145\/3126686"],"URL":"https:\/\/doi.org\/10.1145\/3126686.3126749","relation":{},"subject":[],"published":{"date-parts":[[2017,10,23]]},"assertion":[{"value":"2017-10-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}