{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,19]],"date-time":"2025-09-19T08:56:59Z","timestamp":1758272219120,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":27,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,8,17]],"date-time":"2020-08-17T00:00:00Z","timestamp":1597622400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,8,17]]},"DOI":"10.1145\/3404397.3404432","type":"proceedings-article","created":{"date-parts":[[2020,8,9]],"date-time":"2020-08-09T03:54:26Z","timestamp":1596945266000},"page":"1-10","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Developing a Loss Prediction-based Asynchronous Stochastic Gradient Descent Algorithm for Distributed Training of Deep Neural Networks"],"prefix":"10.1145","author":[{"given":"Junyu","family":"Li","sequence":"first","affiliation":[{"name":"University of Warwick"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ligang","family":"He","sequence":"additional","affiliation":[{"name":"University of Warwick"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shenyuan","family":"Ren","sequence":"additional","affiliation":[{"name":"University of Oxford"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rui","family":"Mao","sequence":"additional","affiliation":[{"name":"Shenzhen University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,8,17]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467(2016).","author":"Abadi Mart\u00edn","year":"2016","unstructured":"Mart\u00edn Abadi , Ashish Agarwal , Paul Barham , Eugene Brevdo , Zhifeng Chen , Craig Citro , Greg\u00a0 S Corrado , Andy Davis , Jeffrey Dean , Matthieu Devin , 2016 . Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467(2016). Mart\u00edn Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg\u00a0S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, 2016. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467(2016)."},{"key":"e_1_3_2_1_2_1","volume-title":"QSGD: Communication-efficient SGD via gradient quantization and encoding. In Advances in Neural Information Processing Systems. 1709\u20131720.","author":"Alistarh Dan","year":"2017","unstructured":"Dan Alistarh , Demjan Grubic , Jerry Li , Ryota Tomioka , and Milan Vojnovic . 2017 . QSGD: Communication-efficient SGD via gradient quantization and encoding. In Advances in Neural Information Processing Systems. 1709\u20131720. Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. 2017. QSGD: Communication-efficient SGD via gradient quantization and encoding. In Advances in Neural Information Processing Systems. 1709\u20131720."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1137\/16M1080173"},{"key":"e_1_3_2_1_4_1","unstructured":"Sorathan Chaturapruek John\u00a0C Duchi and Christopher R\u00e9. 2015. Asynchronous stochastic convex optimization: the noise is in the noise and SGD don\u2019t care. In Advances in Neural Information Processing Systems. 1531\u20131539. Sorathan Chaturapruek John\u00a0C Duchi and Christopher R\u00e9. 2015. Asynchronous stochastic convex optimization: the noise is in the noise and SGD don\u2019t care. In Advances in Neural Information Processing Systems. 1531\u20131539."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"crossref","unstructured":"Kai Chen and Qiang Huo. 2016. Scalable training of deep learning machines by incremental block training with intra-block parallel optimization and blockwise model-update filtering. In 2016 ieee international conference on acoustics speech and signal processing (icassp). IEEE 5880\u20135884. Kai Chen and Qiang Huo. 2016. Scalable training of deep learning machines by incremental block training with intra-block parallel optimization and blockwise model-update filtering. In 2016 ieee international conference on acoustics speech and signal processing (icassp). IEEE 5880\u20135884.","DOI":"10.1109\/ICASSP.2016.7472805"},{"key":"e_1_3_2_1_6_1","unstructured":"Dipankar Das Sasikanth Avancha Dheevatsa Mudigere Karthikeyan Vaidynathan Srinivas Sridharan Dhiraj Kalamkar Bharat Kaul and Pradeep Dubey. 2016. Distributed deep learning using synchronous stochastic gradient descent. arXiv preprint arXiv:1602.06709(2016). Dipankar Das Sasikanth Avancha Dheevatsa Mudigere Karthikeyan Vaidynathan Srinivas Sridharan Dhiraj Kalamkar Bharat Kaul and Pradeep Dubey. 2016. Distributed deep learning using synchronous stochastic gradient descent. arXiv preprint arXiv:1602.06709(2016)."},{"key":"e_1_3_2_1_7_1","unstructured":"Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Andrew Senior Paul Tucker Ke Yang Quoc\u00a0V Le 2012. Large scale distributed deep networks. In Advances in neural information processing systems. 1223\u20131231. Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Andrew Senior Paul Tucker Ke Yang Quoc\u00a0V Le 2012. Large scale distributed deep networks. In Advances in neural information processing systems. 1223\u20131231."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_1_9_1","unstructured":"Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167(2015). Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167(2015)."},{"key":"e_1_3_2_1_10_1","unstructured":"Jakub Kone\u010dn\u1ef3 H\u00a0Brendan McMahan Felix\u00a0X Yu Peter Richt\u00e1rik Ananda\u00a0Theertha Suresh and Dave Bacon. 2016. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492(2016). Jakub Kone\u010dn\u1ef3 H\u00a0Brendan McMahan Felix\u00a0X Yu Peter Richt\u00e1rik Ananda\u00a0Theertha Suresh and Dave Bacon. 2016. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492(2016)."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2018.2833074"},{"key":"e_1_3_2_1_12_1","volume-title":"Deep learning. nature 521, 7553","author":"LeCun Yann","year":"2015","unstructured":"Yann LeCun , Yoshua Bengio , and Geoffrey Hinton . 2015. Deep learning. nature 521, 7553 ( 2015 ), 436. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"crossref","unstructured":"Mu Li David\u00a0G Andersen Alexander\u00a0J Smola and Kai Yu. 2014. Communication efficient distributed machine learning with the parameter server. In Advances in Neural Information Processing Systems. 19\u201327. Mu Li David\u00a0G Andersen Alexander\u00a0J Smola and Kai Yu. 2014. Communication efficient distributed machine learning with the parameter server. In Advances in Neural Information Processing Systems. 19\u201327.","DOI":"10.1145\/2640087.2644155"},{"key":"e_1_3_2_1_14_1","unstructured":"H\u00a0Brendan McMahan Eider Moore Daniel Ramage Seth Hampson 2016. Communication-efficient learning of deep networks from decentralized data. arXiv preprint arXiv:1602.05629(2016). H\u00a0Brendan McMahan Eider Moore Daniel Ramage Seth Hampson 2016. Communication-efficient learning of deep networks from decentralized data. arXiv preprint arXiv:1602.05629(2016)."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.5555\/3060832.3060880"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2017.11.044"},{"key":"e_1_3_2_1_17_1","volume-title":"Sparknet: Training deep networks in spark. arXiv preprint arXiv:1511.06051(2015).","author":"Moritz Philipp","year":"2015","unstructured":"Philipp Moritz , Robert Nishihara , Ion Stoica , and Michael\u00a0 I Jordan . 2015 . Sparknet: Training deep networks in spark. arXiv preprint arXiv:1511.06051(2015). Philipp Moritz, Robert Nishihara, Ion Stoica, and Michael\u00a0I Jordan. 2015. Sparknet: Training deep networks in spark. arXiv preprint arXiv:1511.06051(2015)."},{"key":"e_1_3_2_1_18_1","volume-title":"Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In Advances in neural information processing systems. 693\u2013701.","author":"Recht Benjamin","year":"2011","unstructured":"Benjamin Recht , Christopher Re , Stephen Wright , and Feng Niu . 2011 . Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In Advances in neural information processing systems. 693\u2013701. Benjamin Recht, Christopher Re, Stephen Wright, and Feng Niu. 2011. Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In Advances in neural information processing systems. 693\u2013701."},{"key":"e_1_3_2_1_19_1","unstructured":"Anand Srinivasan Ajay Jain and Parnian Barekatain. 2018. An analysis of the delayed gradients problem in asynchronous sgd. (2018). Anand Srinivasan Ajay Jain and Parnian Barekatain. 2018. An analysis of the delayed gradients problem in asynchronous sgd. (2018)."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180220"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2019.2904348"},{"key":"e_1_3_2_1_22_1","volume-title":"Terngrad: Ternary gradients to reduce communication in distributed deep learning. In Advances in neural information processing systems. 1509\u20131519.","author":"Wen Wei","year":"2017","unstructured":"Wei Wen , Cong Xu , Feng Yan , Chunpeng Wu , Yandan Wang , Yiran Chen , and Hai Li . 2017 . Terngrad: Ternary gradients to reduce communication in distributed deep learning. In Advances in neural information processing systems. 1509\u20131519. Wei Wen, Cong Xu, Feng Yan, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. 2017. Terngrad: Ternary gradients to reduce communication in distributed deep learning. In Advances in neural information processing systems. 1509\u20131519."},{"key":"e_1_3_2_1_23_1","unstructured":"Jiaxiang Wu Weidong Huang Junzhou Huang and Tong Zhang. 2018. Error compensated quantized SGD and its applications to large-scale distributed optimization. arXiv preprint arXiv:1806.08054(2018). Jiaxiang Wu Weidong Huang Junzhou Huang and Tong Zhang. 2018. Error compensated quantized SGD and its applications to large-scale distributed optimization. arXiv preprint arXiv:1806.08054(2018)."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2018.8461870"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3298981"},{"key":"e_1_3_2_1_26_1","volume-title":"Thirtieth AAAI Conference on Artificial Intelligence.","author":"Zhao Shen-Yi","year":"2016","unstructured":"Shen-Yi Zhao and Wu-Jun Li . 2016 . Fast asynchronous parallel stochastic gradient descent: A lock-free approach with convergence guarantee . In Thirtieth AAAI Conference on Artificial Intelligence. Shen-Yi Zhao and Wu-Jun Li. 2016. Fast asynchronous parallel stochastic gradient descent: A lock-free approach with convergence guarantee. In Thirtieth AAAI Conference on Artificial Intelligence."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/3305890.3306107"}],"event":{"name":"ICPP '20: 49th International Conference on Parallel Processing","acronym":"ICPP '20","location":"Edmonton AB Canada"},"container-title":["49th International Conference on Parallel Processing - ICPP"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3404397.3404432","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3404397.3404432","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:31:42Z","timestamp":1750195902000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3404397.3404432"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,17]]},"references-count":27,"alternative-id":["10.1145\/3404397.3404432","10.1145\/3404397"],"URL":"https:\/\/doi.org\/10.1145\/3404397.3404432","relation":{},"subject":[],"published":{"date-parts":[[2020,8,17]]},"assertion":[{"value":"2020-08-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}