{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,13]],"date-time":"2026-11-13T15:29:25Z","timestamp":1794583765771,"version":"4.9.1"},"reference-count":211,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2020,2,6]],"date-time":"2020-02-06T00:00:00Z","timestamp":1580947200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2021,1,31]]},"abstract":"<jats:p>Deep Learning (DL) has had an immense success in the recent past, leading to state-of-the-art results in various domains, such as image recognition and natural language processing. One of the reasons for this success is the increasing size of DL models and the proliferation of vast amounts of training data being available. To keep on improving the performance of DL, increasing the scalability of DL systems is necessary. In this survey, we perform a broad and thorough investigation on challenges, techniques and tools for scalable DL on distributed infrastructures. This incorporates infrastructures for DL, methods for parallel DL training, multi-tenant resource scheduling, and the management of training and model data. Further, we analyze and compare 11 current open-source DL frameworks and tools and investigate which of the techniques are commonly implemented in practice. Finally, we highlight future research trends in DL systems that deserve further research.<\/jats:p>","DOI":"10.1145\/3363554","type":"journal-article","created":{"date-parts":[[2020,2,6]],"date-time":"2020-02-06T16:54:04Z","timestamp":1581008044000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":182,"title":["Scalable Deep Learning on Distributed Infrastructures"],"prefix":"10.1145","volume":"53","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9870-7466","authenticated-orcid":false,"given":"Ruben","family":"Mayer","sequence":"first","affiliation":[{"name":"Technical University of Munich, Boltzmannstrasse, Garching, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hans-Arno","family":"Jacobsen","sequence":"additional","affiliation":[{"name":"Technical University of Munich, Boltzmannstrasse, Garching, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,2,6]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"NVIDIA. NVIDIA Collective Communications Library (NCCL). Retrieved from https:\/\/developer.nvidia.com\/nccl.  NVIDIA. NVIDIA Collective Communications Library (NCCL). Retrieved from https:\/\/developer.nvidia.com\/nccl."},{"key":"e_1_2_1_2_1","unstructured":"NVIDIA. NVIDIA DGX Station. Retrieved from https:\/\/www.nvidia.com\/en-us\/data-center\/dgx-station\/.  NVIDIA. NVIDIA DGX Station. Retrieved from https:\/\/www.nvidia.com\/en-us\/data-center\/dgx-station\/."},{"key":"e_1_2_1_3_1","unstructured":"ONNX Project Contributors. ONNX. Retrieved from https:\/\/onnx.ai\/.  ONNX Project Contributors. ONNX. Retrieved from https:\/\/onnx.ai\/."},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201916)","author":"Abadi Mart\u00edn","year":"2016"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2976749.2978318"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.5555\/2627435.2638571"},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the Workshop on ML Systems in the 31st Annual Conference on Neural Information Processing Systems (NIPS\u201917)","author":"Akiba Takuya","year":"2017"},{"key":"e_1_2_1_8_1","volume-title":"QSGD: Communication-efficient SGD via gradient quantization and encoding. In Advances in Neural Information Processing Systems 30. Curran Associates","author":"Alistarh Dan","year":"2017"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1721654.1721672"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2017.2743240"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2017.2752706"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Baker Bowen","year":"2017"},{"key":"e_1_2_1_13_1","volume-title":"The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines","author":"Barroso Luiz Andre","edition":"1"},{"key":"e_1_2_1_14_1","unstructured":"Tal Ben-Nun and Torsten Hoefler. 2018. Demystifying parallel and distributed deep learning: An in-depth concurrency analysis. Retrieved from http:\/\/arxiv.org\/abs\/1802.09941.  Tal Ben-Nun and Torsten Hoefler. 2018. Demystifying parallel and distributed deep learning: An in-depth concurrency analysis. Retrieved from http:\/\/arxiv.org\/abs\/1802.09941."},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the BigLearning Workshop (NIPS\u201911)","volume":"3","author":"Bergstra James","year":"2011"},{"key":"e_1_2_1_16_1","first-page":"1","article-title":"Random search for hyper-parameter optimization","volume":"13","author":"Bergstra James","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1147\/JRD.2017.2716578"},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the Workshop on ML Systems (NIPS\u201917)","author":"Boag Scott","year":"2017"},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of the Conference on Systems and Machine Learning (SysML\u201919)","author":"Bonawitz Keith","year":"2019"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219861"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-018-04933-y"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1961189.1961199"},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201916)","author":"Chen K.","year":"2016"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of Workshop on Machine Learning Systems (LearningSys\u201915)","author":"Chen Tianqi","year":"2015"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2014.2325029"},{"key":"e_1_2_1_26_1","unstructured":"Sharan Chetlur Cliff Woolley Philippe Vandermersch Jonathan Cohen John Tran Bryan Catanzaro and Evan Shelhamer. 2014. cuDNN: Efficient primitives for deep learning. Retrieved from http:\/\/arxiv.org\/abs\/1410.0759.  Sharan Chetlur Cliff Woolley Philippe Vandermersch Jonathan Cohen John Tran Bryan Catanzaro and Evan Shelhamer. 2014. cuDNN: Efficient primitives for deep learning. Retrieved from http:\/\/arxiv.org\/abs\/1410.0759."},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14)","author":"Chilimbi Trishul","year":"2014"},{"key":"e_1_2_1_28_1","volume-title":"Proceedings of the 14th Workshop on Hot Topics in Operating Systems (HotOS\u201913)","author":"Cipar James","year":"2013"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2012.02.023"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1162\/NECO_a_00052"},{"key":"e_1_2_1_31_1","unstructured":"Microsoft. Multiple GPUs and Machines -- Cognitive Toolkit -- CNTK | Microsoft Docs. Retrieved from https:\/\/docs.microsoft.com\/en-us\/cognitive-toolkit\/multiple-gpus-and-machines.  Microsoft. Multiple GPUs and Machines -- Cognitive Toolkit -- CNTK | Microsoft Docs. Retrieved from https:\/\/docs.microsoft.com\/en-us\/cognitive-toolkit\/multiple-gpus-and-machines."},{"key":"e_1_2_1_32_1","volume-title":"Proceedings of the 30th International Conference on International Conference on Machine Learning","volume":"28","author":"Coates Adam","year":"2013"},{"key":"e_1_2_1_33_1","unstructured":"Apple. Apple CoreML. Retrieved from https:\/\/developer.apple.com\/machine-learning\/.  Apple. Apple CoreML. Retrieved from https:\/\/developer.apple.com\/machine-learning\/."},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201917)","author":"Crankshaw Daniel","year":"2017"},{"key":"e_1_2_1_35_1","volume-title":"Proceedings of the USENIX Conference on USENIX Annual Technical Conference (USENIX ATC\u201914)","author":"Cui Henggang"},{"key":"e_1_2_1_36_1","volume-title":"Proceedings of the 11th European Conference on Computer Systems (EuroSys\u201916)","author":"Cui Henggang"},{"key":"e_1_2_1_37_1","volume-title":"Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI\u201915)","author":"Dai Wei"},{"key":"e_1_2_1_38_1","unstructured":"Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Andrew Senior Paul Tucker Ke Yang Quoc V. Le etal 2012. Large scale distributed deep networks. In Advances in Neural Information Processing Systems. MIT Press 1223--1231.  Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Andrew Senior Paul Tucker Ke Yang Quoc V. Le et al. 2012. Large scale distributed deep networks. In Advances in Neural Information Processing Systems. MIT Press 1223--1231."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1327452.1327492"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1017\/atsip.2013.9"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1515\/pralin-2017-0036"},{"key":"e_1_2_1_43_1","unstructured":"Eclipse Deeplearning4j. Deeplearning4j on Spark: Technical Explanation. Retrieved from https:\/\/deeplearning4j.org\/docs\/latest\/deeplearning4j-scaleout-technicalref.  Eclipse Deeplearning4j. Deeplearning4j on Spark: Technical Explanation. Retrieved from https:\/\/deeplearning4j.org\/docs\/latest\/deeplearning4j-scaleout-technicalref."},{"key":"e_1_2_1_44_1","volume-title":"Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12 (July","author":"Duchi John","year":"2011"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10278-017-9965-6"},{"key":"e_1_2_1_46_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201911)","author":"Farabet C.","year":"2011"},{"key":"e_1_2_1_47_1","volume-title":"Advances in Neural Information Processing Systems 31","author":"Fischer Volker"},{"key":"e_1_2_1_48_1","doi-asserted-by":"crossref","volume-title":"Weka-A Machine Learning Workbench for Data Mining","author":"Frank Eibe","DOI":"10.1007\/978-0-387-09823-4_66"},{"key":"e_1_2_1_49_1","volume-title":"Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation (NSDI\u201911)","author":"Ghodsi Ali","year":"2011"},{"key":"e_1_2_1_50_1","unstructured":"Andrew Gibiansky. [n.d.]. Bringing HPC Techniques to Deep Learning. Retrieved from http:\/\/andrew.gibiansky.com\/blog\/machine-learning\/baidu-allreduce\/. http:\/\/andrew.gibiansky.com\/blog\/machine-learning\/baidu-allreduce\/.  Andrew Gibiansky. [n.d.]. Bringing HPC Techniques to Deep Learning. Retrieved from http:\/\/andrew.gibiansky.com\/blog\/machine-learning\/baidu-allreduce\/. http:\/\/andrew.gibiansky.com\/blog\/machine-learning\/baidu-allreduce\/."},{"key":"e_1_2_1_51_1","volume-title":"Advances in Neural Information Processing Systems","author":"Goodfellow Ian"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2619239.2626334"},{"key":"e_1_2_1_53_1","volume-title":"International Conference on Machine Learning. MIT Press, 1764--1772","author":"Graves Alex","year":"2014"},{"key":"e_1_2_1_54_1","volume-title":"Proceedings of the 18th ACM\/IFIP\/USENIX Middleware Conference (Middleware\u201917)","author":"Gujarati Arpan"},{"key":"e_1_2_1_55_1","volume-title":"Proceedings of the 32nd International Conference on International Conference on Machine Learning","volume":"37","author":"Gupta Suyog","year":"2015"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2009.36"},{"key":"e_1_2_1_57_1","volume-title":"Gibbons","author":"Harlap Aaron","year":"2018"},{"key":"e_1_2_1_58_1","volume-title":"Proceedings of the Conference on Systems and Machine Learning (SysML\u201918)","author":"Harlap Aaron"},{"key":"e_1_2_1_59_1","volume-title":"Proceedings of the 12th European Conference on Computer Systems (EuroSys\u201917)","author":"Harlap Aaron"},{"key":"e_1_2_1_60_1","volume-title":"Proceedings of the Conference on Systems and Machine Learning (SysML\u201919)","author":"Hashemi Sayed Hadi"},{"key":"e_1_2_1_61_1","volume-title":"Proceedings of the ACM\/IEEE 42nd Annual International Symposium on Computer Architecture (ISCA\u201915)","author":"Hauswald J."},{"key":"e_1_2_1_62_1","volume-title":"Proceedings of the IEEE International Symposium on High Performance Computer Architecture (HPCA\u201918)","author":"Hazelwood K.","year":"2018"},{"key":"e_1_2_1_63_1","volume-title":"Yang Yang, and Yanqi Zhou.","author":"Hestness Joel","year":"2017"},{"key":"e_1_2_1_64_1","volume-title":"Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation (NSDI\u201911)","author":"Hindman Benjamin","year":"2011"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2012.2205597"},{"key":"e_1_2_1_66_1","doi-asserted-by":"crossref","unstructured":"G. E. Hinton and R. R. Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science 313 5786 (2006) 504--507. DOI:https:\/\/doi.org\/10.1126\/science.1127647 arXiv:http:\/\/science.sciencemag.org\/content\/313\/5786\/504.full.pdf  G. E. Hinton and R. R. Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science 313 5786 (2006) 504--507. DOI:https:\/\/doi.org\/10.1126\/science.1127647 arXiv:http:\/\/science.sciencemag.org\/content\/313\/5786\/504.full.pdf","DOI":"10.1126\/science.1127647"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_2_1_68_1","first-page":"6","article-title":"A comprehensive survey of deep learning for image captioning","volume":"51","author":"Zakir Hossain M. D.","year":"2019","journal-title":"ACM Comput. Surv."},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/2500887"},{"key":"e_1_2_1_70_1","unstructured":"Yanping Huang Yonglong Cheng Dehao Chen HyoukJoong Lee Jiquan Ngiam Quoc V. Le and Zhifeng Chen. 2018. GPipe: Efficient training of giant neural networks using pipeline parallelism. Retrieved from http:\/\/arxiv.org\/abs\/1811.06965.  Yanping Huang Yonglong Cheng Dehao Chen HyoukJoong Lee Jiquan Ngiam Quoc V. Le and Zhifeng Chen. 2018. GPipe: Efficient training of giant neural networks using pipeline parallelism. Retrieved from http:\/\/arxiv.org\/abs\/1811.06965."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3187009.3177734"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1113\/jphysiol.1962.sp006837"},{"key":"e_1_2_1_73_1","volume-title":"Proceedings of the 31st International Conference on International Conference on Machine Learning","volume":"32","author":"Hutter Frank","year":"2014"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.284"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925974"},{"key":"e_1_2_1_76_1","volume-title":"Proceedings of the IEEE International Conference on Cloud Engineering (IC2E\u201918)","author":"Ishakian V.","year":"2018"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/3286490.3286561"},{"key":"e_1_2_1_78_1","unstructured":"Myeongjae Jeon Shivaram Venkataraman Amar Phanishayee Junjie Qian Wencong Xiao and Fan Yang. 2019. Analysis of large-scale multi-tenant GPU clusters for DNN training workloads. arXiv preprint arXiv:1901.05758.  Myeongjae Jeon Shivaram Venkataraman Amar Phanishayee Junjie Qian Wencong Xiao and Fan Yang. 2019. Analysis of large-scale multi-tenant GPU clusters for DNN training workloads. arXiv preprint arXiv:1901.05758."},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/3190508.3190530"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2654889"},{"key":"e_1_2_1_81_1","volume-title":"Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research), Jennifer Dy and Andreas Krause (Eds.)","volume":"80","author":"Jia Zhihao","year":"2018"},{"key":"e_1_2_1_82_1","volume-title":"Beyond data and model parallelism for deep neural networks. CoRR abs\/1807.05358","author":"Jia Zhihao","year":"2018"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3035933"},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00065"},{"key":"e_1_2_1_85_1","volume-title":"Proceedings of the 26th International Conference on Neural Information Processing Systems","volume":"1","author":"Johnson Rie","year":"2013"},{"key":"e_1_2_1_86_1","volume-title":"Proceedings of the ACM\/IEEE 44th Annual International Symposium on Computer Architecture (ISCA\u201917)","author":"Jouppi N. P."},{"key":"e_1_2_1_87_1","first-page":"I","article-title":"QMDP-net: Deep learning for planning under partial observability","volume":"30","author":"Karkus Peter","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_88_1","volume-title":"Proceedings of the 11th European Conference on Computer Systems (EuroSys\u201916)","author":"Kim Jin Kyu"},{"key":"e_1_2_1_89_1","volume-title":"Pietzuch","author":"Koliousis Alexandros","year":"2019"},{"key":"e_1_2_1_90_1","volume-title":"Proceedings of the NIPS Workshop on Private Multi-Party Machine Learning.","author":"Konec\u0306n\u00fd Jakub","year":"2016"},{"key":"e_1_2_1_91_1","unstructured":"Alex Krizhevsky. 2014. One weird trick for parallelizing convolutional neural networks. Retrieved from http:\/\/arxiv.org\/abs\/1404.5997.  Alex Krizhevsky. 2014. One weird trick for parallelizing convolutional neural networks. Retrieved from http:\/\/arxiv.org\/abs\/1404.5997."},{"key":"e_1_2_1_92_1","volume-title":"Proceedings of the 25th International Conference on Neural Information Processing Systems","volume":"1","author":"Krizhevsky Alex"},{"key":"e_1_2_1_93_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3054775"},{"key":"e_1_2_1_94_1","unstructured":"Intel. Data Management Tailored for Machine Learning Workloads in Kubernetes. Retrieved from https:\/\/www.intel.ai\/kubernetes-volume-controller-kvc-data-management-tailored-for-machine-learning-workloads-in-kubernetes.  Intel. Data Management Tailored for Machine Learning Workloads in Kubernetes. Retrieved from https:\/\/www.intel.ai\/kubernetes-volume-controller-kvc-data-management-tailored-for-machine-learning-workloads-in-kubernetes."},{"key":"e_1_2_1_95_1","volume-title":"Deep learning. Nature 521, 7553","author":"LeCun Yann","year":"2015"},{"key":"e_1_2_1_96_1","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_2_1_97_1","doi-asserted-by":"publisher","DOI":"10.1145\/1815961.1816021"},{"key":"e_1_2_1_98_1","volume-title":"Proceedings of International Conference on Machine Learning (ICML\u201916)","author":"Lee Lee Yun Seong","year":"2016"},{"key":"e_1_2_1_99_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364914549607"},{"key":"e_1_2_1_100_1","volume-title":"Jieyang Chen, Jiajia Li, Xu Liu, Nathan R. Tallent, and Kevin J. Barker.","author":"Li Ang","year":"2019"},{"key":"e_1_2_1_101_1","doi-asserted-by":"publisher","DOI":"10.1145\/2741948.2741965"},{"key":"e_1_2_1_102_1","doi-asserted-by":"publisher","DOI":"10.5555\/2685048.2685095"},{"key":"e_1_2_1_103_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2017.02.006"},{"key":"e_1_2_1_104_1","doi-asserted-by":"publisher","DOI":"10.1145\/3187009.3177737"},{"key":"e_1_2_1_105_1","volume-title":"Advances in Neural Information Processing Systems","author":"Lian Xiangru"},{"key":"e_1_2_1_106_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Lin Yujun","year":"2018"},{"key":"e_1_2_1_107_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2017.07.005"},{"key":"e_1_2_1_108_1","volume-title":"Proceedings of the Deep Learning Day at SIGKDD Conference on Knowledge Discovery and Data Mining (KDD\u201918)","author":"Liu Jiayi","year":"2018"},{"key":"e_1_2_1_109_1","volume-title":"A review of automatic selection methods for machine learning algorithms and hyper-parameter values. Netw. Model. Anal. Health Info. Bioinfo. 5, 1 (23","author":"Luo Gang","year":"2016"},{"key":"e_1_2_1_110_1","volume-title":"Proceedings of the 7th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud\u201915)","author":"Mai Luo","year":"2015"},{"key":"e_1_2_1_111_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807167.1807184"},{"key":"e_1_2_1_112_1","unstructured":"Dominic Masters and Carlo Luschi. 2018. Revisiting small batch training for deep neural networks. Retrieved from arxiv:1804.07612 http:\/\/arxiv.org\/abs\/1804.07612.  Dominic Masters and Carlo Luschi. 2018. Revisiting small batch training for deep neural networks. Retrieved from arxiv:1804.07612 http:\/\/arxiv.org\/abs\/1804.07612."},{"key":"e_1_2_1_113_1","doi-asserted-by":"publisher","DOI":"10.1145\/3154842.3154843"},{"key":"e_1_2_1_114_1","volume-title":"Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS\u201917)","author":"McMahan H. Brendan","year":"2017"},{"key":"e_1_2_1_115_1","first-page":"1","article-title":"MLlib: Machine learning in apache spark","volume":"17","author":"Meng Xiangrui","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_2_1_116_1","volume-title":"Proceedings of the IEEE 33rd International Conference on Data Engineering (ICDE\u201917)","author":"Miao H.","year":"2017"},{"key":"e_1_2_1_117_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Mirhoseini Azalia","year":"2018"},{"key":"e_1_2_1_118_1","volume-title":"Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research), Doina Precup and Yee Whye Teh (Eds.)","volume":"70","author":"Mirhoseini Azalia","year":"2017"},{"key":"e_1_2_1_119_1","volume-title":"Proceedings of the 29th International Conference on Machine Learning (ICML\u201912)","author":"Mnih Volodymyr"},{"key":"e_1_2_1_120_1","volume-title":"Proceedings of International Conference on Learning Representations. arXiv preprint arXiv:1511","author":"Moritz Philipp"},{"key":"e_1_2_1_121_1","volume-title":"Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201918)","author":"Moritz Philipp","year":"2018"},{"key":"e_1_2_1_122_1","unstructured":"Apache MXNet. Distributed Training in MXNet. Retrieved from https:\/\/mxnet.incubator.apache.org\/versions\/master\/faq\/distributed_training.html.  Apache MXNet. Distributed Training in MXNet. Retrieved from https:\/\/mxnet.incubator.apache.org\/versions\/master\/faq\/distributed_training.html."},{"key":"e_1_2_1_123_1","unstructured":"Apache MXNet. Does mxnet support Stale Synchronous Parallel (aka. SSP). Retrieved from https:\/\/github.com\/apache\/incubator-mxnet\/issues\/841.  Apache MXNet. Does mxnet support Stale Synchronous Parallel (aka. SSP). Retrieved from https:\/\/github.com\/apache\/incubator-mxnet\/issues\/841."},{"key":"e_1_2_1_124_1","unstructured":"Apache MXNet. Gradient Compression : mxnet documentation. Retrieved from https:\/\/mxnet.incubator.apache.org\/versions\/master\/faq\/gradient_compression.html.  Apache MXNet. Gradient Compression : mxnet documentation. Retrieved from https:\/\/mxnet.incubator.apache.org\/versions\/master\/faq\/gradient_compression.html."},{"key":"e_1_2_1_125_1","unstructured":"Apache MXNet. MXNet. Retrieved from https:\/\/mxnet.apache.org\/.  Apache MXNet. MXNet. Retrieved from https:\/\/mxnet.apache.org\/."},{"key":"e_1_2_1_126_1","unstructured":"Apache MXNet. MXNet Graph Optimization and Quantization based on subgraph and MKL-DNN. Retrieved from https:\/\/cwiki.apache.org\/confluence\/display\/MXNET\/MXNet+Graph+Optimization+and+Quantization+based+on+subgraph+and+MKL-DNN.  Apache MXNet. MXNet Graph Optimization and Quantization based on subgraph and MKL-DNN. Retrieved from https:\/\/cwiki.apache.org\/confluence\/display\/MXNET\/MXNet+Graph+Optimization+and+Quantization+based+on+subgraph+and+MKL-DNN."},{"key":"e_1_2_1_127_1","volume-title":"Proceedings of the NIPS Workshop on Systems for Machine Learning.","author":"Narayanan Deepak","year":"2018"},{"key":"e_1_2_1_128_1","unstructured":"Tencent. Tencent ncnn. Retrieved from https:\/\/github.com\/Tencent\/ncnn.  Tencent. Tencent ncnn. Retrieved from https:\/\/github.com\/Tencent\/ncnn."},{"key":"e_1_2_1_129_1","unstructured":"Intel. Neon. Retrieved from https:\/\/github.com\/NervanaSystems\/neon.  Intel. Neon. Retrieved from https:\/\/github.com\/NervanaSystems\/neon."},{"key":"e_1_2_1_130_1","doi-asserted-by":"publisher","DOI":"10.1145\/3286490.3286559"},{"key":"e_1_2_1_131_1","doi-asserted-by":"crossref","volume-title":"The Quest for Artificial Intelligence: A History of Ideas and Achievements","author":"Nilsson Nils","DOI":"10.1017\/CBO9780511819346"},{"key":"e_1_2_1_132_1","doi-asserted-by":"publisher","DOI":"10.1145\/3102980.3102998"},{"key":"e_1_2_1_133_1","volume-title":"Proceedings of the NIPS Workshop on Distributed Machine Learning and Matrix Computations.","author":"Noel Cyprien","year":"2014"},{"key":"e_1_2_1_134_1","doi-asserted-by":"publisher","DOI":"10.1145\/2733373.2807410"},{"key":"e_1_2_1_135_1","unstructured":"Kalin Ovtcharov Olatunji Ruwase Joo-Young Kim Jeremy Fowers Karin Strauss and Eric Chung. 2015. Accelerating Deep Convolutional Neural Networks Using Specialized Hardware. Retrieved from https:\/\/www.microsoft.com\/en-us\/research\/publication\/accelerating-deep-convolutional-neural-networks-using-specialized-hardware\/.  Kalin Ovtcharov Olatunji Ruwase Joo-Young Kim Jeremy Fowers Karin Strauss and Eric Chung. 2015. Accelerating Deep Convolutional Neural Networks Using Specialized Hardware. Retrieved from https:\/\/www.microsoft.com\/en-us\/research\/publication\/accelerating-deep-convolutional-neural-networks-using-specialized-hardware\/."},{"key":"e_1_2_1_136_1","doi-asserted-by":"publisher","DOI":"10.1145\/3286490.3286562"},{"key":"e_1_2_1_137_1","unstructured":"PaddlePaddle. PaddlePaddle. Retrieved from http:\/\/paddlepaddle.org\/.  PaddlePaddle. PaddlePaddle. Retrieved from http:\/\/paddlepaddle.org\/."},{"key":"e_1_2_1_138_1","volume-title":"Noh","author":"Park Jay H.","year":"2019"},{"key":"e_1_2_1_139_1","volume-title":"Proceedings of the Conference and Workshop on Neural Information Processing Systems (NIPS-W\u201917)","author":"Paszke Adam","year":"2017"},{"key":"e_1_2_1_140_1","volume-title":"Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12 (Oct.","author":"Pedregosa Fabian","year":"2011"},{"key":"e_1_2_1_141_1","doi-asserted-by":"publisher","DOI":"10.1145\/3190508.3190517"},{"key":"e_1_2_1_142_1","volume-title":"Hoard: A distributed data caching system to accelerate deep-learning training on the cloud.","author":"Pinto Christian","year":"2018"},{"key":"e_1_2_1_143_1","doi-asserted-by":"publisher","DOI":"10.1145\/3234150"},{"key":"e_1_2_1_144_1","unstructured":"OpenMined. OpenMined\/PySyft: A library for encrypted privacy preserving deep learning. Retrieved from https:\/\/github.com\/OpenMined\/PySyft.  OpenMined. OpenMined\/PySyft: A library for encrypted privacy preserving deep learning. Retrieved from https:\/\/github.com\/OpenMined\/PySyft."},{"key":"e_1_2_1_145_1","unstructured":"PyTorch Community. Multiprocessing Best Practies\u2014PyTorch Master Documentation. Retrieved from https:\/\/pytorch.org\/docs\/stable\/notes\/multiprocessing.html.  PyTorch Community. Multiprocessing Best Practies\u2014PyTorch Master Documentation. Retrieved from https:\/\/pytorch.org\/docs\/stable\/notes\/multiprocessing.html."},{"key":"e_1_2_1_146_1","volume-title":"Proceedings of the USENIX Annual Technical Conference (USENIX ATC 18)","author":"Qiao Aurick"},{"key":"e_1_2_1_147_1","unstructured":"Facebook. QNNPACK: Open-source library for optimized mobile deep learning. Retrieved from https:\/\/code.fb.com\/ml-applications\/qnnpack\/.  Facebook. QNNPACK: Open-source library for optimized mobile deep learning. Retrieved from https:\/\/code.fb.com\/ml-applications\/qnnpack\/."},{"key":"e_1_2_1_148_1","doi-asserted-by":"publisher","DOI":"10.1145\/3135974.3135994"},{"key":"e_1_2_1_149_1","volume-title":"Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In Advances in Neural Information Processing Systems","author":"Recht Benjamin","year":"2011"},{"key":"e_1_2_1_150_1","doi-asserted-by":"publisher","DOI":"10.1145\/3199605"},{"key":"e_1_2_1_151_1","unstructured":"Sebastian Ruder. 2016. An overview of gradient descent optimization algorithms. Retrieved from http:\/\/arxiv.org\/abs\/1609.04747.  Sebastian Ruder. 2016. An overview of gradient descent optimization algorithms. Retrieved from http:\/\/arxiv.org\/abs\/1609.04747."},{"key":"e_1_2_1_152_1","doi-asserted-by":"publisher","DOI":"10.1038\/323533a0"},{"key":"e_1_2_1_153_1","unstructured":"Theo Ryffel Andrew Trask Morten Dahl Bobby Wagner Jason Mancuso Daniel Rueckert and Jonathan Passerat-Palmbach. 2018. A generic framework for privacy preserving deep learning. Retrieved from http:\/\/arxiv.org\/abs\/1811.04017.  Theo Ryffel Andrew Trask Morten Dahl Bobby Wagner Jason Mancuso Daniel Rueckert and Jonathan Passerat-Palmbach. 2018. A generic framework for privacy preserving deep learning. Retrieved from http:\/\/arxiv.org\/abs\/1811.04017."},{"key":"e_1_2_1_154_1","unstructured":"Christopher De Sa Megan Leszczynski Jian Zhang Alana Marzoev Christopher R. Aberger Kunle Olukotun and Christopher R\u00e9. 2018. High-accuracy low-precision training. Retrieved from http:\/\/arxiv.org\/abs\/1803.03383.  Christopher De Sa Megan Leszczynski Jian Zhang Alana Marzoev Christopher R. Aberger Kunle Olukotun and Christopher R\u00e9. 2018. High-accuracy low-precision training. Retrieved from http:\/\/arxiv.org\/abs\/1803.03383."},{"key":"e_1_2_1_155_1","unstructured":"SAS. SAS. Retrieved from https:\/\/www.sas.com\/.  SAS. SAS. Retrieved from https:\/\/www.sas.com\/."},{"key":"e_1_2_1_156_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2014.09.003"},{"key":"e_1_2_1_157_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0140525X00005756"},{"key":"e_1_2_1_158_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2945397"},{"key":"e_1_2_1_159_1","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2014-274"},{"key":"e_1_2_1_160_1","volume-title":"Horovod: Fast and easy distributed deep learning in TensorFlow.","author":"Sergeev Alexander","year":"2018"},{"key":"e_1_2_1_161_1","volume-title":"Mesh-tensorflow: Deep learning for supercomputers. In Advances in Neural Information Processing Systems","author":"Shazeer Noam","year":"2018"},{"key":"e_1_2_1_162_1","doi-asserted-by":"publisher","DOI":"10.1145\/2810103.2813687"},{"key":"e_1_2_1_163_1","unstructured":"Alex Smola. [n.d.]. What is the Parameter Server? Retrieved from https:\/\/www.quora.com\/What-is-the-Parameter-Server.  Alex Smola. [n.d.]. What is the Parameter Server? Retrieved from https:\/\/www.quora.com\/What-is-the-Parameter-Server."},{"key":"e_1_2_1_164_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920931"},{"key":"e_1_2_1_165_1","volume-title":"Advances in Neural Information Processing Systems 26","author":"Socher Richard"},{"key":"e_1_2_1_166_1","doi-asserted-by":"publisher","DOI":"10.1145\/2806777.2806945"},{"key":"e_1_2_1_167_1","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2015-354"},{"key":"e_1_2_1_168_1","unstructured":"Sainbayar Sukhbaatar and Rob Fergus. 2014. Learning from noisy labels with deep neural networks. Retrieved from http:\/\/arxiv.org\/abs\/1406.2080.  Sainbayar Sukhbaatar and Rob Fergus. 2014. Learning from noisy labels with deep neural networks. Retrieved from http:\/\/arxiv.org\/abs\/1406.2080."},{"key":"e_1_2_1_169_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_2_1_170_1","volume-title":"Proceedings of the USENIX Workshop on Hot Topics in Edge Computing (HotEdge'18)","author":"Tao Zeyi","year":"2018"},{"key":"e_1_2_1_171_1","unstructured":"TensorFlow. Distributed Training in TensorFlow. Retrieved from https:\/\/www.tensorflow.org\/guide\/distribute_strategy. https:\/\/www.tensorflow.org\/guide\/distribute_strategy.  TensorFlow. Distributed Training in TensorFlow. Retrieved from https:\/\/www.tensorflow.org\/guide\/distribute_strategy. https:\/\/www.tensorflow.org\/guide\/distribute_strategy."},{"key":"e_1_2_1_172_1","unstructured":"TensorFlow. Post-training quantization. Retrieved from https:\/\/www.tensorflow.org\/lite\/performance\/post_training_quantization.  TensorFlow. Post-training quantization. Retrieved from https:\/\/www.tensorflow.org\/lite\/performance\/post_training_quantization."},{"key":"e_1_2_1_173_1","unstructured":"TensorFlow. TensorFlow Federated. Retrieved from https:\/\/www.tensorflow.org\/federated.  TensorFlow. TensorFlow Federated. Retrieved from https:\/\/www.tensorflow.org\/federated."},{"key":"e_1_2_1_174_1","volume-title":"Proceedings of Workshop on Machine Learning Systems (LearningSys) in The 29th Annual Conference on Neural Information Processing Systems (NIPS\u201915)","author":"Tokui Seiya","year":"2015"},{"key":"e_1_2_1_175_1","doi-asserted-by":"publisher","DOI":"10.1145\/79173.79181"},{"key":"e_1_2_1_176_1","volume-title":"Proceedings of the Deep Learning and Unsupervised Feature Learning Workshop (NIPS\u201911)","author":"Vanhoucke Vincent"},{"key":"e_1_2_1_177_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939502.2939516"},{"key":"e_1_2_1_178_1","doi-asserted-by":"publisher","DOI":"10.1145\/2523616.2523633"},{"key":"e_1_2_1_179_1","doi-asserted-by":"publisher","DOI":"10.1145\/2741948.2741964"},{"key":"e_1_2_1_180_1","unstructured":"Abhinav Vishnu Charles Siegel and Jeffrey Daily. 2016. Distributed TensorFlow with MPI. Retrieved from http:\/\/arxiv.org\/abs\/1603.02339.  Abhinav Vishnu Charles Siegel and Jeffrey Daily. 2016. Distributed TensorFlow with MPI. Retrieved from http:\/\/arxiv.org\/abs\/1603.02339."},{"key":"e_1_2_1_181_1","first-page":"3","article-title":"DLAU: A scalable deep-learning accelerator unit on FPGA","volume":"36","author":"Wang C.","year":"2017","journal-title":"IEEE Trans. Comput.-Aid. Design Integr. Circ. Syst."},{"key":"e_1_2_1_182_1","volume-title":"Proceedings of the NIPS Workshop on Distributed Machine Learning and Matrix Computations.","author":"Wang Minjie","year":"2014"},{"key":"e_1_2_1_183_1","doi-asserted-by":"publisher","DOI":"10.1145\/2637166.2637231"},{"key":"e_1_2_1_184_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274808.3274828"},{"key":"e_1_2_1_185_1","doi-asserted-by":"publisher","DOI":"10.1145\/3003665.3003669"},{"key":"e_1_2_1_186_1","doi-asserted-by":"publisher","DOI":"10.1145\/2987550.2987586"},{"key":"e_1_2_1_187_1","volume-title":"Proceedings of the 6th ACM Symposium on Cloud Computing (SoCC\u201915)","author":"Wei Jinliang"},{"key":"e_1_2_1_188_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2742793"},{"key":"e_1_2_1_189_1","unstructured":"wekadl4j [n.d.]. Retrieved from https:\/\/github.com\/Waikato\/wekaDeeplearning4j.  wekadl4j [n.d.]. Retrieved from https:\/\/github.com\/Waikato\/wekaDeeplearning4j."},{"key":"e_1_2_1_190_1","volume-title":"Terngrad: Ternary gradients to reduce communication in distributed deep learning. In Advances in Neural Information Processing Systems","author":"Wen Wei","year":"2017"},{"key":"e_1_2_1_191_1","unstructured":"Yonghui Wu Mike Schuster Zhifeng Chen Quoc V. Le Mohammad Norouzi Wolfgang Macherey Maxim Krikun Yuan Cao Qin Gao Klaus Macherey Jeff Klingner Apurva Shah Melvin Johnson Xiaobing Liu Lukasz Kaiser Stephan Gouws Yoshikiyo Kato Taku Kudo Hideto Kazawa Keith Stevens George Kurian Nishant Patil Wei Wang Cliff Young Jason Smith Jason Riesa Alex Rudnick Oriol Vinyals Greg Corrado Macduff Hughes and Jeffrey Dean. 2016. Google\u2019s neural machine translation system: Bridging the gap between human and machine translation. Retrieved from http:\/\/arxiv.org\/abs\/1609.08144.  Yonghui Wu Mike Schuster Zhifeng Chen Quoc V. Le Mohammad Norouzi Wolfgang Macherey Maxim Krikun Yuan Cao Qin Gao Klaus Macherey Jeff Klingner Apurva Shah Melvin Johnson Xiaobing Liu Lukasz Kaiser Stephan Gouws Yoshikiyo Kato Taku Kudo Hideto Kazawa Keith Stevens George Kurian Nishant Patil Wei Wang Cliff Young Jason Smith Jason Riesa Alex Rudnick Oriol Vinyals Greg Corrado Macduff Hughes and Jeffrey Dean. 2016. Google\u2019s neural machine translation system: Bridging the gap between human and machine translation. Retrieved from http:\/\/arxiv.org\/abs\/1609.08144."},{"key":"e_1_2_1_192_1","volume-title":"Yu","author":"Wu Zonghan","year":"2019"},{"key":"e_1_2_1_193_1","unstructured":"xgboost [n.d.]. XGBoost. Retrieved from https:\/\/xgboost.ai\/.  xgboost [n.d.]. XGBoost. Retrieved from https:\/\/xgboost.ai\/."},{"key":"e_1_2_1_194_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201915)","author":"Xiao Tong","year":"2015"},{"key":"e_1_2_1_195_1","volume-title":"Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201918)","author":"Xiao Wencong","year":"2018"},{"key":"e_1_2_1_196_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBDATA.2015.2472014"},{"key":"e_1_2_1_197_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783270"},{"key":"e_1_2_1_198_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCI.2018.2840738"},{"key":"e_1_2_1_199_1","doi-asserted-by":"publisher","DOI":"10.1145\/3190508.3190551"},{"key":"e_1_2_1_200_1","volume-title":"Presented as part of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201912). USENIX","author":"Zaharia Matei"},{"key":"e_1_2_1_201_1","doi-asserted-by":"publisher","DOI":"10.1145\/2966986.2967011"},{"key":"e_1_2_1_202_1","doi-asserted-by":"publisher","DOI":"10.1145\/2684746.2689060"},{"key":"e_1_2_1_203_1","volume-title":"Proceedings of the IEEE 38th International Conference on Distributed Computing Systems (ICDCS\u201918)","author":"Zhang C.","year":"2018"},{"key":"e_1_2_1_204_1","volume-title":"Proceedings of the IEEE 16th International Conference on Data Mining (ICDM\u201916)","author":"Zhang H.","year":"2016"},{"key":"e_1_2_1_205_1","volume-title":"Proceedings of the Symposium on Cloud Computing (SoCC\u201917)","author":"Zhang Haoyu"},{"key":"e_1_2_1_206_1","volume-title":"Proceedings of the USENIX Conference on Usenix Annual Technical Conference (USENIX ATC\u201917)","author":"Zhang Hao"},{"key":"e_1_2_1_207_1","volume-title":"Proceedings of the SIAM International Conference on Data Mining. 785--793","author":"Zhao Huasha","year":"1972"},{"key":"e_1_2_1_208_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.485"},{"key":"e_1_2_1_209_1","unstructured":"Shuchang Zhou Zekun Ni Xinyu Zhou He Wen Yuxin Wu and Yuheng Zou. 2016. DoReFa-Net: Training low bitwidth convolutional neural networks with low bitwidth gradients. Retrieved from http:\/\/arxiv.org\/abs\/1606.06160.  Shuchang Zhou Zekun Ni Xinyu Zhou He Wen Yuxin Wu and Yuheng Zou. 2016. DoReFa-Net: Training low bitwidth convolutional neural networks with low bitwidth gradients. Retrieved from http:\/\/arxiv.org\/abs\/1606.06160."},{"key":"e_1_2_1_210_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Zoph Barret","year":"2017"},{"key":"e_1_2_1_211_1","doi-asserted-by":"publisher","DOI":"10.14778\/2733004.2733082"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3363554","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3363554","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:44:25Z","timestamp":1750189465000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3363554"}},"subtitle":["Challenges, Techniques, and Tools"],"short-title":[],"issued":{"date-parts":[[2020,2,6]]},"references-count":211,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,1,31]]}},"alternative-id":["10.1145\/3363554"],"URL":"https:\/\/doi.org\/10.1145\/3363554","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,2,6]]},"assertion":[{"value":"2019-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-09-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-02-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}