{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T18:39:12Z","timestamp":1768415952893,"version":"3.49.0"},"reference-count":61,"publisher":"Association for Computing Machinery (ACM)","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:p>Deep neural networks (DNNs) are becoming increasingly deeper, wider, and non-linear due to the growing demands on prediction accuracy and analysis quality. Training wide and deep neural networks require large amounts of storage resources such as memory because the intermediate activation data must be saved in the memory during forward propagation and then restored for backward propagation. However, state-of-the-art accelerators such as GPUs are only equipped with very limited memory capacities due to hardware design constraints, which significantly limits the maximum batch size and hence performance speedup when training large-scale DNNs. Traditional memory saving techniques either suffer from performance overhead or are constrained by limited interconnect bandwidth or specific interconnect technology.<\/jats:p>\n          <jats:p>In this paper, we propose a novel memory-efficient CNN training framework (called COMET) that leverages error-bounded lossy compression to significantly reduce the memory requirement for training in order to allow training larger models or to accelerate training. Our framework purposely adopts error-bounded lossy compression with a strict error-controlling mechanism. Specifically, we perform a theoretical analysis on the compression error propagation from the altered activation data to the gradients, and empirically investigate the impact of altered gradients over the training process. Based on these analyses, we optimize the error-bounded lossy compression and propose an adaptive error-bound control scheme for activation data compression. Experiments demonstrate that our proposed framework can significantly reduce the training memory consumption by up to 13.5X over the baseline training and 1.8X over another state-of-the-art compression-based framework, respectively, with little or no accuracy loss.<\/jats:p>","DOI":"10.14778\/3503585.3503597","type":"journal-article","created":{"date-parts":[[2022,4,14]],"date-time":"2022-04-14T22:18:07Z","timestamp":1649974687000},"page":"886-899","source":"Crossref","is-referenced-by-count":23,"title":["COMET"],"prefix":"10.14778","volume":"15","author":[{"given":"Sian","family":"Jin","sequence":"first","affiliation":[{"name":"Washington State University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chengming","family":"Zhang","sequence":"additional","affiliation":[{"name":"Washington State University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xintong","family":"Jiang","sequence":"additional","affiliation":[{"name":"McGill University, Montreal, QC, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yunhe","family":"Feng","sequence":"additional","affiliation":[{"name":"University of Washington"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hui","family":"Guan","sequence":"additional","affiliation":[{"name":"University of Massachusetts"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guanpeng","family":"Li","sequence":"additional","affiliation":[{"name":"University of Iowa"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuaiwen Leon","family":"Song","sequence":"additional","affiliation":[{"name":"University of Sydney, Sydney, NSW, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dingwen","family":"Tao","sequence":"additional","affiliation":[{"name":"Washington State University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,4,14]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467","author":"Abadi Mart\u00edn","year":"2016","unstructured":"Mart\u00edn Abadi , Ashish Agarwal , Paul Barham , Eugene Brevdo , Zhifeng Chen , Craig Citro , Greg S Corrado , Andy Davis , Jeffrey Dean , Matthieu Devin , 2016 . Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016). Mart\u00edn Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. 2016. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016)."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3126908.3126933"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3320060"},{"key":"e_1_2_1_4_1","unstructured":"Bridge-2 system. 2020. https:\/\/www.psc.edu\/resources\/bridges-2\/. (Accessed on 12\/22\/2021).  Bridge-2 system. 2020. https:\/\/www.psc.edu\/resources\/bridges-2\/. (Accessed on 12\/22\/2021)."},{"key":"e_1_2_1_5_1","volume-title":"Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174","author":"Chen Tianqi","year":"2016","unstructured":"Tianqi Chen , Bing Xu , Chiyuan Zhang , and Carlos Guestrin . 2016. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174 ( 2016 ). Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. 2016. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174 (2016)."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA45697.2020.00080"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390177"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2901318.2901323"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2016.11"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00957"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2005.6"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA45697.2020.00075"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/MM.2017.37"},{"key":"e_1_2_1_15_1","unstructured":"Aidan N Gomez Mengye Ren Raquel Urtasun and Roger B Grosse. 2017. The reversible residual network: Backpropagation without storing activations. In Advances in neural information processing systems. 2214--2224.  Aidan N Gomez Mengye Ren Raquel Urtasun and Roger B Grosse. 2017. The reversible residual network: Backpropagation without storing activations. In Advances in neural information processing systems. 2214--2224."},{"key":"e_1_2_1_16_1","volume-title":"large minibatch sgd: Training imagenet in 1 hour. arXiv preprint arXiv.1706.02677","author":"Goyal Priya","year":"2017","unstructured":"Priya Goyal , Piotr Doll\u00e1r , Ross Girshick , Pieter Noordhuis , Lukasz Wesolowski , Aapo Kyrola , Andrew Tulloch , Yangqing Jia , and Kaiming He. 2017. Accurate , large minibatch sgd: Training imagenet in 1 hour. arXiv preprint arXiv.1706.02677 ( 2017 ). Priya Goyal, Piotr Doll\u00e1r, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. 2017. Accurate, large minibatch sgd: Training imagenet in 1 hour. arXiv preprint arXiv.1706.02677 (2017)."},{"key":"e_1_2_1_17_1","unstructured":"NVIDIA V100 TENSOR CORE GPU. 2020. https:\/\/www.nvidia.com\/en-us\/data-center\/v100\/. (Accessed on 12\/22\/2021).  NVIDIA V100 TENSOR CORE GPU. 2020. https:\/\/www.nvidia.com\/en-us\/data-center\/v100\/. (Accessed on 12\/22\/2021)."},{"key":"e_1_2_1_18_1","volume-title":"Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149","author":"Han Song","year":"2015","unstructured":"Song Han , Huizi Mao , and William J Dally . 2015. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 ( 2015 ). Song Han, Huizi Mao, and William J Dally. 2015. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 (2015)."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_20_1","volume-title":"Neural networks for perception","author":"Hecht-Nielsen Robert","unstructured":"Robert Hecht-Nielsen . 1992. Theory of the backpropagation neural network . In Neural networks for perception . Elsevier , 65--93. Robert Hecht-Nielsen. 1992. Theory of the backpropagation neural network. In Neural networks for perception. Elsevier, 65--93."},{"key":"e_1_2_1_21_1","volume-title":"Gpipe: Efficient training of giant neural networks using pipeline parallelism. In Advances in neural information processing systems. 103--112.","author":"Huang Yanping","year":"2019","unstructured":"Yanping Huang , Youlong Cheng , Ankur Bapna , Orhan Firat , Dehao Chen , Mia Chen , HyoukJoong Lee , Jiquan Ngiam , Quoc V Le , Yonghui Wu , 2019 . Gpipe: Efficient training of giant neural networks using pipeline parallelism. In Advances in neural information processing systems. 103--112. Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Dehao Chen, Mia Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, Yonghui Wu, et al. 2019. Gpipe: Efficient training of giant neural networks using pipeline parallelism. In Advances in neural information processing systems. 103--112."},{"key":"e_1_2_1_22_1","volume-title":"Computer Graphics Forum","author":"Ibarria Lawrence","unstructured":"Lawrence Ibarria , Peter Lindstrom , Jarek Rossignac , and Andrzej Szymczak . 2003. Out-of-core compression and decompression of large n-dimensional scalar fields . In Computer Graphics Forum , Vol. 22 . Wiley Online Library , 343--348. Lawrence Ibarria, Peter Lindstrom, Jarek Rossignac, and Andrzej Szymczak. 2003. Out-of-core compression and decompression of large n-dimensional scalar fields. In Computer Graphics Forum, Vol. 22. Wiley Online Library, 343--348."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2654889"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3243904"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3307681.3326608"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3437801.3441597"},{"key":"e_1_2_1_27_1","volume-title":"Novel Dataset for Fine-Grained Image Categorization. In First Workshop on FineGrained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition","author":"Khosla Aditya","year":"2011","unstructured":"Aditya Khosla , Nityananda Jayadevaprakash , Bangpeng Yao , and Li Fei-Fei . 2011 . Novel Dataset for Fine-Grained Image Categorization. In First Workshop on FineGrained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition . Colorado Springs, CO. Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. 2011. Novel Dataset for Fine-Grained Image Categorization. In First Workshop on FineGrained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, CO."},{"key":"e_1_2_1_28_1","unstructured":"Alex Krizhevsky Ilya Sutskever and Geoffrey EHinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.  Alex Krizhevsky Ilya Sutskever and Geoffrey EHinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2018.00054"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2017.101"},{"key":"e_1_2_1_31_1","volume-title":"Visualizing the loss landscape of neural nets. arXiv preprint arXiv:1712.09913","author":"Li Hao","year":"2017","unstructured":"Hao Li , Zheng Xu , Gavin Taylor , Christoph Studer , and Tom Goldstein . 2017. Visualizing the loss landscape of neural nets. arXiv preprint arXiv:1712.09913 ( 2017 ). Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. 2017. Visualizing the loss landscape of neural nets. arXiv preprint arXiv:1712.09913 (2017)."},{"key":"e_1_2_1_32_1","doi-asserted-by":"crossref","unstructured":"Xin Liang Sheng Di Dingwen Tao Sihuan Li Shaomeng Li Hanqi Guo Zizhong Chen and Franck Cappello. 2018. Error-Controlled Lossy Compression Optimized for High Compression Ratios of Scientific Datasets. (2018).  Xin Liang Sheng Di Dingwen Tao Sihuan Li Shaomeng Li Hanqi Guo Zizhong Chen and Franck Cappello. 2018. Error-Controlled Lossy Compression Optimized for High Compression Ratios of Scientific Datasets. (2018).","DOI":"10.1109\/BigData.2018.8622520"},{"key":"e_1_2_1_33_1","volume-title":"Deep gradient compression: Reducing the communication bandwidth for distributed training. arXiv preprint arXiv:1712.01887","author":"Lin Yujun","year":"2017","unstructured":"Yujun Lin , Song Han , Huizi Mao , Yu Wang , and William J Dally . 2017. Deep gradient compression: Reducing the communication bandwidth for distributed training. arXiv preprint arXiv:1712.01887 ( 2017 ). Yujun Lin, Song Han, Huizi Mao, Yu Wang, and William J Dally. 2017. Deep gradient compression: Reducing the communication bandwidth for distributed training. arXiv preprint arXiv:1712.01887 (2017)."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346458"},{"key":"e_1_2_1_35_1","volume-title":"Error distributions of lossy floating-point compressors","author":"Lindstrom Peter","unstructured":"Peter Lindstrom . 2017. Error distributions of lossy floating-point compressors . Technical Report. Lawrence Livermore National Lab.(LLNL), Livermore, CA (United States) . Peter Lindstrom. 2017. Error distributions of lossy floating-point compressors. Technical Report. Lawrence Livermore National Lab.(LLNL), Livermore, CA (United States)."},{"key":"e_1_2_1_36_1","unstructured":"Longhorn subsystem. 2020. https:\/\/www.tacc.utexas.edu\/systems\/longhorn. (Accessed on 12\/22\/2021).  Longhorn subsystem. 2020. https:\/\/www.tacc.utexas.edu\/systems\/longhorn. (Accessed on 12\/22\/2021)."},{"key":"e_1_2_1_37_1","unstructured":"Paulius Micikevicius Sharan Narang Jonah Alben Gregory Diamos Erich Elsen David Garcia Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh etal 2017. Mixed precision training. arXiv preprint arXiv:1710.03740 (2017).  Paulius Micikevicius Sharan Narang Jonah Alben Gregory Diamos Erich Elsen David Garcia Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh et al. 2017. Mixed precision training. arXiv preprint arXiv:1710.03740 (2017)."},{"key":"e_1_2_1_38_1","unstructured":"MIG. 2020. NVIDIA Multi-Instance GPU. https:\/\/www.nvidia.com\/en-us\/technologies\/multi-instance-gpu\/. (Accessed on 12\/22\/2021).  MIG. 2020. NVIDIA Multi-Instance GPU. https:\/\/www.nvidia.com\/en-us\/technologies\/multi-instance-gpu\/. (Accessed on 12\/22\/2021)."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.14778\/3407790.3407816"},{"key":"e_1_2_1_40_1","volume-title":"Pytorch: An imperative style, high-performance deep learning library. In Advances in neural information processing systems. 8026--8037.","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke , Sam Gross , Francisco Massa , Adam Lerer , James Bradbury , Gregory Chanan , Trevor Killeen , Zeming Lin , Natalia Gimelshein , Luca Antiga , 2019 . Pytorch: An imperative style, high-performance deep learning library. In Advances in neural information processing systems. 8026--8037. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. In Advances in neural information processing systems. 8026--8037."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.5555\/3433701.3433826"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.5555\/3195638.3195660"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2018.00017"},{"key":"e_1_2_1_44_1","volume-title":"An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747","author":"Ruder Sebastian","year":"2016","unstructured":"Sebastian Ruder . 2016. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747 ( 2016 ). Sebastian Ruder. 2016. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747 (2016)."},{"key":"e_1_2_1_45_1","volume-title":"Horovod: fast and easy distributed deep learning in TensorFlow. arXiv preprint arXiv:1802.05799","author":"Sergeev Alexander","year":"2018","unstructured":"Alexander Sergeev and Mike Del Balso . 2018. Horovod: fast and easy distributed deep learning in TensorFlow. arXiv preprint arXiv:1802.05799 ( 2018 ). Alexander Sergeev and Mike Del Balso. 2018. Horovod: fast and easy distributed deep learning in TensorFlow. arXiv preprint arXiv:1802.05799 (2018)."},{"key":"e_1_2_1_46_1","volume-title":"Measuring the effects of data parallelism on neural network training. arXiv preprint arXiv:1811.03600","author":"Shallue Christopher J","year":"2018","unstructured":"Christopher J Shallue , Jaehoon Lee , Joseph Antognini , Jascha Sohl-Dickstein , Roy Frostig , and George E Dahl . 2018. Measuring the effects of data parallelism on neural network training. arXiv preprint arXiv:1811.03600 ( 2018 ). Christopher J Shallue, Jaehoon Lee, Joseph Antognini, Jascha Sohl-Dickstein, Roy Frostig, and George E Dahl. 2018. Measuring the effects of data parallelism on neural network training. arXiv preprint arXiv:1811.03600 (2018)."},{"key":"e_1_2_1_47_1","volume-title":"Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman . 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 ( 2014 ). Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)."},{"key":"e_1_2_1_48_1","volume-title":"Super-convergence: Very fast training of neural networks using large learning rates. In Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications","author":"Smith Leslie N","year":"2019","unstructured":"Leslie N Smith and Nicholay Topin . 2019 . Super-convergence: Very fast training of neural networks using large learning rates. In Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications , Vol. 11006 . International Society for Optics and Photonics , 1100612. Leslie N Smith and Nicholay Topin. 2019. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, Vol. 11006. International Society for Optics and Photonics, 1100612."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.14529\/jsfi140205"},{"key":"e_1_2_1_50_1","unstructured":"Sum of normally distributed random variables. [n. d.]. https:\/\/en.wikipedia.org\/wiki\/Sum_of_normally_distributed_random_variables. (Accessed on 12\/22\/2021).  Sum of normally distributed random variables. [n. d.]. https:\/\/en.wikipedia.org\/wiki\/Sum_of_normally_distributed_random_variables. (Accessed on 12\/22\/2021)."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_2_1_52_1","volume-title":"International Conference on Machine Learning. PMLR, 6105--6114","author":"Tan Mingxing","year":"2019","unstructured":"Mingxing Tan and Quoc Le . 2019 . Efficientnet: Rethinking model scaling for convolutional neural networks . In International Conference on Machine Learning. PMLR, 6105--6114 . Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning. PMLR, 6105--6114."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2017.115"},{"key":"e_1_2_1_54_1","volume-title":"JPEG2000 image compression fundamentals, standards and practice: image compression fundamentals, standards and practice","author":"Taubman David","unstructured":"David Taubman and Michael Marcellin . 2012. JPEG2000 image compression fundamentals, standards and practice: image compression fundamentals, standards and practice . Vol. 642 . Springer Science & Business Media . David Taubman and Michael Marcellin. 2012. JPEG2000 image compression fundamentals, standards and practice: image compression fundamentals, standards and practice. Vol. 642. Springer Science & Business Media."},{"key":"e_1_2_1_55_1","volume-title":"Robert Underwood, Sian Jin, Xin Liang, Jon Calhoun, Dingwen Tao, and Franck Cappello.","author":"Tian Jiannan","year":"2020","unstructured":"Jiannan Tian , Sheng Di , Kai Zhao , Cody Rivera , Megan Hickman Fulp , Robert Underwood, Sian Jin, Xin Liang, Jon Calhoun, Dingwen Tao, and Franck Cappello. 2020 . cuSZ: An Efficient GPU-Based Error-Bounded Lossy Compression Framework for Scientific Data . (2020), 3--15. Jiannan Tian, Sheng Di, Kai Zhao, Cody Rivera, Megan Hickman Fulp, Robert Underwood, Sian Jin, Xin Liang, Jon Calhoun, Dingwen Tao, and Franck Cappello. 2020. cuSZ: An Efficient GPU-Based Error-Bounded Lossy Compression Framework for Scientific Data. (2020), 3--15."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/30.125072"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783273"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3178487.3178491"},{"key":"e_1_2_1_59_1","series-title":"Journal of Physics: Conference Series","volume-title":"An overview of overfitting and its solutions","author":"Ying Xue","year":"2022","unstructured":"Xue Ying . 2019. An overview of overfitting and its solutions . In Journal of Physics: Conference Series , Vol. 1168 . IOP Publishing , 02 2022 . Xue Ying. 2019. An overview of overfitting and its solutions. In Journal of Physics: Conference Series, Vol. 1168. IOP Publishing, 022022."},{"key":"e_1_2_1_60_1","volume-title":"Scaling sgd batch size to 32k for imagenet training. arXiv preprint arXiv:1708.03888 6","author":"You Yang","year":"2017","unstructured":"Yang You , Igor Gitman , and Boris Ginsburg . 2017. Scaling sgd batch size to 32k for imagenet training. arXiv preprint arXiv:1708.03888 6 ( 2017 ). Yang You, Igor Gitman, and Boris Ginsburg. 2017. Scaling sgd batch size to 32k for imagenet training. arXiv preprint arXiv:1708.03888 6 (2017)."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3295500.3356137"},{"key":"e_1_2_1_62_1","unstructured":"Zstandard. 2020. http:\/\/facebook.github.io\/zstd\/. (Accessed on 12\/22\/2021).  Zstandard. 2020. http:\/\/facebook.github.io\/zstd\/. (Accessed on 12\/22\/2021)."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3503585.3503597","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T10:29:39Z","timestamp":1672223379000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3503585.3503597"}},"subtitle":["a novel memory-efficient deep learning training framework by using error-bounded lossy compression"],"short-title":[],"issued":{"date-parts":[[2021,12]]},"references-count":61,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["10.14778\/3503585.3503597"],"URL":"https:\/\/doi.org\/10.14778\/3503585.3503597","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2021,12]]}}}