{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2022,11,3]],"date-time":"2022-11-03T10:55:07Z","timestamp":1667472907910},"publisher-location":"New York, NY, USA","reference-count":46,"publisher":"ACM","funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2110259"]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,7,26]]},"DOI":"10.1145\/3466772.3467056","type":"proceedings-article","created":{"date-parts":[[2021,6,29]],"date-time":"2021-06-29T10:38:19Z","timestamp":1624963099000},"source":"Crossref","is-referenced-by-count":1,"title":["GT-STORM"],"prefix":"10.1145","author":[{"given":"Xin","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Statistics, Iowa State University, Ames, IA, U.S.A"}]},{"given":"Jia","family":"Liu","sequence":"additional","affiliation":[{"name":"Dept. of Electrical and Computer Engineering, The Ohio State University, Columbus, OH, U.S.A"}]},{"given":"Zhengyuan","family":"Zhu","sequence":"additional","affiliation":[{"name":"Department of Statistics, Iowa State University, Ames, IA, U.S.A"}]},{"given":"Elizabeth Serena","family":"Bentley","sequence":"additional","affiliation":[{"name":"Air Force Research Laboratory, Information Directorate, Rome, NY, U.S.A"}]}],"member":"320","published-online":{"date-parts":[[2021,7,26]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1014052.1014097"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1137\/16M1080173"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2012.2219061"},{"key":"e_1_3_2_1_4_1","volume-title":"Distributed Learning in the Non-Convex World: From Batch to Streaming Data, and Beyond. arXiv preprint arXiv:2001.04786","author":"Chang Tsung-Hui","year":"2020","unstructured":"Tsung-Hui Chang , Mingyi Hong , Hoi-To Wai , Xinwei Zhang , and Songtao Lu. 2020. Distributed Learning in the Non-Convex World: From Batch to Streaming Data, and Beyond. arXiv preprint arXiv:2001.04786 ( 2020 ). Tsung-Hui Chang, Mingyi Hong, Hoi-To Wai, Xinwei Zhang, and Songtao Lu. 2020. Distributed Learning in the Non-Convex World: From Batch to Streaming Data, and Beyond. arXiv preprint arXiv:2001.04786 (2020)."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2014.2385045"},{"key":"e_1_3_2_1_6_1","unstructured":"Ashok Cutkosky and Francesco Orabona. 2019. Momentum-based variance reduction in non-convex SGD. In Advances in Neural Information Processing Systems. 15210--15219. Ashok Cutkosky and Francesco Orabona. 2019. Momentum-based variance reduction in non-convex SGD. In Advances in Neural Information Processing Systems. 15210--15219."},{"key":"e_1_3_2_1_7_1","unstructured":"Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Marc'aurelio Ranzato Andrew Senior Paul Tucker Ke Yang etal 2012. Large scale distributed deep networks. In Advances in neural information processing systems. 1223--1231. Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Marc'aurelio Ranzato Andrew Senior Paul Tucker Ke Yang et al. 2012. Large scale distributed deep networks. In Advances in neural information processing systems. 1223--1231."},{"key":"e_1_3_2_1_8_1","volume-title":"SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives. In Advances in neural information processing systems. 1646--1654.","author":"Defazio Aaron","year":"2014","unstructured":"Aaron Defazio , Francis Bach , and Simon Lacoste-Julien . 2014 . SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives. In Advances in neural information processing systems. 1646--1654. Aaron Defazio, Francis Bach, and Simon Lacoste-Julien. 2014. SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives. In Advances in neural information processing systems. 1646--1654."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSIPN.2016.2524588"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2017.2666776"},{"key":"e_1_3_2_1_11_1","volume-title":"Zhouchen Lin, and Tong Zhang.","author":"Fang Cong","year":"2018","unstructured":"Cong Fang , Chris Junchi Li , Zhouchen Lin, and Tong Zhang. 2018 . Spider : Near-optimal non-convex optimization via stochastic path-integrated differential estimator. In Advances in Neural Information Processing Systems . 689--699. Cong Fang, Chris Junchi Li, Zhouchen Lin, and Tong Zhang. 2018. Spider: Near-optimal non-convex optimization via stochastic path-integrated differential estimator. In Advances in Neural Information Processing Systems. 689--699."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1137\/120880811"},{"key":"e_1_3_2_1_13_1","unstructured":"Zhanhong Jiang Aditya Balu Chinmay Hegde and Soumik Sarkar. 2017. Collaborative deep learning in fixed topology networks. In Advances in Neural Information Processing Systems. 5904--5914. Zhanhong Jiang Aditya Balu Chinmay Hegde and Soumik Sarkar. 2017. Collaborative deep learning in fixed topology networks. In Advances in Neural Information Processing Systems. 5904--5914."},{"key":"e_1_3_2_1_14_1","volume-title":"On Consensus-Disagreement Tradeoff in Distributed Optimization. In 2018 Annual American Control Conference (ACC). IEEE, 571--576","author":"Jiang Zhanhong","year":"2018","unstructured":"Zhanhong Jiang , Kushal Mukherjee , and Soumik Sarkar . 2018 . On Consensus-Disagreement Tradeoff in Distributed Optimization. In 2018 Annual American Control Conference (ACC). IEEE, 571--576 . Zhanhong Jiang, Kushal Mukherjee, and Soumik Sarkar. 2018. On Consensus-Disagreement Tradeoff in Distributed Optimization. In 2018 Annual American Control Conference (ACC). IEEE, 571--576."},{"key":"e_1_3_2_1_15_1","unstructured":"Rie Johnson and Tong Zhang. 2013. Accelerating stochastic gradient descent using predictive variance reduction. In Advances in neural information processing systems. 315--323. Rie Johnson and Tong Zhang. 2013. Accelerating stochastic gradient descent using predictive variance reduction. In Advances in neural information processing systems. 315--323."},{"key":"e_1_3_2_1_16_1","volume-title":"Error feedback fixes SignSGD and other gradient compression schemes. arXiv preprint arXiv:1901.09847","author":"Karimireddy Sai Praneeth","year":"2019","unstructured":"Sai Praneeth Karimireddy , Quentin Rebjock , Sebastian U Stich , and Martin Jaggi . 2019. Error feedback fixes SignSGD and other gradient compression schemes. arXiv preprint arXiv:1901.09847 ( 2019 ). Sai Praneeth Karimireddy, Quentin Rebjock, Sebastian U Stich, and Martin Jaggi. 2019. Error feedback fixes SignSGD and other gradient compression schemes. arXiv preprint arXiv:1901.09847 (2019)."},{"key":"e_1_3_2_1_17_1","volume-title":"Decentralized stochastic optimization and gossip algorithms with compressed communication. arXiv preprint arXiv:1902.00340","author":"Koloskova Anastasia","year":"2019","unstructured":"Anastasia Koloskova , Sebastian U Stich , and Martin Jaggi . 2019. Decentralized stochastic optimization and gossip algorithms with compressed communication. arXiv preprint arXiv:1902.00340 ( 2019 ). Anastasia Koloskova, Sebastian U Stich, and Martin Jaggi. 2019. Decentralized stochastic optimization and gossip algorithms with compressed communication. arXiv preprint arXiv:1902.00340 (2019)."},{"key":"e_1_3_2_1_18_1","unstructured":"Lihua Lei Cheng Ju Jianbo Chen and Michael I Jordan. 2017. Non-convex finite-sum optimization via scsg methods. In Advances in Neural Information Processing Systems. 2348--2358. Lihua Lei Cheng Ju Jianbo Chen and Michael I Jordan. 2017. Non-convex finite-sum optimization via scsg methods. In Advances in Neural Information Processing Systems. 2348--2358."},{"key":"e_1_3_2_1_19_1","unstructured":"Xiangru Lian Ce Zhang Huan Zhang Cho-Jui Hsieh Wei Zhang and Ji Liu. 2017. Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent. In Advances in Neural Information Processing Systems. 5330--5340. Xiangru Lian Ce Zhang Huan Zhang Cho-Jui Hsieh Wei Zhang and Ji Liu. 2017. Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent. In Advances in Neural Information Processing Systems. 5330--5340."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/DSW.2019.8755807"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/2946645.2946706"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSIPN.2016.2613678"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2013.2254478"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2018.2817461"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2008.2009515"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2016.7798402"},{"key":"e_1_3_2_1_27_1","volume-title":"Proceedings of the 34th International Conference on Machine Learning-Volume 70","author":"Nguyen Lam M","year":"2017","unstructured":"Lam M Nguyen , Jie Liu , Katya Scheinberg , and Martin Tak\u00e1c . 2017 . SARAH: A novel method for machine learning problems using stochastic recursive gradient . In Proceedings of the 34th International Conference on Machine Learning-Volume 70 . JMLR. org, 2613--2621. Lam M Nguyen, Jie Liu, Katya Scheinberg, and Martin Tak\u00e1c. 2017. SARAH: A novel method for machine learning problems using stochastic recursive gradient. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2613--2621."},{"key":"e_1_3_2_1_28_1","volume-title":"Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In Advances in neural information processing systems. 693--701.","author":"Recht Benjamin","year":"2011","unstructured":"Benjamin Recht , Christopher Re , Stephen Wright , and Feng Niu . 2011 . Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In Advances in neural information processing systems. 693--701. Benjamin Recht, Christopher Re, Stephen Wright, and Feng Niu. 2011. Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In Advances in neural information processing systems. 693--701."},{"key":"e_1_3_2_1_29_1","volume-title":"International conference on machine learning. 314--323","author":"Reddi Sashank J","year":"2016","unstructured":"Sashank J Reddi , Ahmed Hefny , Suvrit Sra , Barnab\u00e1s P\u00f3czos , and Alex Smola . 2016 . Stochastic variance reduction for nonconvex optimization . In International conference on machine learning. 314--323 . Sashank J Reddi, Ahmed Hefny, Suvrit Sra, Barnab\u00e1s P\u00f3czos, and Alex Smola. 2016. Stochastic variance reduction for nonconvex optimization. In International conference on machine learning. 314--323."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1137\/14096668X"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2019.2943230"},{"key":"e_1_3_2_1_32_1","volume-title":"ICML 2020","author":"Sun Haoran","year":"2019","unstructured":"Haoran Sun , Songtao Lu , and Mingyi Hong . 2019 . Improving the sample and communication complexity for decentralized non-convex optimization: A joint gradient estimation and tracking approach . ICML 2020 (2019). Haoran Sun, Songtao Lu, and Mingyi Hong. 2019. Improving the sample and communication complexity for decentralized non-convex optimization: A joint gradient estimation and tracking approach. ICML 2020 (2019)."},{"key":"e_1_3_2_1_33_1","volume-title":"Hybrid Stochastic Gradient Descent Algorithms for Stochastic Nonconvex Optimization. arXiv preprint arXiv:1905.05920","author":"Tran-Dinh Quoc","year":"2019","unstructured":"Quoc Tran-Dinh , Nhan H Pham , Dzung T Phan , and Lam M Nguyen . 2019. Hybrid Stochastic Gradient Descent Algorithms for Stochastic Nonconvex Optimization. arXiv preprint arXiv:1905.05920 ( 2019 ). Quoc Tran-Dinh, Nhan H Pham, Dzung T Phan, and Lam M Nguyen. 2019. Hybrid Stochastic Gradient Descent Algorithms for Stochastic Nonconvex Optimization. arXiv preprint arXiv:1905.05920 (2019)."},{"key":"e_1_3_2_1_35_1","volume-title":"Distributed stochastic multi-task learning with graph regularization. arXiv preprint arXiv:1802.03830","author":"Wang Weiran","year":"2018","unstructured":"Weiran Wang , Jialei Wang , Mladen Kolar , and Nathan Srebro . 2018. Distributed stochastic multi-task learning with graph regularization. arXiv preprint arXiv:1802.03830 ( 2018 ). Weiran Wang, Jialei Wang, Mladen Kolar, and Nathan Srebro. 2018. Distributed stochastic multi-task learning with graph regularization. arXiv preprint arXiv:1802.03830 (2018)."},{"key":"e_1_3_2_1_36_1","unstructured":"Zhe Wang Kaiyi Ji Yi Zhou Yingbin Liang and Vahid Tarokh. 2019. SpiderBoost and Momentum: Faster Variance Reduction Algorithms. In Advances in Neural Information Processing Systems. 2403--2413. Zhe Wang Kaiyi Ji Yi Zhou Yingbin Liang and Vahid Tarokh. 2019. SpiderBoost and Momentum: Faster Variance Reduction Algorithms. In Advances in Neural Information Processing Systems. 2403--2413."},{"key":"e_1_3_2_1_37_1","volume-title":"Cubic regularization with momentum for nonconvex optimization. arXiv preprint arXiv:1810.03763","author":"Wang Zhe","year":"2018","unstructured":"Zhe Wang , Yi Zhou , Yingbin Liang , and Guanghui Lan . 2018. Cubic regularization with momentum for nonconvex optimization. arXiv preprint arXiv:1810.03763 ( 2018 ). Zhe Wang, Yi Zhou, Yingbin Liang, and Guanghui Lan. 2018. Cubic regularization with momentum for nonconvex optimization. arXiv preprint arXiv:1810.03763 (2018)."},{"key":"e_1_3_2_1_38_1","volume-title":"Variance-reduced decentralized stochastic optimization with gradient tracking. arXiv preprint arXiv:1909.11774","author":"Xin Ran","year":"2019","unstructured":"Ran Xin , Usman A Khan , and Soummya Kar . 2019. Variance-reduced decentralized stochastic optimization with gradient tracking. arXiv preprint arXiv:1909.11774 ( 2019 ). Ran Xin, Usman A Khan, and Soummya Kar. 2019. Variance-reduced decentralized stochastic optimization with gradient tracking. arXiv preprint arXiv:1909.11774 (2019)."},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1137\/130943170"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2018.2872003"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2018.2818081"},{"key":"e_1_3_2_1_42_1","volume-title":"Distributed Linear Model Clustering over Networks: A Tree-Based Fused-Lasso ADMM Approach. arXiv preprint arXiv:1905.11549","author":"Zhang Xin","year":"2019","unstructured":"Xin Zhang , Jia Liu , and Zhengyuan Zhu . 2019. Distributed Linear Model Clustering over Networks: A Tree-Based Fused-Lasso ADMM Approach. arXiv preprint arXiv:1905.11549 ( 2019 ). Xin Zhang, Jia Liu, and Zhengyuan Zhu. 2019. Distributed Linear Model Clustering over Networks: A Tree-Based Fused-Lasso ADMM Approach. arXiv preprint arXiv:1905.11549 (2019)."},{"key":"e_1_3_2_1_43_1","volume-title":"Bentley","author":"Zhang Xin","year":"2020","unstructured":"Xin Zhang , Jia Liu , Zhengyuan Zhu , and Elizabeth S . Bentley . 2020 . GT-STORM: Taming Sample , Communication, and Memory Complexities in Decentralized Non-Convex Learning . https:\/\/kevinliu-osu-ece.github.io\/publications\/GT-STORM_TR.pdf Xin Zhang, Jia Liu, Zhengyuan Zhu, and Elizabeth S. Bentley. 2020. GT-STORM: Taming Sample, Communication, and Memory Complexities in Decentralized Non-Convex Learning. https:\/\/kevinliu-osu-ece.github.io\/publications\/GT-STORM_TR.pdf"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2011.2114734"},{"key":"e_1_3_2_1_45_1","unstructured":"Pan Zhou Xiaotong Yuan and Jiashi Feng. 2018. New insight into hybrid stochastic gradient descent: Beyond with-replacement sampling and convexity. In Advances in Neural Information Processing Systems. 1234--1243. Pan Zhou Xiaotong Yuan and Jiashi Feng. 2018. New insight into hybrid stochastic gradient descent: Beyond with-replacement sampling and convexity. In Advances in Neural Information Processing Systems. 1234--1243."},{"key":"e_1_3_2_1_46_1","volume-title":"Generalization error bounds with probabilistic guarantee for sgd in nonconvex optimization. arXiv preprint arXiv:1802.06903","author":"Zhou Yi","year":"2018","unstructured":"Yi Zhou , Yingbin Liang , and Huishuai Zhang . 2018. Generalization error bounds with probabilistic guarantee for sgd in nonconvex optimization. arXiv preprint arXiv:1802.06903 ( 2018 ). Yi Zhou, Yingbin Liang, and Huishuai Zhang. 2018. Generalization error bounds with probabilistic guarantee for sgd in nonconvex optimization. arXiv preprint arXiv:1802.06903 (2018)."},{"key":"e_1_3_2_1_47_1","unstructured":"Martin Zinkevich Markus Weimer Lihong Li and Alex J Smola. 2010. Parallelized stochastic gradient descent. In Advances in neural information processing systems. 2595--2603. Martin Zinkevich Markus Weimer Lihong Li and Alex J Smola. 2010. Parallelized stochastic gradient descent. In Advances in neural information processing systems. 2595--2603."}],"event":{"name":"MobiHoc '21: The Twenty-second International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing","location":"Shanghai China","acronym":"MobiHoc '21","sponsor":["SIGMOBILE ACM Special Interest Group on Mobility of Systems, Users, Data and Computing"]},"container-title":["Proceedings of the Twenty-second International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3466772.3467056","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,26]],"date-time":"2022-07-26T10:19:36Z","timestamp":1658830776000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3466772.3467056"}},"subtitle":["Taming Sample, Communication, and Memory Complexities in Decentralized Non-Convex Learning"],"short-title":[],"issued":{"date-parts":[[2021,7,26]]},"references-count":46,"alternative-id":["10.1145\/3466772.3467056","10.1145\/3466772"],"URL":"http:\/\/dx.doi.org\/10.1145\/3466772.3467056","relation":{},"published":{"date-parts":[[2021,7,26]]}}}