{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T08:18:03Z","timestamp":1776154683856,"version":"3.50.1"},"reference-count":137,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2024,10,8]],"date-time":"2024-10-08T00:00:00Z","timestamp":1728345600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Swedish Science Foundation (SSF), Korean Ministry of Science and ICT (MSIT), and the IITP","award":["IITP-2024-2020-0-01461 and IITP-2022-0-00420"],"award-info":[{"award-number":["IITP-2024-2020-0-01461 and IITP-2022-0-00420"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Sen. Netw."],"published-print":{"date-parts":[[2024,11,30]]},"abstract":"<jats:p>The recent breakthroughs in machine learning (ML) and deep learning (DL) have catalyzed the design and development of various intelligent systems over wide application domains. While most existing machine learning models require large memory and computing power, efforts have been made to deploy some models on resource-constrained devices as well. A majority of the early application systems focused on exploiting the inference capabilities of ML and DL models, where data captured from different mobile and embedded sensing components are processed through these models for application goals such as classification and segmentation. More recently, the concept of exploiting the mobile and embedded computing resources for ML\/DL model training has gained attention, as such capabilities allow (i) the training of models via local data without the need to share data over wireless links, thus enabling privacy-preserving computation by design, (ii) model personalization and environment adaptation, and (iii) deployment of accurate models in remote and hardly accessible locations without stable internet connectivity. This work summarizes and analyzes state-of-the-art systems research that allows such on-device model training capabilities and provides a survey of on-device training from a systems perspective.<\/jats:p>","DOI":"10.1145\/3696003","type":"journal-article","created":{"date-parts":[[2024,9,14]],"date-time":"2024-09-14T07:23:17Z","timestamp":1726298597000},"page":"1-39","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":25,"title":["On-device Training: A First Overview on Existing Systems"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9839-3820","authenticated-orcid":false,"given":"Shuai","family":"Zhu","sequence":"first","affiliation":[{"name":"Computer Science Department, RISE Research Institutes of Sweden, Kista, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2586-8573","authenticated-orcid":false,"given":"Thiemo","family":"Voigt","sequence":"additional","affiliation":[{"name":"RISE Research Institutes of Sweden, Kista, Sweden and Uppsala University, Uppsala, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4877-0875","authenticated-orcid":false,"given":"Fatemeh","family":"Rahimian","sequence":"additional","affiliation":[{"name":"RISE Research Institutes of Sweden, Kista, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0799-4039","authenticated-orcid":false,"given":"Jeonggil","family":"Ko","sequence":"additional","affiliation":[{"name":"Yonsei University, Seodaemun-gu, Korea (the Republic of) and POSTECH, Pohang, Korea (the Republic of)"}]}],"member":"320","published-online":{"date-parts":[[2024,10,8]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-023-00843-2"},{"key":"e_1_3_2_3_2","unstructured":"Amazon Web Services. 2024. FreeRTOS 202406.01 LTS. https:\/\/www.freertos.org"},{"key":"e_1_3_2_4_2","first-page":"992","volume-title":"Proceedings of the European Conference on Artificial Intelligence (ECAI\u201920)","author":"Beaulieu Shawn","year":"2020","unstructured":"Shawn Beaulieu, Lapo Frati, Thomas Miconi, Joel Lehman, Kenneth O. Stanley, Jeff Clune, and Nick Cheney. 2020. Learning to continually learn. In Proceedings of the European Conference on Artificial Intelligence (ECAI\u201920). IOS Press, 992\u20131001."},{"key":"e_1_3_2_5_2","first-page":"119","volume-title":"Proceedings of the 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201922)","author":"Bhardwaj Romil","year":"2022","unstructured":"Romil Bhardwaj, Zhengxu Xia, Ganesh Ananthanarayanan, Junchen Jiang, Yuanchao Shu, Nikolaos Karianakis, Kevin Hsieh, Paramvir Bahl, and Ion Stoica. 2022. Ekya: Continuous learning of video analytics models on edge compute servers. In Proceedings of the 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201922). 119\u2013135."},{"key":"e_1_3_2_6_2","first-page":"446","volume-title":"Proceedings of the 13th European Conference on Computer Vision (ECCV\u201914)","author":"Bossard Lukas","year":"2014","unstructured":"Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. 2014. Food-101\u2014Mining discriminative components with random forests. In Proceedings of the 13th European Conference on Computer Vision (ECCV\u201914). Springer, 446\u2013461."},{"key":"e_1_3_2_7_2","first-page":"11285","article-title":"TinyTL: Reduce memory, not parameters for efficient on-device learning","volume":"33","author":"Cai Han","year":"2020","unstructured":"Han Cai, Chuang Gan, Ligeng Zhu, and Song Han. 2020. TinyTL: Reduce memory, not parameters for efficient on-device learning. Advan. Neural Inf. Process. Syst. 33 (2020), 11285\u201311297.","journal-title":"Advan. Neural Inf. Process. Syst."},{"key":"e_1_3_2_8_2","article-title":"ProxylessNAS: Direct neural architecture search on target task and hardware","author":"Cai Han","year":"2018","unstructured":"Han Cai, Ligeng Zhu, and Song Han. 2018. ProxylessNAS: Direct neural architecture search on target task and hardware. arXiv preprint arXiv:1812.00332 (2018).","journal-title":"arXiv preprint arXiv:1812.00332"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/FG.2018.00020"},{"key":"e_1_3_2_10_2","first-page":"578","volume-title":"Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201918)","year":"2018","unstructured":"Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. TVM: An automated End-to-End optimizing compiler for deep learning. In Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201918). 578\u2013594."},{"key":"e_1_3_2_11_2","article-title":"Training deep nets with sublinear memory cost","author":"Chen Tianqi","year":"2016","unstructured":"Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. 2016. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174 (2016).","journal-title":"arXiv preprint arXiv:1604.06174"},{"key":"e_1_3_2_12_2","unstructured":"Speech Commands. 1804. A Dataset for Limited-vocabulary Speech Recognition. Retrieved from https:\/\/arxiv.org\/abs\/1804.03209(cs:28.12.2020)"},{"key":"e_1_3_2_13_2","unstructured":"NVIDIA Corporation. 2024. NVIDIA Jetson Platform. Retrieved from https:\/\/developer.nvidia.com\/embedded-computing"},{"key":"e_1_3_2_14_2","article-title":"BinaryConnect: Training deep neural networks with binary weights during propagations","volume":"28","author":"Courbariaux Matthieu","year":"2015","unstructured":"Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. 2015. BinaryConnect: Training deep neural networks with binary weights during propagations. Advan. Neural Inf. Process. Syst. 28 (2015).","journal-title":"Advan. Neural Inf. Process. Syst."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2019.2914438"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_2_17_2","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).","journal-title":"arXiv preprint arXiv:1810.04805"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3450494"},{"key":"e_1_3_2_19_2","unstructured":"Jeff Donahue Yangqing Jia Oriol Vinyals Judy Hoffman Ning Zhang Eric Tzeng and Trevor Darrell. 2014. DeCAF: A deep convolutional activation feature for generic visual recognition. In Proceedings of the 31st International Conference on Machine Learning (Proceedings of Machine Learning Research) Eric P. Xing and Tony Jebara (Eds.). PMLR Bejing China 647--655. Retrieved from https:\/\/proceedings.mlr.press\/v32\/donahue14.html"},{"key":"e_1_3_2_20_2","unstructured":"Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly Jakob Uszkoreit and Neil Houlsby. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)."},{"key":"e_1_3_2_21_2","article-title":"CGaP: Continuous growth and pruning for efficient deep learning","author":"Du Xiaocong","year":"2019","unstructured":"Xiaocong Du, Zheng Li, and Yu Cao. 2019. CGaP: Continuous growth and pruning for efficient deep learning. arXiv preprint arXiv:1905.11533 (2019).","journal-title":"arXiv preprint arXiv:1905.11533"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/LCN.2004.38"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1561\/0400000042"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3241539.3241559"},{"key":"e_1_3_2_25_2","article-title":"Fast graph representation learning with PyTorch Geometric","author":"Fey Matthias","year":"2019","unstructured":"Matthias Fey and Jan Eric Lenssen. 2019. Fast graph representation learning with PyTorch Geometric. arXiv preprint arXiv:1903.02428 (2019).","journal-title":"arXiv preprint arXiv:1903.02428"},{"key":"e_1_3_2_26_2","article-title":"The lottery ticket hypothesis: Finding sparse, trainable neural networks","author":"Frankle Jonathan","year":"2018","unstructured":"Jonathan Frankle and Michael Carbin. 2018. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635 (2018).","journal-title":"arXiv preprint arXiv:1803.03635"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.5555\/3433701.3433723"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3498361.3539765"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10710-017-9314-z"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-021-01453-z"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/347837.347846"},{"key":"e_1_3_2_32_2","first-page":"2829","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Gu Shixiang","year":"2016","unstructured":"Shixiang Gu, Timothy Lillicrap, Ilya Sutskever, and Sergey Levine. 2016. Continuous deep q-learning with model-based acceleration. In Proceedings of the International Conference on Machine Learning. PMLR, 2829\u20132838."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.123"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.155"},{"key":"e_1_3_2_36_2","first-page":"177","volume-title":"Proceedings of the 9th Annual Conference of the Cognitive Science Society","author":"Hinton Geoffrey E.","year":"1987","unstructured":"Geoffrey E. Hinton and David C. Plaut. 1987. Using fast weights to deblur old memories. In Proceedings of the 9th Annual Conference of the Cognitive Science Society. 177\u2013186."},{"key":"e_1_3_2_37_2","article-title":"Autoencoders, minimum description length and Helmholtz free energy","volume":"6","author":"Hinton Geoffrey E.","year":"1993","unstructured":"Geoffrey E. Hinton and Richard Zemel. 1993. Autoencoders, minimum description length and Helmholtz free energy. Advan. Neural Inf. Process. Syst. 6 (1993).","journal-title":"Advan. Neural Inf. Process. Syst."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSCC.2014.6757323"},{"key":"e_1_3_2_39_2","article-title":"MobileNets: Efficient convolutional neural networks for mobile vision applications","author":"Howard Andrew G.","year":"2017","unstructured":"Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017).","journal-title":"arXiv preprint arXiv:1704.04861"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3373376.3378530"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.243"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3581791.3596852"},{"key":"e_1_3_2_43_2","article-title":"GPipe: Efficient training of giant neural networks using pipeline parallelism","volume":"32","author":"Huang Yanping","year":"2019","unstructured":"Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Dehao Chen, Mia Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V. Le, Yonghui Wu, et\u00a0al. 2019. GPipe: Efficient training of giant neural networks using pipeline parallelism. Advan. Neural Inf. Process. Syst. 32 (2019).","journal-title":"Advan. Neural Inf. Process. Syst."},{"key":"e_1_3_2_44_2","article-title":"SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size","author":"Iandola Forrest N.","year":"2016","unstructured":"Forrest N. Iandola, Song Han, Matthew W. Moskewicz, Khalid Ashraf, William J. Dally, and Kurt Keutzer. 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size. arXiv preprint arXiv:1602.07360 (2016).","journal-title":"arXiv preprint arXiv:1602.07360"},{"key":"e_1_3_2_45_2","article-title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","volume":"1502","author":"Ioffe Sergey","year":"2015","unstructured":"Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. CoRR abs\/1502.03167 (2015).","journal-title":"CoRR"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2019.2918225"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00286"},{"key":"e_1_3_2_48_2","first-page":"497","article-title":"Checkmate: Breaking the memory wall with optimal tensor rematerialization","volume":"2","author":"Jain Paras","year":"2020","unstructured":"Paras Jain, Ajay Jain, Aniruddha Nrusimha, Amir Gholami, Pieter Abbeel, Joseph Gonzalez, Kurt Keutzer, and Ion Stoica. 2020. Checkmate: Breaking the memory wall with optimal tensor rematerialization. Proc. Mach. Learn. Syst. 2 (2020), 497\u2013511.","journal-title":"Proc. Mach. Learn. Syst."},{"key":"e_1_3_2_49_2","article-title":"Meta-learning representations for continual learning","volume":"32","author":"Javed Khurram","year":"2019","unstructured":"Khurram Javed and Martha White. 2019. Meta-learning representations for continual learning. Advan. Neural Inf. Process. Syst. 32 (2019).","journal-title":"Advan. Neural Inf. Process. Syst."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.57"},{"key":"e_1_3_2_51_2","first-page":"1","article-title":"MNN: A universal and efficient inference engine","volume":"2","year":"2020","unstructured":"Xiaotang Jiang, Huan Wang, Yiliu Chen, Ziqi Wu, Lichuan Wang, Bin Zou, Yafeng Yang, Zongyang Cui, Yu Cai, Tianhang Yu, Chengfei Lv, and Zhihua Wu. 2020. MNN: A universal and efficient inference engine. Proc. Mach. Learn. Syst. 2 (2020), 1\u201313.","journal-title":"Proc. Mach. Learn. Syst."},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3079856.3080246"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.301"},{"key":"e_1_3_2_54_2","first-page":"172","article-title":"Accelerating training and inference of graph neural networks with fast sampling and pipelining","volume":"4","author":"Kaler Tim","year":"2022","unstructured":"Tim Kaler, Nickolas Stathas, Anne Ouyang, Alexandros-Stavros Iliopoulos, Tao Schardl, Charles E. Leiserson, and Jie Chen. 2022. Accelerating training and inference of graph neural networks with fast sampling and pipelining. Proc. Mach. Learn. Syst. 4 (2022), 172\u2013189.","journal-title":"Proc. Mach. Learn. Syst."},{"key":"e_1_3_2_55_2","volume-title":"Proceedings of the CVPR Workshop on Fine-grained Visual Categorization (FGVC\u201911)","volume":"2","author":"Khosla Aditya","year":"2011","unstructured":"Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Fei-Fei Li. 2011. Novel dataset for fine-grained image categorization: Stanford dogs. In Proceedings of the CVPR Workshop on Fine-grained Visual Categorization (FGVC\u201911), Vol. 2."},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3458864.3468161"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3302424.3303950"},{"key":"e_1_3_2_58_2","article-title":"Adam: A method for stochastic optimization","author":"Kingma Diederik P.","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).","journal-title":"arXiv preprint arXiv:1412.6980"},{"key":"e_1_3_2_59_2","article-title":"Dynamic tensor rematerialization","author":"Kirisame Marisa","year":"2020","unstructured":"Marisa Kirisame, Steven Lyubomirsky, Altan Haan, Jennifer Brennan, Mike He, Jared Roesch, Tianqi Chen, and Zachary Tatlock. 2020. Dynamic tensor rematerialization. arXiv preprint arXiv:2006.09616 (2020).","journal-title":"arXiv preprint arXiv:2006.09616"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2013.77"},{"key":"e_1_3_2_61_2","unstructured":"Alex Krizhevsky. 2009. Learning multiple layers of features from tiny images. Retrieved from https:\/\/www.cs.toronto.edu\/\u223ckriz\/learning-features-2009-TR.pdf"},{"key":"e_1_3_2_62_2","volume-title":"Advances in Neural Information Processing Systems","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, F. Pereira, C. J. Burges, L. Bottou, and K. Q. Weinberger (Eds.). Vol. 25. Curran Associates, Inc. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2012\/file\/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2016.05.004"},{"key":"e_1_3_2_64_2","article-title":"A graph theoretic framework of recomputation algorithms for memory-efficient backpropagation","volume":"32","author":"Kusumoto Mitsuru","year":"2019","unstructured":"Mitsuru Kusumoto, Takuya Inoue, Gentaro Watanabe, Takuya Akiba, and Masanori Koyama. 2019. A graph theoretic framework of recomputation algorithms for memory-efficient backpropagation. Advan. Neural Inf. Process. Syst. 32 (2019).","journal-title":"Advan. Neural Inf. Process. Syst."},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/1964897.1964918"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3625687.3625804"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3372224.3419194"},{"key":"e_1_3_2_68_2","unstructured":"Ya Le and Xuan Yang. 2015. Tiny ImageNet visual recognition Challenge. Retrieved from https:\/\/cs231n.stanford.edu\/2015\/reports.html"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00932"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3489517.3530587"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-27139-2_7"},{"key":"e_1_3_2_73_2","first-page":"1839","article-title":"TensorFlow Lite: On-device machine learning framework","volume":"57","author":"Li Shuangfeng","year":"2020","unstructured":"Shuangfeng Li. 2020. TensorFlow Lite: On-device machine learning framework. J. Comput. Res. Dev. 57 (2020), 1839.","journal-title":"J. Comput. Res. Dev."},{"key":"e_1_3_2_74_2","first-page":"11711","article-title":"MCUNet: Tiny deep learning on IoT devices","volume":"33","year":"2020","unstructured":"Ji Lin, Wei-Ming Chen, Yujun Lin, John Cohn, Chuang Gan, and Song Han. 2020. MCUNet: Tiny deep learning on IoT devices. Advan. Neural Inf. Process. Syst. 33 (2020), 11711\u201311722.","journal-title":"Advan. Neural Inf. Process. Syst."},{"key":"e_1_3_2_75_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems","author":"Lin Ji","year":"2022","unstructured":"Ji Lin, Ligeng Zhu, Wei-ming Chen, Wei-chen Wang, Chuang Gan, and Song Han. 2022. On-device training under 256KB memory. In Proceedings of the Annual Conference on Neural Information Processing Systems."},{"key":"e_1_3_2_76_2","article-title":"Towards accurate binary convolutional neural network","volume":"30","author":"Lin Xiaofan","year":"2017","unstructured":"Xiaofan Lin, Cong Zhao, and Wei Pan. 2017. Towards accurate binary convolutional neural network. Advan. Neural Inf. Process. Syst. 30 (2017).","journal-title":"Advan. Neural Inf. Process. Syst."},{"key":"e_1_3_2_77_2","article-title":"Neural networks with few multiplications","author":"Lin Zhouhan","year":"2015","unstructured":"Zhouhan Lin, Matthieu Courbariaux, Roland Memisevic, and Yoshua Bengio. 2015. Neural networks with few multiplications. arXiv preprint arXiv:1510.03009 (2015).","journal-title":"arXiv preprint arXiv:1510.03009"},{"key":"e_1_3_2_78_2","article-title":"Dynamic sparse graph for efficient deep learning","author":"Liu Liu","year":"2018","unstructured":"Liu Liu, Lei Deng, Xing Hu, Maohua Zhu, Guoqi Li, Yufei Ding, and Yuan Xie. 2018. Dynamic sparse graph for efficient deep learning. arXiv preprint arXiv:1810.00859 (2018).","journal-title":"arXiv preprint arXiv:1810.00859"},{"key":"e_1_3_2_79_2","unstructured":"Ziwei Liu Ping Luo Xiaogang Wang and Xiaoou Tang. 2018. Large-scale CelebFaces attributes (CelebA) dataset. Retrieved 20 September 2024 from https:\/\/mmlab.ie.cuhk.edu.hk\/projects\/CelebA.html"},{"key":"e_1_3_2_80_2","article-title":"Gradient episodic memory for continual learning","volume":"30","author":"Lopez-Paz David","year":"2017","unstructured":"David Lopez-Paz and Marc\u2019Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. Advan. Neural Inf. Process. Syst. 30 (2017).","journal-title":"Advan. Neural Inf. Process. Syst."},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1145\/3570361.3613297"},{"key":"e_1_3_2_82_2","article-title":"Fine-grained visual classification of aircraft","author":"Maji Subhransu","year":"2013","unstructured":"Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. 2013. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151 (2013).","journal-title":"arXiv preprint arXiv:1306.5151"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.102.3.419"},{"key":"e_1_3_2_84_2","first-page":"109","volume-title":"Psychology of Learning and Motivation","author":"McCloskey Michael","year":"1989","unstructured":"Michael McCloskey and Neal J. Cohen. 1989. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of Learning and Motivation. Vol. 24. Elsevier, 109\u2013165."},{"key":"e_1_3_2_85_2","first-page":"1273","volume-title":"Artificial Intelligence and Statistics","author":"McMahan Brendan","year":"2017","unstructured":"Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics. PMLR, 1273\u20131282."},{"key":"e_1_3_2_86_2","unstructured":"Paulius Micikevicius Sharan Narang Jonah Alben Gregory Diamos Erich Elsen David Garcia Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh and Hao Wu. 2017. Mixed precision training. arXiv preprint arXiv:1710.03740 (2017)."},{"key":"e_1_3_2_87_2","article-title":"Accelerating sparse deep neural networks","author":"Mishra Asit","year":"2021","unstructured":"Asit Mishra, Jorge Albericio Latorre, Jeff Pool, Darko Stosic, Dusan Stosic, Ganesh Venkatesh, Chong Yu, and Paulius Micikevicius. 2021. Accelerating sparse deep neural networks. arXiv preprint arXiv:2104.08378 (2021).","journal-title":"arXiv preprint arXiv:2104.08378"},{"key":"e_1_3_2_88_2","article-title":"Playing Atari with deep reinforcement learning","author":"Mnih Volodymyr","year":"2013","unstructured":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing Atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013).","journal-title":"arXiv preprint arXiv:1312.5602"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01152"},{"key":"e_1_3_2_91_2","article-title":"Pruning convolutional neural networks for resource efficient inference","author":"Molchanov Pavlo","year":"2016","unstructured":"Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. 2016. Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:1611.06440 (2016).","journal-title":"arXiv preprint arXiv:1611.06440"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICVGIP.2008.47"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2019.01.012"},{"key":"e_1_3_2_94_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2012.6248092"},{"key":"e_1_3_2_95_2","first-page":"17573","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Patil Shishir G.","year":"2022","unstructured":"Shishir G. Patil, Paras Jain, Prabal Dutta, Ion Stoica, and Joseph Gonzalez. 2022. POET: Training neural networks on tiny devices with integrated rematerialization and paging. In Proceedings of the International Conference on Machine Learning. PMLR, 17573\u201317583."},{"key":"e_1_3_2_96_2","doi-asserted-by":"publisher","DOI":"10.1145\/3373376.3378505"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2958684"},{"key":"e_1_3_2_98_2","volume-title":"Proceedings of the International Conference on Embedded Wireless Systems and Networks (EWSN\u201922)","author":"Profentzas Christos","year":"2022","unstructured":"Christos Profentzas, Magnus Almgren, and Olaf Landsiedel. 2022. MiniLearn: On-device learning for low-power IoT devices. In Proceedings of the International Conference on Embedded Wireless Systems and Networks (EWSN\u201922). Junction Publishing, USA."},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.1145\/3534678.3539293"},{"key":"e_1_3_2_100_2","article-title":"Adaptive federated optimization","author":"Reddi Sashank","year":"2020","unstructured":"Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Kone\u010dn\u1ef3, Sanjiv Kumar, and H. Brendan McMahan. 2020. Adaptive federated optimization. arXiv preprint arXiv:2003.00295 (2020).","journal-title":"arXiv preprint arXiv:2003.00295"},{"key":"e_1_3_2_101_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN52387.2021.9533927"},{"key":"e_1_3_2_102_2","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO.2016.7783721"},{"key":"e_1_3_2_103_2","article-title":"Learning to learn without forgetting by maximizing transfer and minimizing interference","author":"Riemer Matthew","year":"2018","unstructured":"Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. 2018. Learning to learn without forgetting by maximizing transfer and minimizing interference. arXiv preprint arXiv:1810.11910 (2018).","journal-title":"arXiv preprint arXiv:1810.11910"},{"key":"e_1_3_2_104_2","article-title":"Glow: Graph lowering compiler techniques for neural networks","author":"Rotem Nadav","year":"2018","unstructured":"Nadav Rotem, Jordan Fix, Saleem Abdulrasool, Garret Catron, Summer Deng, Roman Dzhabarov, Nick Gibson, James Hegeman, Meghan Lele, Roman Levenstein, et\u00a0al. 2018. Glow: Graph lowering compiler techniques for neural networks. arXiv preprint arXiv:1805.00907 (2018).","journal-title":"arXiv preprint arXiv:1805.00907"},{"key":"e_1_3_2_105_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-15-3020-3_31"},{"key":"e_1_3_2_106_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"e_1_3_2_107_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSEN.2022.3210773"},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00474"},{"key":"e_1_3_2_109_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2014.131"},{"key":"e_1_3_2_110_2","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2015.9"},{"key":"e_1_3_2_111_2","volume-title":"Proceedings of the AAAI Spring Symposium Series","author":"Silver Daniel L.","year":"2013","unstructured":"Daniel L. Silver, Qiang Yang, and Lianghao Li. 2013. Lifelong machine learning systems: Beyond learning algorithms. In Proceedings of the AAAI Spring Symposium Series."},{"key":"e_1_3_2_112_2","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).","journal-title":"arXiv preprint arXiv:1409.1556"},{"key":"e_1_3_2_113_2","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.5786"},{"key":"e_1_3_2_114_2","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton Richard S.","year":"2018","unstructured":"Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. MIT Press."},{"key":"e_1_3_2_115_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_3_2_116_2","volume-title":"Modern Operating Systems","author":"Tanenbaum Andrew","year":"2009","unstructured":"Andrew Tanenbaum. 2009. Modern Operating Systems. Pearson Education, Inc."},{"issue":"2","key":"e_1_3_2_117_2","first-page":"58","article-title":"The problem of concept drift: Definitions and related work","volume":"106","author":"Tsymbal Alexey","year":"2004","unstructured":"Alexey Tsymbal. 2004. The problem of concept drift: Definitions and related work. Computer Sci. Depart., Trinity Coll. Dub. 106, 2 (2004), 58.","journal-title":"Computer Sci. Depart., Trinity Coll. Dub."},{"key":"e_1_3_2_118_2","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Processing Systems I. Guyon U. Von Luxburg S. Bengio H. Wallach R. Fergus S. Vishwanathan and R. Garnett (Eds.). Curran Associates Inc. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf"},{"key":"e_1_3_2_119_2","unstructured":"Oriol Vinyals Charles Blundell Timothy Lillicrap Koray Kavukcuoglu and Daan Wierstra. 2016. Matching networks for one shot learning. Advan. Neural Inf. Process. Syst. 29 (2016)."},{"key":"e_1_3_2_120_2","unstructured":"Catherine Wah Steve Branson Peter Welinder Pietro Perona and Serge Belongie. 2011. The caltech-ucsd birds-200-2011 dataset. California Institute of Technology. https:\/\/www.vision.caltech.edu\/datasets\/cub_200_2011\/"},{"key":"e_1_3_2_121_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447993.3448625"},{"key":"e_1_3_2_122_2","doi-asserted-by":"publisher","DOI":"10.1145\/3498361.3538928"},{"key":"e_1_3_2_123_2","first-page":"1995","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Wang Ziyu","year":"2016","unstructured":"Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Hasselt, Marc Lanctot, and Nando Freitas. 2016. Dueling network architectures for deep reinforcement learning. In Proceedings of the International Conference on Machine Learning. PMLR, 1995\u20132003."},{"key":"e_1_3_2_124_2","article-title":"Speech commands: A dataset for limited-vocabulary speech recognition","author":"Warden Pete","year":"2018","unstructured":"Pete Warden. 2018. Speech commands: A dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209 (2018).","journal-title":"arXiv preprint arXiv:1804.03209"},{"key":"e_1_3_2_125_2","unstructured":"Christopher John Cornish Hellaby Watkins. 1989. Learning from delayed rewards. (1989). https:\/\/www.cs.rhul.ac.uk\/home\/chrisw\/new_thesis.pdf"},{"key":"e_1_3_2_126_2","doi-asserted-by":"publisher","DOI":"10.1145\/3495243.3560545"},{"key":"e_1_3_2_127_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313591"},{"key":"e_1_3_2_128_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMC.2019.2893250"},{"key":"e_1_3_2_129_2","doi-asserted-by":"publisher","DOI":"10.1145\/3386901.3388948"},{"key":"e_1_3_2_130_2","doi-asserted-by":"publisher","DOI":"10.1145\/3241539.3241563"},{"key":"e_1_3_2_131_2","doi-asserted-by":"publisher","DOI":"10.1145\/3442381.3449851"},{"key":"e_1_3_2_132_2","doi-asserted-by":"publisher","DOI":"10.1145\/3372224.3419185"},{"key":"e_1_3_2_133_2","first-page":"377","volume-title":"Proceedings of the 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201917)","author":"Zhang Haoyu","year":"2017","unstructured":"Haoyu Zhang, Ganesh Ananthanarayanan, Peter Bodik, Matthai Philipose, Paramvir Bahl, and Michael J. Freedman. 2017. Live video analytics at scale with approximation and Delay-Tolerance. In Proceedings of the 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201917). 377\u2013392."},{"key":"e_1_3_2_134_2","doi-asserted-by":"publisher","DOI":"10.1145\/3485447.3512148"},{"key":"e_1_3_2_135_2","doi-asserted-by":"publisher","DOI":"10.1145\/3384419.3430716"},{"key":"e_1_3_2_136_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2022.3153408"},{"key":"e_1_3_2_137_2","article-title":"Neural architecture search with reinforcement learning","author":"Zoph Barret","year":"2016","unstructured":"Barret Zoph and Quoc V. Le. 2016. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016).","journal-title":"arXiv preprint arXiv:1611.01578"},{"key":"e_1_3_2_138_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00907"}],"container-title":["ACM Transactions on Sensor Networks"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3696003","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3696003","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:04:30Z","timestamp":1750291470000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3696003"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,8]]},"references-count":137,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,11,30]]}},"alternative-id":["10.1145\/3696003"],"URL":"https:\/\/doi.org\/10.1145\/3696003","relation":{},"ISSN":["1550-4859","1550-4867"],"issn-type":[{"value":"1550-4859","type":"print"},{"value":"1550-4867","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,8]]},"assertion":[{"value":"2023-05-09","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-04","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}