{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T02:40:52Z","timestamp":1783564852988,"version":"3.55.0"},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2024,1,19]],"date-time":"2024-01-19T00:00:00Z","timestamp":1705622400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62172441 and 61772553"],"award-info":[{"award-number":["62172441 and 61772553"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Natural Science Foundation of Hunan Province","award":["2023JJ30696"],"award-info":[{"award-number":["2023JJ30696"]}]},{"name":"Opening Project of State Key Laboratory of Nickel and Cobalt Resources Comprehensive Utilization","award":["GZSYS-KY-2022-018, GZSYS-KY-2022-024"],"award-info":[{"award-number":["GZSYS-KY-2022-018, GZSYS-KY-2022-024"]}]},{"name":"Shenzhen Science and Technology Program","award":["JCYJ20220818103200002"],"award-info":[{"award-number":["JCYJ20220818103200002"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Embed. Comput. Syst."],"published-print":{"date-parts":[[2024,1,31]]},"abstract":"<jats:p>Edge intelligence has emerged as a promising paradigm to accelerate DNN inference by model partitioning, which is particularly useful for intelligent scenarios that demand high accuracy and low latency. However, the dynamic nature of the edge environment and the diversity of end devices pose a significant challenge for DNN model partitioning strategies. Meanwhile, limited resources of the edge server make it difficult to manage resource allocation efficiently among multiple devices. In addition, most of the existing studies disregard the different service requirements of the DNN inference tasks, such as its high accuracy-sensitive or high latency-sensitive. To address these challenges, we propose a Multi-Compression Scale DNN Inference Acceleration (MCIA) based on cloud-edge-end collaboration. We model this problem as a mixed-integer multi-dimensional optimization problem, jointly optimizing the DNN model version choice, the partitioning choice, and the allocation of computational and bandwidth resources to maximize the tradeoff between inference accuracy and latency depending on the property of the tasks. Initially, we train multiple versions of DNN inference models with different compression scales in the cloud, and deploy them to end devices and edge server. Next, a deep reinforcement learning-based algorithm is developed for joint decision making of adaptive collaborative inference and resource allocation based on the current multi-compression scale models and the task property. Experimental results show that MCIA can adapt to heterogeneous devices and dynamic networks, and has superior performance compared with other methods.<\/jats:p>","DOI":"10.1145\/3634704","type":"journal-article","created":{"date-parts":[[2023,11,28]],"date-time":"2023-11-28T12:14:16Z","timestamp":1701173656000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":34,"title":["Multi-Compression Scale DNN Inference Acceleration based on Cloud-Edge-End Collaboration"],"prefix":"10.1145","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5264-5703","authenticated-orcid":false,"given":"Huamei","family":"Qi","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, Central South University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-3516-5363","authenticated-orcid":false,"given":"Fang","family":"Ren","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3559-043X","authenticated-orcid":false,"given":"Leilei","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3898-4303","authenticated-orcid":false,"given":"Ping","family":"Jiang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7013-9081","authenticated-orcid":false,"given":"Shaohua","family":"Wan","sequence":"additional","affiliation":[{"name":"Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2740-8025","authenticated-orcid":false,"given":"Xiaoheng","family":"Deng","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University, China and Shenzhen Research Institute, Central South University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,1,19]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"[Online]. 2016. The GFLOPS\/W of the various machines in the VMW Research Group. https:\/\/web.eece.maine.edu\/vweaver\/group\/green_machines.html"},{"key":"e_1_3_1_3_2","article-title":"DRL based partial offloading for maximizing sum computation rate of FDMA-based wireless powered mobile edge computing","volume":"214","author":"Chen Wenchao","year":"2022","unstructured":"Wenchao Chen, Guanqun Shen, Kaikai Chi, Shubin Zhang, and Xiaolong Chen. 2022. DRL based partial offloading for maximizing sum computation rate of FDMA-based wireless powered mobile edge computing. Computer Networks 214 (2022), 109158.","journal-title":"Computer Networks"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/MNET.2018.1700145"},{"key":"e_1_3_1_5_2","article-title":"Intelligent delay-aware partial computing task offloading for multi-user industrial Internet of Things through edge computing","author":"Deng Xiaoheng","year":"2021","unstructured":"Xiaoheng Deng, Jian Yin, Peiyuan Guan, Neal N. Xiong, Lan Zhang, and Shahid Mumtaz. 2021. Intelligent delay-aware partial computing task offloading for multi-user industrial Internet of Things through edge computing. IEEE Internet of Things Journal (2021).","journal-title":"IEEE Internet of Things Journal"},{"key":"e_1_3_1_6_2","article-title":"Deep reinforcement learning-based resource allocation for cloud gaming via edge computing","author":"Deng Xiaoheng","year":"2022","unstructured":"Xiaoheng Deng, Jingjing Zhang, Honggang Zhang, and Ping Jiang. 2022. Deep reinforcement learning-based resource allocation for cloud gaming via edge computing. IEEE Internet of Things Journal (2022).","journal-title":"IEEE Internet of Things Journal"},{"key":"e_1_3_1_7_2","first-page":"855","volume-title":"2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)","author":"Dey Swarnava","year":"2019","unstructured":"Swarnava Dey, Jayeeta Mondal, and Arijit Mukherjee. 2019. Offloaded execution of deep learning inference at edge: Challenges and insights. In 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE, 855\u2013861."},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNSM.2021.3116665"},{"key":"e_1_3_1_9_2","article-title":"Multi-exit DNN inference acceleration based on multi-dimensional optimization for edge intelligence","author":"Dong Fang","year":"2022","unstructured":"Fang Dong, Huitian Wang, Dian Shen, Zhaowu Huang, Qiang He, Jinghui Zhang, Liangsheng Wen, and Tingting Zhang. 2022. Multi-exit DNN inference acceleration based on multi-dimensional optimization for edge intelligence. IEEE Transactions on Mobile Computing (2022).","journal-title":"IEEE Transactions on Mobile Computing"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMC.2019.2947893"},{"key":"e_1_3_1_11_2","first-page":"1","volume-title":"2019 IEEE\/ACM International Symposium on Low Power Electronics and Design (ISLPED)","author":"Eshratifar Amir Erfan","year":"2019","unstructured":"Amir Erfan Eshratifar, Amirhossein Esmaili, and Massoud Pedram. 2019. BottleNet: A deep learning architecture for intelligent mobile cloud computing services. In 2019 IEEE\/ACM International Symposium on Low Power Electronics and Design (ISLPED). IEEE, 1\u20136."},{"key":"e_1_3_1_12_2","article-title":"Task partitioning and offloading in DNN-task enabled mobile edge computing networks","author":"Gao Mingjin","year":"2021","unstructured":"Mingjin Gao, Rujing Shen, Long Shi, Wen Qi, Jun Li, and Yonghui Li. 2021. Task partitioning and offloading in DNN-task enabled mobile edge computing networks. IEEE Transactions on Mobile Computing (2021).","journal-title":"IEEE Transactions on Mobile Computing"},{"key":"e_1_3_1_13_2","article-title":"Soft actor-critic algorithms and applications","author":"Haarnoja Tuomas","year":"2018","unstructured":"Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, et\u00a0al. 2018. Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905 (2018).","journal-title":"arXiv preprint arXiv:1812.05905"},{"key":"e_1_3_1_14_2","article-title":"Learning both weights and connections for efficient neural network","volume":"28","author":"Han Song","year":"2015","unstructured":"Song Han, Jeff Pool, John Tran, and William Dally. 2015. Learning both weights and connections for efficient neural network. Advances in Neural Information Processing Systems 28 (2015).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_15_2","article-title":"Multi-agent collaborative inference via DNN decoupling: Intermediate feature compression and edge learning","author":"Hao Zhiwei","year":"2022","unstructured":"Zhiwei Hao, Guanyu Xu, Yong Luo, Han Hu, Jianping An, and Shiwen Mao. 2022. Multi-agent collaborative inference via DNN decoupling: Intermediate feature compression and edge learning. IEEE Transactions on Mobile Computing (2022).","journal-title":"IEEE Transactions on Mobile Computing"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_17_2","article-title":"Distilling the knowledge in a neural network","author":"Hinton Geoffrey","year":"2015","unstructured":"Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).","journal-title":"arXiv preprint arXiv:1503.02531"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00140"},{"key":"e_1_3_1_19_2","article-title":"MobileNets: Efficient convolutional neural networks for mobile vision applications","author":"Howard Andrew G.","year":"2017","unstructured":"Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017).","journal-title":"arXiv preprint arXiv:1704.04861"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM.2019.8737614"},{"key":"e_1_3_1_21_2","doi-asserted-by":"crossref","first-page":"892","DOI":"10.1109\/INFCOMW.2019.8845240","volume-title":"IEEE INFOCOM 2019-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)","author":"Huang Yutao","year":"2019","unstructured":"Yutao Huang, Feng Wang, Fangxin Wang, and Jiangchuan Liu. 2019. DeePar: A hybrid device-edge-cloud execution framework for mobile deep learning applications. In IEEE INFOCOM 2019-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). IEEE, 892\u2013897."},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/TGCN.2021.3122911"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3093337.3037698"},{"key":"e_1_3_1_24_2","unstructured":"Alex Krizhevsky Geoffrey Hinton et\u00a0al. 2009. Learning multiple layers of features from tiny images. (2009)."},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TWC.2019.2946140"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00637"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","unstructured":"Neiwen Ling Xuan Huang Zhihe Zhao Nan Guan Zhenyu Yan and Guoliang Xing. 2023. BlastNet: Exploiting duo-blocks for cross-processor real-time DNN inference. ACM Transactions on Embedded Computing Systems New York NY USA. 10.1145\/3560905.3568520","DOI":"10.1145\/3560905.3568520"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2022.10.033"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2021.3099368"},{"key":"e_1_3_1_30_2","article-title":"Edge intelligence: Concepts, architectures, applications and future directions","author":"Mendez Javier","year":"2021","unstructured":"Javier Mendez, Kay Bierzynski, M.P. Cu\u00e9llar, and Diego Morales. 2021. Edge intelligence: Concepts, architectures, applications and future directions. ACM Transactions on Embedded Computing Systems (TECS) (2021).","journal-title":"ACM Transactions on Embedded Computing Systems (TECS)"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3564121.3564796"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.91"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00474"},{"key":"e_1_3_1_34_2","article-title":"Proximal policy optimization algorithms","author":"Schulman John","year":"2017","unstructured":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017).","journal-title":"arXiv preprint arXiv:1707.06347"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1002\/wics.56"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2023.3274307"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2023.3304304"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMC.2019.2957804"},{"key":"e_1_3_1_39_2","article-title":"CrossFuser: Multi-modal feature fusion for end-to-end autonomous driving under unseen weather conditions","author":"Wu Weishang","year":"2023","unstructured":"Weishang Wu, Xiaoheng Deng, Ping Jiang, Shaohua Wan, and Yuanxiong Guo. 2023. CrossFuser: Multi-modal feature fusion for end-to-end autonomous driving under unseen weather conditions. IEEE Transactions on Intelligent Transportation Systems (2023).","journal-title":"IEEE Transactions on Intelligent Transportation Systems"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2019.2959035"},{"key":"e_1_3_1_41_2","article-title":"Computation migration oriented resource allocation in mobile social clouds","author":"Yi Bo","year":"2021","unstructured":"Bo Yi, Xingwei Wang, Min Huang, Qiang He, and Fuliang Li. 2021. Computation migration oriented resource allocation in mobile social clouds. IEEE Transactions on Cloud Computing (2021).","journal-title":"IEEE Transactions on Cloud Computing"},{"issue":"2","key":"e_1_3_1_42_2","article-title":"CoEdge: Cooperative DNN inference with adaptive workload partitioning over heterogeneous edge devices","volume":"29","author":"Zeng Liekang","year":"2020","unstructured":"Liekang Zeng, Xu Chen, Zhi Zhou, Lei Yang, and Junshan Zhang. 2020. CoEdge: Cooperative DNN inference with adaptive workload partitioning over heterogeneous edge devices. IEEE\/ACM Transactions on Networking 29, 2 (2020).","journal-title":"IEEE\/ACM Transactions on Networking"},{"issue":"8","key":"e_1_3_1_43_2","doi-asserted-by":"crossref","first-page":"7605","DOI":"10.1109\/TVT.2021.3068255","article-title":"Deep reinforcement learning based resource management for DNN inference in industrial IoT","volume":"70","author":"Zhang Weiting","year":"2021","unstructured":"Weiting Zhang, Dong Yang, Haixia Peng, Wen Wu, Wei Quan, Hongke Zhang, and Xuemin Shen. 2021. Deep reinforcement learning based resource management for DNN inference in industrial IoT. IEEE Transactions on Vehicular Technology 70, 8 (2021), 7605\u20137618.","journal-title":"IEEE Transactions on Vehicular Technology"},{"key":"e_1_3_1_44_2","article-title":"MESON: A mobility-aware dependent task offloading scheme for urban vehicular edge computing","author":"Zhao Liang","year":"2023","unstructured":"Liang Zhao, Enchao Zhang, Shaohua Wan, Ammar Hawbani, Ahmed Y. Al-Dubai, Geyong Min, and Albert Y. Zomaya. 2023. MESON: A mobility-aware dependent task offloading scheme for urban vehicular edge computing. IEEE Transactions on Mobile Computing (2023).","journal-title":"IEEE Transactions on Mobile Computing"}],"container-title":["ACM Transactions on Embedded Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3634704","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3634704","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:35:49Z","timestamp":1750178149000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3634704"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,19]]},"references-count":43,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1,31]]}},"alternative-id":["10.1145\/3634704"],"URL":"https:\/\/doi.org\/10.1145\/3634704","relation":{},"ISSN":["1539-9087","1558-3465"],"issn-type":[{"value":"1539-9087","type":"print"},{"value":"1558-3465","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,19]]},"assertion":[{"value":"2023-06-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-14","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-01-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}