{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T05:06:43Z","timestamp":1750309603088,"version":"3.41.0"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2025,3,21]],"date-time":"2025-03-21T00:00:00Z","timestamp":1742515200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["U23B2001, 62171057, 62101064, 62201072, 62001054, 62071067"],"award-info":[{"award-number":["U23B2001, 62171057, 62101064, 62201072, 62001054, 62071067"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Ministry of Education and China Mobile Joint Fund","award":["MCM20200202, MCM20180101"],"award-info":[{"award-number":["MCM20200202, MCM20180101"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2025,3,31]]},"abstract":"<jats:p>\n            Parallelizing CNN inference on heterogeneous edge clusters with data parallelism has gained popularity as a way to meet real-time requirements without sacrificing model accuracy. However, existing algorithms struggle to find optimal parallel granularity for complex CNNS, the structure of which is a directed acyclic graph (DAG) rather than a chain, and the parallel dimension is inflexible. To distribute the workload of modern CNNs on heterogeneous devices is also proven as NP-hard problem. In this article, we introduce\n            <jats:italic>DeepZoning<\/jats:italic>\n            , a versatile and cooperative inference framework that combines both model and data parallelism to accelerate CNN inference. DeepZoning employs two algorithms at different levels: (1) a low-level Adaptive Workload Partition algorithm that uses linear programming and takes spatial and channel dimensions into optimization during the search for feature map distribution on heterogeneous devices, and (2) a high-level Model Partition algorithm that finds the optimal model granularity and organizes complex CNNs into sequential zones to balance communication and computation during execution. Our experimental evaluations show that DeepZoning is effective, achieving up to a 3.02\u00d7 speed improvement on our experimental prototype compared to state-of-the-art algorithms.\n          <\/jats:p>","DOI":"10.1145\/3701995","type":"journal-article","created":{"date-parts":[[2024,10,28]],"date-time":"2024-10-28T09:49:32Z","timestamp":1730108972000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["DeepZoning: Re-accelerate CNN Inference with Zoning Graph for Heterogeneous Edge Cluster"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2182-2228","authenticated-orcid":false,"given":"Jingyu","family":"Wang","sequence":"first","affiliation":[{"name":"State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-2401-5297","authenticated-orcid":false,"given":"Ruilong","family":"Ma","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8300-0270","authenticated-orcid":false,"given":"Xiang","family":"Yang","sequence":"additional","affiliation":[{"name":"Meituan, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0829-4624","authenticated-orcid":false,"given":"Qi","family":"Qi","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3345-1732","authenticated-orcid":false,"given":"Zirui","family":"Zhuang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4627-6307","authenticated-orcid":false,"given":"Jing","family":"Wang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1486-0573","authenticated-orcid":false,"given":"Jianxin","family":"Liao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9831-2202","authenticated-orcid":false,"given":"Song","family":"Guo","sequence":"additional","affiliation":[{"name":"The Hong Kong Polytechnic University, Hong Kong, Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,3,21]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467078"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.3390\/s21030779"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2021.3100298"},{"key":"e_1_3_1_5_2","unstructured":"Moran Shkolnik Brian Chmiel Ron Banner Gil Shomron Yury Nahshan Alex Bronstein and Uri Weiser. 2020. Robust quantization: One model to rule them all. In Proceedings of the 34th International Conference on Neural Information Processing Systems (Vancouver BC Canada). Curran Associates Inc. Article 446 10 pages."},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3519598"},{"issue":"53","key":"e_1_3_1_7_2","first-page":"157","article-title":"V12. 1: User\u2019s manual for CPLEX","volume":"46","author":"Cplex IBM ILOG","year":"2009","unstructured":"IBM ILOG Cplex. 2009. V12. 1: User\u2019s manual for CPLEX. International Business Machines Corporation 46, 53 (2009), 157.","journal-title":"International Business Machines Corporation"},{"key":"e_1_3_1_8_2","first-page":"13678","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Du Xuefeng","year":"2022","unstructured":"Xuefeng Du, Xin Wang, Gabriel Gozum, and Yixuan Li. 2022. Unknown-aware object detection: Learning what you don\u2019t know from videos in the wild. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 13678\u201313688."},{"issue":"1","key":"e_1_3_1_9_2","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1016\/j.dcan.2022.04.032","article-title":"Edge robotics: Are we ready? An experimental evaluation of current vision and future directions","volume":"9","author":"Groshev Milan","year":"2023","unstructured":"Milan Groshev, Gabriele Baldoni, Luca Cominardi, Antonio de la Oliva, and Robert Gazda. 2023. Edge robotics: Are we ready? An experimental evaluation of current vision and future directions. Digital Communications and Networks 9, 1 (2023), 166\u2013174.","journal-title":"Digital Communications and Networks"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_11_2","first-page":"4062","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"He Mingjie","year":"2022","unstructured":"Mingjie He, Jie Zhang, Shiguang Shan, and Xilin Chen. 2022. Enhancing face recognition with self-supervised 3D reconstruction. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 4062\u20134071."},{"key":"e_1_3_1_12_2","first-page":"1097","volume-title":"Proceedings of the 2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS\u201922)","author":"Hou Xueyu","year":"2022","unstructured":"Xueyu Hou, Yongjie Guan, Tao Han, and Ning Zhang. 2022. Distredge: Speeding up convolutional neural network inference on distributed edge devices. In Proceedings of the 2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS\u201922). IEEE, 1097\u20131107."},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM48880.2022.9796896"},{"key":"e_1_3_1_14_2","unstructured":"Bert Hubert. [n. d.]. tc\u2014show\/manipulate traffic control settings. 8 ([n. d.]). https:\/\/linux.die.net\/man\/8\/tc"},{"key":"e_1_3_1_15_2","first-page":"2279","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Jia Zhihao","year":"2018","unstructured":"Zhihao Jia, Sina Lin, Charles R. Qi, and Alex Aiken. 2018. Exploring hidden dimensions in parallelizing convolutional neural networks. In Proceedings of the International Conference on Machine Learning. 2279\u20132288."},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3093337.3037698"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","unstructured":"Mijung Kim and K. Sel\u00e7uk Candan. 2012. SBV-Cut: Vertex-cut based graph partitioning using structural balance vertices. Data Knowl. Eng. 72 (Feb. 2012) 285\u2013303. 10.1016\/j.datak.2011.11.004","DOI":"10.1016\/j.datak.2011.11.004"},{"key":"e_1_3_1_18_2","first-page":"18750","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Kim Minchul","year":"2022","unstructured":"Minchul Kim, Anil K. Jain, and Xiaoming Liu. 2022. AdaFace: Quality adaptive margin for face recognition. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 18750\u201318759."},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3065386"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TWC.2019.2946140"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/PADSW.2018.8645013"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3409964.3461828"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2022.3187772"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","unstructured":"Zengpeng Li Huiqun Yu Guisheng Fan Jiayin Zhang and Jin Xu. 2024. Energy-efficient offloading for DNN-based applications in edge-cloud computing: A hybrid chaotic evolutionary approach. J. Parallel Distrib. Comput. 187 C (May 2024) 15 pages. 10.1016\/j.jpdc.2024.104850","DOI":"10.1016\/j.jpdc.2024.104850"},{"key":"e_1_3_1_25_2","first-page":"1396","volume-title":"Proceedings of the Design, Automation and Test in Europe Conference and Exhibition (DATE\u201917)","author":"Mao Jiachen","year":"2017","unstructured":"Jiachen Mao, Xiang Chen, Kent W Nixon, Christopher Krieger, and Yiran Chen. 2017. Modnn: Local distributed mobile computing system for deep neural network. In Proceedings of the Design, Automation and Test in Europe Conference and Exhibition (DATE\u201917). IEEE, 1396\u20131401."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCAD.2017.8203852"},{"key":"e_1_3_1_27_2","first-page":"749","volume-title":"Proceedings of the 2022 14th International Conference on COMmunication Systems and NETworkS (COMSNETS\u201922)","author":"Parthasarathy Arjun","year":"2022","unstructured":"Arjun Parthasarathy and Bhaskar Krishnamachari. 2022. DEFER: Distributed edge inference for deep neural networks. In Proceedings of the 2022 14th International Conference on COMmunication Systems and NETworkS (COMSNETS\u201922). IEEE, 749\u2013753."},{"key":"e_1_3_1_28_2","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. CoRR abs\/1409.1556 (2014). https:\/\/api.semanticscholar.org\/CorpusID:14124313"},{"key":"e_1_3_1_29_2","first-page":"77","volume-title":"Embedded Computer Systems: Architectures, Modeling, and Simulation: 19th International Conference, SAMOS 2019, Samos, Greece, July 7\u201311, 2019, Proceedings 19","author":"Stahl Rafael","year":"2019","unstructured":"Rafael Stahl, Zhuoran Zhao, Daniel Mueller-Gritschneder, Andreas Gerstlauer, and Ulf Schlichtmann. 2019. Fully distributed deep learning inference on resource-constrained edge devices. In Embedded Computer Systems: Architectures, Modeling, and Simulation: 19th International Conference, SAMOS 2019, Samos, Greece, July 7\u201311, 2019, Proceedings 19. Springer, 77\u201390."},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_3_1_32_2","doi-asserted-by":"crossref","first-page":"9897","DOI":"10.1109\/ICPR48806.2021.9412841","volume-title":"Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR\u201921)","author":"Trusov Anton","year":"2021","unstructured":"Anton Trusov, Elena Limonova, Dmitry Slugin, Dmitry Nikolaev, and Vladimir V. Arlazarov. 2021. Fast implementation of 4-bit convolutional neural networks for mobile devices. In Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR\u201921). IEEE, 9897\u20139903."},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPSW52791.2021.00128"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","unstructured":"Shaohua Wan Songtao Ding and Chen Chen. 2022. Edge computing enabled video segmentation for real-time traffic monitoring in internet of vehicles. Pattern Recognition 121 C (Jan. 2022) 108146. 10.1016\/j.patcog.2021.108146","DOI":"10.1016\/j.patcog.2021.108146"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","unstructured":"Jianxin Wang Ming K. Lim Chao Wang and Ming-Lang Tseng. 2021. The evolution of the Internet of Things (IoT) over the past 20 years. Computers & Industrial Engineering 155 (2021) 107174. 10.1016\/j.cie.2021.107174","DOI":"10.1016\/j.cie.2021.107174"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447993.3448625"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","unstructured":"Carole-Jean Wu David Brooks Kevin Chen Douglas Chen Sy Choudhury Marat Dukhan Kim Hazelwood Eldad Isaac Yangqing Jia Bill Jia Tommer Leyvand Hao Lu Yang Lu Lin Qiao Brandon Reagen Joe Spisak Fei Sun Andrew Tulloch Peter Vajda Xiaodong Wang Yanghan Wang Bram Wasti Yiming Wu Ran Xian Sungjoo Yoo and Peizhao Zhang. 2019. Machine learning at facebook: Understanding inference at the edge. In 2019 IEEE International Symposium on High Performance Computer Architecture (HPCA). 331\u2013344. 10.1109\/HPCA.2019.00048","DOI":"10.1109\/HPCA.2019.00048"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.634"},{"key":"e_1_3_1_39_2","first-page":"1","volume-title":"Proceedings of the 56th Annual Design Automation Conference 2019","author":"Xu Zirui","year":"2019","unstructured":"Zirui Xu, Fuxun Yu, Chenchen Liu, and Xiang Chen. 2019. Reform: Static and dynamic resource-aware dnn reconfiguration framework for mobile device. In Proceedings of the 56th Annual Design Automation Conference 2019. 1\u20136."},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2022.3177782"},{"key":"e_1_3_1_41_2","first-page":"12","volume-title":"Proceedings of the 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS\u201921)","author":"Yang Xiang","year":"2021","unstructured":"Xiang Yang, Qi Qi, Jingyu Wang, Song Guo, and Jianxin Liao. 2021. Towards efficient inference: Adaptively cooperate in heterogeneous iot edge cluster. In Proceedings of the 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS\u201921). IEEE, 12\u201323."},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3274783.3274840"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNET.2020.3042320"},{"key":"e_1_3_1_44_2","article-title":"Robust video anomaly detection framework via prior knowledge and multi-path frame prediction","author":"Zhang Menghao","year":"2023","unstructured":"Menghao Zhang, Jingyu Wang, Jing Wang, Q. Qi, Zirui Zhuang, Haifeng Sun, and Ning Xiao. 2023. Robust video anomaly detection framework via prior knowledge and multi-path frame prediction. In Proceedings of the ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2023).","journal-title":"In Proceedings of the ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2021.3058532"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3350755.3400267"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2018.2858384"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3318216.3363312"},{"key":"e_1_3_1_49_2","first-page":"9581","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Zhou Wenzhang","year":"2022","unstructured":"Wenzhang Zhou, Dawei Du, Libo Zhang, Tiejian Luo, and Yanjun Wu. 2022. Multi-granularity alignment domain adaptation for object detection. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 9581\u20139590."}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3701995","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3701995","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:57:16Z","timestamp":1750298236000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3701995"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,21]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,3,31]]}},"alternative-id":["10.1145\/3701995"],"URL":"https:\/\/doi.org\/10.1145\/3701995","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"type":"print","value":"1544-3566"},{"type":"electronic","value":"1544-3973"}],"subject":[],"published":{"date-parts":[[2025,3,21]]},"assertion":[{"value":"2024-02-17","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-03","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-21","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}