{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T21:39:20Z","timestamp":1768340360267,"version":"3.49.0"},"reference-count":46,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,5,6]],"date-time":"2024-05-06T00:00:00Z","timestamp":1714953600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Science Foundation of China","doi-asserted-by":"crossref","award":["62172031"],"award-info":[{"award-number":["62172031"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Beijing Municipal Natural Science Foundation","award":["L191019"],"award-info":[{"award-number":["L191019"]}]},{"name":"Key Natural Science Foundation of Education Department of Anhui","award":["KJ2021A0046"],"award-info":[{"award-number":["KJ2021A0046"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Internet Technol."],"published-print":{"date-parts":[[2024,5,31]]},"abstract":"<jats:p>Deep learning technology has grown significantly in new application scenarios such as smart cities and driverless vehicles, but its deployment needs to consume a lot of resources. It is usually difficult to execute inference task solely on resource-constrained Intelligent Internet-of-Things (IoT) devices to meet strictly service delay requirements. CNN-based inference task is usually offloaded to the edge server or cloud. However, it may lead to unstable performance and privacy leaks. To address the above challenges, this article aims to design a low latency distributed inference framework, EdgeCI, which assigns inference tasks to locally idle, connected, and resource-constrained IoT device cluster networks. EdgeCI exploits two key optimization knobs, including: (1) Auction-based Workload Assignment Scheme (AWAS), which achieves the workload balance by assigning each workload partition to the more matching IoT device; (2) Fused-Layer parallelization strategy based on non-recursive Dynamic Programming (DPFL), which is aimed at further minimizing the inference time. We have implemented EdgeCI based on PyTorch and evaluated its performance with VGG-16 and ResNet-34 image recognition models. The experimental results prove that our proposed AWAS and DPFL outperform the typical state-of-the-art solutions. When they are well combined, EdgeCI can improve inference speed by 34.72% to 43.52%. EdgeCI outperforms the state-of-the art approaches on our edge cluster.<\/jats:p>","DOI":"10.1145\/3656041","type":"journal-article","created":{"date-parts":[[2024,4,2]],"date-time":"2024-04-02T13:38:49Z","timestamp":1712065129000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["EdgeCI: Distributed Workload Assignment and Model Partitioning for CNN Inference on Edge Clusters"],"prefix":"10.1145","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2747-6637","authenticated-orcid":false,"given":"Yanming","family":"Chen","sequence":"first","affiliation":[{"name":"School of Compute Science and Technology, Anhui University, Hefei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-5541-0341","authenticated-orcid":false,"given":"Tong","family":"Luo","sequence":"additional","affiliation":[{"name":"School of Compute Science and Technology, Anhui University, Hefei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6407-7467","authenticated-orcid":false,"given":"Weiwei","family":"Fang","sequence":"additional","affiliation":[{"name":"School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0394-4635","authenticated-orcid":false,"given":"Neal. N.","family":"Xiong","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Mathematics, Sul Ross State University, Alpine, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,5,6]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2019.2948888"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/QRS-C51114.2020.00045"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3297858.3304011"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/MNET.011.2000195"},{"key":"e_1_3_1_6_2","article-title":"TD-MDB: A truth discovery based multi-dimensional bidding strategy for federated learning in industrial IoT systems","author":"Zeng Pengjie","year":"2023","unstructured":"Pengjie Zeng, Anfeng Liu, Neal N. Xiong, Shaobo Zhang, and Mianxiong Dong. 2023. TD-MDB: A truth discovery based multi-dimensional bidding strategy for federated learning in industrial IoT systems. IEEE Internet Things J. (2023).","journal-title":"IEEE Internet Things J."},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2020.2990979"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2017.08.022"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM.2018.8486241"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2020.2984333"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2017.2761740"},{"key":"e_1_3_1_12_2","article-title":"Multi-agent collaborative inference via DNN decoupling: Intermediate feature compression and edge learning","author":"Hao Zhiwei","year":"2022","unstructured":"Zhiwei Hao, Guanyu Xu, Yong Luo, Han Hu, Jianping An, and Shiwen Mao. 2022. Multi-agent collaborative inference via DNN decoupling: Intermediate feature compression and edge learning. IEEE Trans. Mob. Comput. 22, 10 (2022), 6041\u20136055.","journal-title":"IEEE Trans. Mob. Comput."},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2019.2918951"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2020.2976475"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2020.3044930"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2020.2979669"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM.2019.8737614"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3093337.3037698"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2023.3242730"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2017.226"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICC45855.2022.9839083"},{"key":"e_1_3_1_22_2","article-title":"SGPL: An intelligent game-based secure collaborative communication scheme for metaverse over 5G and beyond networks","author":"Chen Miaojiang","year":"2023","unstructured":"Miaojiang Chen, Anfeng Liu, Neal N. Xiong, Hongbing Song, and Victor C. M. Leung. 2023. SGPL: An intelligent game-based secure collaborative communication scheme for metaverse over 5G and beyond networks. IEEE J. Select. Areas Commun. 42, 3 (2023), 767\u2013782.","journal-title":"IEEE J. Select. Areas Commun."},{"key":"e_1_3_1_23_2","article-title":"The state of mobile network experience","author":"Boyland Peter","year":"2019","unstructured":"Peter Boyland. 2019. The state of mobile network experience. Open Sig.May (2019).","journal-title":"Open Sig."},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.comnet.2022.109150"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.23919\/DATE.2017.7927211"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3318216.3363312"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2020.2972000"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551638"},{"key":"e_1_3_1_29_2","article-title":"PICO: Pipeline inference framework for versatile CNNs on diverse mobile devices","author":"Yang Xiang","year":"2023","unstructured":"Xiang Yang, Zikang Xu, Qi Qi, Jingyu Wang, Haifeng Sun, Jianxin Liao, and Song Guo. 2023. PICO: Pipeline inference framework for versatile CNNs on diverse mobile devices. IEEE Trans. Mob. Comput. 23, 4 (2023), 2712\u20132730.","journal-title":"IEEE Trans. Mob. Comput."},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICC45855.2022.9838547"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2022.10.033"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2022.11.008"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2018.2858384"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNET.2020.3042320"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS53621.2022.00110"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-95388-1_21"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2021.3058532"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/TWC.2018.2881705"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2019.2933890"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM.2019.8737532"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM41043.2020.9155237"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2022.3205410"},{"key":"e_1_3_1_43_2","article-title":"A distributed algorithm for the assignment problem","author":"Bertsekas Dimitri P.","year":"1979","unstructured":"Dimitri P. Bertsekas. 1979. A distributed algorithm for the assignment problem. Lab. Inf. Decis. Syst. Work. Pap., MIT (1979).","journal-title":"Lab. Inf. Decis. Syst. Work. Pap., MIT"},{"issue":"1","key":"e_1_3_1_44_2","first-page":"39","article-title":"The auction algorithm: A distributed relaxation method for the assignment problem","volume":"36","author":"Bertsekas Dimitri P.","year":"1988","unstructured":"Dimitri P. Bertsekas and John N. Tsitsiklis. 1988. The auction algorithm: A distributed relaxation method for the assignment problem. Biased Rand. Walks 36, 1 (1988), 39\u201358.","journal-title":"Biased Rand. Walks"},{"key":"e_1_3_1_45_2","article-title":"tc\u2013show\/manipulate traffic control settings","author":"Hubert Bert","year":"2001","unstructured":"Bert Hubert. 2001. tc\u2013show\/manipulate traffic control settings. Linux Manpage. Retrieved from https:\/\/man7.org\/linux\/man-pages\/man8\/tc","journal-title":"Linux Manpage. Retrieved from"},{"key":"e_1_3_1_46_2","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).","journal-title":"arXiv preprint arXiv:1409.1556"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"}],"container-title":["ACM Transactions on Internet Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3656041","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3656041","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:48Z","timestamp":1750291428000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3656041"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,6]]},"references-count":46,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,5,31]]}},"alternative-id":["10.1145\/3656041"],"URL":"https:\/\/doi.org\/10.1145\/3656041","relation":{},"ISSN":["1533-5399","1557-6051"],"issn-type":[{"value":"1533-5399","type":"print"},{"value":"1557-6051","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,6]]},"assertion":[{"value":"2023-05-29","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-26","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}