{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T15:56:51Z","timestamp":1782835011231,"version":"3.54.5"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2025,3,19]],"date-time":"2025-03-19T00:00:00Z","timestamp":1742342400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2025,3,31]]},"abstract":"<jats:p>In recent years, deploying deep learning models on edge devices has become pervasive, driven by the increasing demand for intelligent edge computing solutions across various industries. From industrial automation to intelligent surveillance and healthcare, edge devices are being leveraged for real-time analytics and decision-making. Existing methods face two challenges when deploying machine learning models on edge devices. The first challenge is handling the execution order of operators with a simple strategy, which can lead to a potential waste of memory resources when dealing with directed acyclic graph structure models. The second challenge is that they usually process operators of a model one by one to optimize the inference latency, which may lead to the optimization problem getting trapped in local optima.<\/jats:p>\n          <jats:p>We present MemoriaNova, comprising BTSearch and GenEFlow, to solve these two problems. BTSearch is a graph state backtracking algorithm with efficient pruning and hashing strategies designed to minimize memory overhead during inference and enlarge latency optimization search space. GenEFlow, based on genetic algorithms (GA), integrates latency modeling, and memory constraints to optimize distributed inference latency. This innovative approach considers a comprehensive search space for model partitioning, ensuring robust and adaptable solutions. We implement BTSearch and GenEFlow and test them on 11 deep-learning models with different structures and scales. The results show that BTSearch can reach 12% memory optimization compared with the widely used random execution strategy. At the same time, GenEFlow reduces inference latency by 33.9% in distributed systems with four-edge devices.<\/jats:p>","DOI":"10.1145\/3701997","type":"journal-article","created":{"date-parts":[[2024,10,28]],"date-time":"2024-10-28T09:49:32Z","timestamp":1730108972000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["MemoriaNova: Optimizing Memory-Aware Model Inference for Edge Computing"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-3022-5174","authenticated-orcid":false,"given":"Renjun","family":"Zhang","sequence":"first","affiliation":[{"name":"Shanghai Jiao Tong University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-9530-8344","authenticated-orcid":false,"given":"Tianming","family":"Zhang","sequence":"additional","affiliation":[{"name":"Shanghai Jiao Tong University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9373-8474","authenticated-orcid":false,"given":"Zinuo","family":"Cai","sequence":"additional","affiliation":[{"name":"Shanghai Jiao Tong University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5462-5773","authenticated-orcid":false,"given":"Dongmei","family":"Li","sequence":"additional","affiliation":[{"name":"Beijing Institute of Microelectronics Technology, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9592-8490","authenticated-orcid":false,"given":"Ruhui","family":"Ma","sequence":"additional","affiliation":[{"name":"Computer Science, Shanghai Jiao Tong University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9754-6496","authenticated-orcid":false,"given":"Buyya","family":"Rajkumar","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Melbourne, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,3,19]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2013.57"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00383444"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2023.3303877"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2023.3345401"},{"issue":"9","key":"e_1_3_1_6_2","article-title":"Smart video surveillance system based on edge computing","volume":"21","author":"Cob-Parro Antonio Carlos","year":"2021","unstructured":"Antonio Carlos Cob-Parro, Cristina Losada-Guti\u00e9rrez, Marta Marr\u00f3n-Romera, Alfredo Gardel-Vicente, and Ignacio Bravo-Mu\u00f1oz. 2021. Smart video surveillance system based on edge computing. Sensors 21, 9 (2021).","journal-title":"Sensors"},{"issue":"3","key":"e_1_3_1_7_2","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1021\/acs.accounts.9b00470","article-title":"A design-to-device pipeline for data-driven materials discovery","volume":"53","author":"Cole Jacqueline M.","year":"2020","unstructured":"Jacqueline M. Cole. 2020. A design-to-device pipeline for data-driven materials discovery. Accounts Chem. Res. 53, 3 (2020), 599\u2013610.","journal-title":"Accounts Chem. Res."},{"key":"e_1_3_1_8_2","article-title":"Bert: Pre-training of deep bidirectional transformers for language understanding","volume":"1810","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. CoRR abs\/1810.04805, (2018). Retrieved from https:\/\/arxiv.org\/abs\/1810.04805","journal-title":"CoRR"},{"key":"e_1_3_1_9_2","first-page":"167","article-title":"Ios: Inter-operator scheduler for cnn acceleration","volume":"3","author":"Ding Yaoyao","year":"2021","unstructured":"Yaoyao Ding, Ligeng Zhu, Zhihao Jia, Gennady Pekhimenko, and Song Han. 2021. Ios: Inter-operator scheduler for cnn acceleration. Proc. Mach. Learn. Syst. 3 (2021), 167\u2013180.","journal-title":"Proc. Mach. Learn. Syst."},{"issue":"3","key":"e_1_3_1_10_2","doi-asserted-by":"crossref","first-page":"2206","DOI":"10.3390\/su15032206","article-title":"Convolutional neural network-based personalized program recommendation system for smart television users","volume":"15","author":"Dudekula Khasim Vali","year":"2023","unstructured":"Khasim Vali Dudekula, Hussain Syed, Mohamed Iqbal Mahaboob Basha, Sudhakar Ilango Swamykan, Purna Prakash Kasaraneni, Yellapragada Venkata Pavan Kumar, Aymen Flah, and Ahmad Taher Azar. 2023. Convolutional neural network-based personalized program recommendation system for smart television users. Sustainability 15, 3 (2023), 2206.","journal-title":"Sustainability"},{"issue":"7","key":"e_1_3_1_11_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3544836","article-title":"Scheduling IoT applications in edge and fog computing environments: A taxonomy and future directions","volume":"55","author":"Goudarzi Mohammad","year":"2022","unstructured":"Mohammad Goudarzi, Marimuthu Palaniswami, and Rajkumar Buyya. 2022. Scheduling IoT applications in edge and fog computing environments: A taxonomy and future directions. Comput. Surv. 55, 7 (2022), 1\u201341.","journal-title":"Comput. Surv."},{"issue":"2","key":"e_1_3_1_12_2","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1007\/s42979-022-01641-9","article-title":"IoT-Enhanced smart door locking system with security","volume":"4","author":"Guntur Jalalu","year":"2023","unstructured":"Jalalu Guntur, S. Srinivasulu Raju, T. Niranjan, Sai Kiran Kilaru, Rakesh Dronavalli, and N. Surya Seshu Kumar. 2023. IoT-Enhanced smart door locking system with security. SN Comput. Sci. 4, 2 (2023), 209.","journal-title":"SN Comput. Sci."},{"key":"e_1_3_1_13_2","first-page":"545","volume-title":"Machine Learning and Principles and Practice of Knowledge Discovery in Databases","author":"Guo Xiaotian","year":"2023","unstructured":"Xiaotian Guo, Andy D. Pimentel, and Todor Stefanov. 2023. Hierarchical design space exploration for distributed CNN inference at the edge. In Machine Learning and Principles and Practice of Knowledge Discovery in Databases. Springer Nature Switzerland, Cham, 545\u2013556."},{"key":"e_1_3_1_14_2","article-title":"Deep residual learning for image recognition","volume":"1512","author":"He Kaiming","year":"2015","unstructured":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep residual learning for image recognition. CoRR abs\/1512.03385, (2015). Retrieved from http:\/\/arxiv.org\/abs\/1512.03385","journal-title":"CoRR"},{"key":"e_1_3_1_15_2","article-title":"Searching for MobileNetV3","volume":"1905","author":"Howard Andrew","year":"2019","unstructured":"Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, Quoc V. Le, and Hartwig Adam. 2019. Searching for MobileNetV3. CoRR abs\/1905.02244, (2019). Retrieved from http:\/\/arxiv.org\/abs\/1905.02244","journal-title":"CoRR"},{"key":"e_1_3_1_16_2","article-title":"MobileNets: Efficient convolutional neural networks for mobile vision applications","volume":"1704","author":"Howard Andrew G.","year":"2017","unstructured":"Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. MobileNets: Efficient convolutional neural networks for mobile vision applications. CoRR abs\/1704.04861, (2017). Retrieved from http:\/\/arxiv.org\/abs\/1704.04861","journal-title":"CoRR"},{"key":"e_1_3_1_17_2","first-page":"330","volume-title":"Proceedings of the IEEE Conference on Computer Communications (INFOCOM\u201922)","author":"Hu Chenghao","year":"2022","unstructured":"Chenghao Hu and Baochun Li. 2022. Distributed inference with deep learning models across heterogeneous edge devices. In Proceedings of the IEEE Conference on Computer Communications (INFOCOM\u201922). 330\u2013339."},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3081333.3081360"},{"key":"e_1_3_1_19_2","article-title":"SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size","volume":"1602","author":"Iandola Forrest N.","year":"2016","unstructured":"Forrest N. Iandola, Matthew W. Moskewicz, Khalid Ashraf, Song Han, William J. Dally, and Kurt Keutzer. 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size. CoRR abs\/1602.07360, (2016). Retrieved from http:\/\/arxiv.org\/abs\/1602.07360","journal-title":"CoRR"},{"key":"e_1_3_1_20_2","unstructured":"Jazzbin et al.2020. Geatpy: The genetic and evolutionary algorithm toolbox with high performance in Python."},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2024.3360448"},{"key":"e_1_3_1_22_2","first-page":"135","volume-title":"The Greedy Algorithm","author":"Jungnickel Dieter","year":"2013","unstructured":"Dieter Jungnickel. 2013. The Greedy Algorithm. Springer, Berlin, 135\u2013161."},{"issue":"2","key":"e_1_3_1_23_2","article-title":"Face recognition systems: A survey","volume":"20","author":"Kortli Yassin","year":"2020","unstructured":"Yassin Kortli, Maher Jridi, Ayman Al Falou, and Mohamed Atri. 2020. Face recognition systems: A survey. Sensors 20, 2 (2020).","journal-title":"Sensors"},{"key":"e_1_3_1_24_2","article-title":"Patent claim generation by fine-tuning OpenAI GPT-2","volume":"1907","author":"Lee Jieh-Sheng","year":"2019","unstructured":"Jieh-Sheng Lee and Jieh Hsiang. 2019. Patent claim generation by fine-tuning OpenAI GPT-2. CoRR abs\/1907.02052, (2019). Retrieved from http:\/\/arxiv.org\/abs\/1907.02052","journal-title":"CoRR"},{"key":"e_1_3_1_25_2","unstructured":"Zehan Li Xin Zhang Yanzhao Zhang Dingkun Long Pengjun Xie and Meishan Zhang. 2023. Towards general text embeddings with multi-stage contrastive learning. Retrieved from https:\/\/arxiv.org\/abs\/2308.03281"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3210240.3210337"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46448-0_2"},{"issue":"1","key":"e_1_3_1_28_2","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1007\/s10107-022-01906-4","article-title":"Resolvent splitting for sums of monotone operators with minimal lifting","volume":"201","author":"Malitsky Yura","year":"2023","unstructured":"Yura Malitsky and Matthew K. Tam. 2023. Resolvent splitting for sums of monotone operators with minimal lifting. Math. Program. 201, 1 (2023), 231\u2013262.","journal-title":"Math. Program."},{"key":"e_1_3_1_29_2","first-page":"854","volume-title":"Proceedings of the IEEE Conference on Computer Communications (INFOCOM\u201920)","author":"Mohammed Thaha","year":"2020","unstructured":"Thaha Mohammed, Carlee Joe-Wong, Rohit Babbar, and Mario Di Francesco. 2020. Distributed inference acceleration with adaptive DNN partitioning and offloading. In Proceedings of the IEEE Conference on Computer Communications (INFOCOM\u201920). IEEE, 854\u2013863."},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE53745.2022.00241"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jretconser.2022.103186"},{"key":"e_1_3_1_32_2","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201920)","author":"Radosavovic Ilija","year":"2020","unstructured":"Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollar. 2020. Designing network design spaces. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201920)."},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.690"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11633-022-1391-7"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2023.3280970"},{"key":"e_1_3_1_36_2","unstructured":"Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. Retrieved from https:\/\/arxiv.org\/abs\/1409.1556"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1155\/2018\/7068349"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2023.10.012"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3489517.3530472"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3516807.3516820"},{"key":"e_1_3_1_43_2","first-page":"1","volume-title":"Proceedings of the IEEE Conference on Computer Communications (INFOCOM\u201923)","author":"Xu Zhiying","year":"2023","unstructured":"Zhiying Xu, Hongding Peng, and Wei Wang. 2023. AGO: Boosting mobile AI inference performance by removing constraints on graph optimization. In Proceedings of the IEEE Conference on Computer Communications (INFOCOM\u201923). 1\u201310."},{"key":"e_1_3_1_44_2","article-title":"Compressing deep neural network structures for sensing systems with a compressor-critic framework","volume":"1706","author":"Yao Shuochao","year":"2017","unstructured":"Shuochao Yao, Yiran Zhao, Aston Zhang, Lu Su, and Tarek F. Abdelzaher. 2017. Compressing deep neural network structures for sensing systems with a compressor-critic framework. CoRR abs\/1706.01215, (2017). Retrieved from http:\/\/arxiv.org\/abs\/1706.01215","journal-title":"CoRR"},{"issue":"5","key":"e_1_3_1_45_2","doi-asserted-by":"crossref","first-page":"1123","DOI":"10.1016\/j.dcan.2022.09.010","article-title":"Accurate threat hunting in industrial internet of things edge devices","volume":"9","author":"Yazdinejad Abbas","year":"2023","unstructured":"Abbas Yazdinejad, Behrouz Zolfaghari, Ali Dehghantanha, Hadis Karimipour, Gautam Srivastava, and Reza M Parizi. 2023. Accurate threat hunting in industrial internet of things edge devices. Digit. Commun. Netw. 9, 5 (2023), 1123\u20131130.","journal-title":"Digit. Commun. Netw."},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2017.2762418"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNET.2020.3042320"},{"issue":"2","key":"e_1_3_1_48_2","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1109\/JSAC.2022.3227023","article-title":"OSTTD: Offloading of splittable tasks with topological dependence in multi-tier computing networks","volume":"41","author":"Zhang Rui","year":"2022","unstructured":"Rui Zhang, Xuesen Chu, Ruhui Ma, Meng Zhang, Liwei Lin, Honghao Gao, and Haibing Guan. 2022. OSTTD: Offloading of splittable tasks with topological dependence in multi-tier computing networks. IEEE J. Select. Areas Commun. 41, 2 (2022), 555\u2013568.","journal-title":"IEEE J. Select. Areas Commun."},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2018.2858384"}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3701997","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3701997","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:57:16Z","timestamp":1750298236000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3701997"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,19]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,3,31]]}},"alternative-id":["10.1145\/3701997"],"URL":"https:\/\/doi.org\/10.1145\/3701997","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"value":"1544-3566","type":"print"},{"value":"1544-3973","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,19]]},"assertion":[{"value":"2024-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-19","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-19","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}