{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,22]],"date-time":"2026-03-22T22:43:52Z","timestamp":1774219432400,"version":"3.50.1"},"reference-count":44,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,5,11]],"date-time":"2024-05-11T00:00:00Z","timestamp":1715385600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Embed. Comput. Syst."],"published-print":{"date-parts":[[2024,5,31]]},"abstract":"<jats:p>Deep Learning is ubiquitous today and is increasingly moving from the cloud down to the edge of networked infrastructures, where it enables embedded applications to perform complex inference tasks close to the data sources, reducing long-distance data movement and alleviating the need for a powerful cloud infrastructure. Edge-class multi-processor system on chip (MPSoC) devices featuring an on-chip FPGA fabric offer key advantages for Deep Learning inference tasks, especially for complex applications where multiple models may be run concurrently in the same platform. In this work, we propose an approach and a practical framework for the systematic characterization of multithreaded Deep Learning inference on edge FPGA MPSoCs. We instantiate the framework into a real-world MPSoC platform, targeting Xilinx Vitis-AI as a representative example of a commercial Deep Learning acceleration toolkit for edge environments. We design a comprehensive experimental campaign and apply it to the platform for several convolutional neural networks, each trained on three different datasets. We show that our approach can be used for both hardware- and software-level analysis of a target system. Among other findings, the analysis revealed a suboptimal behavior of the underlying toolkit runtime, involving the utilization of the accelerator cores and the uneven software latency of the support library, influenced by the shapes of the input tensors.<\/jats:p>","DOI":"10.1145\/3611015","type":"journal-article","created":{"date-parts":[[2023,8,4]],"date-time":"2023-08-04T09:50:13Z","timestamp":1691142613000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["An Approach to the Systematic Characterization of Multitask Accelerated CNN Inference in Edge MPSoCs"],"prefix":"10.1145","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1685-8736","authenticated-orcid":false,"given":"Alessandro","family":"Cilardo","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering and Information Technology, University of Naples Federico II, Naples, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1631-1597","authenticated-orcid":false,"given":"Vincenzo","family":"Maisto","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Information Technology, University of Naples Federico II, Naples, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0401-9687","authenticated-orcid":false,"given":"Nicola","family":"Mazzocca","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Information Technology, University of Naples Federico II, Naples, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4099-5244","authenticated-orcid":false,"given":"Franca","family":"Rocco Di Torrepadula","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Information Technology, University of Naples Federico II, Naples, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,5,11]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.sysarc.2019.01.007"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSSC.2016.2616357"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2017.2765695"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2021.3124125"},{"key":"e_1_3_1_6_2","unstructured":"Matthieu Courbariaux Itay Hubara Daniel Soudry Ran El-Yaniv and Yoshua Bengio. 2016. Binarized neural networks: Training deep neural networks with weights and activations constrained to +1 or -1. arXiv:1602.02830 (2016). https:\/\/arxiv.org\/abs\/1602.02830"},{"key":"e_1_3_1_7_2","unstructured":"NVIDIA. 2022. CUDA Deep Neural Network (cuDNN) | NVIDIA Developer. (Oct. 2022). Retrieved October 1 2022 from https:\/\/developer.nvidia.com\/cudnn"},{"key":"e_1_3_1_8_2","unstructured":"Xilinx. 2022. DPUv3E for Alveo Accelerator Card with HBM. (Oct. 2022). Retrieved October 1 2022 from https:\/\/www.xilinx.com\/developer\/articles\/dpuv3e-for-alveo-accelerator-card-with-hbm.html"},{"issue":"2","key":"e_1_3_1_9_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3140659.3080246","article-title":"In-datacenter performance analysis of a tensor processing unit","volume":"45","author":"al. Norman P. Jouppi et","year":"2017","unstructured":"Norman P. Jouppi et al. 2017. In-datacenter performance analysis of a tensor processing unit. SIGARCH Computer Architecture News 45, 2 (June 2017), 1\u201312.","journal-title":"SIGARCH Computer Architecture News"},{"key":"e_1_3_1_10_2","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1109\/MICRO50266.2020.00062","volume-title":"Proceedings of the 2020 53rd Annual IEEE\/ACM International Symposium on Microarchitecture (MICRO\u201920)","author":"Ghodrati Soroush","year":"2020","unstructured":"Soroush Ghodrati, Byung Hoon Ahn, Joon Kyung Kim, Sean Kinzer, Brahmendra Reddy Yatham, Navateja Alla, Hardik Sharma, Mohammad Alian, Eiman Ebrahimi, Nam Sung Kim, Cliff Young, and Hadi Esmaeilzadeh. 2020. Planaria: Dynamic architecture fission for spatial multi-tenant acceleration of deep neural networks. In Proceedings of the 2020 53rd Annual IEEE\/ACM International Symposium on Microarchitecture (MICRO\u201920). 681\u2013697."},{"key":"e_1_3_1_11_2","article-title":"Compressing deep convolutional networks using vector quantization","author":"Gong Yunchao","year":"2014","unstructured":"Yunchao Gong, Liu Liu, Ming Yang, and Lubomir Bourdev. 2014. Compressing deep convolutional networks using vector quantization. arXiv preprint arXiv:1412.6115 (2014).","journal-title":"arXiv preprint arXiv:1412.6115"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-021-01453-z"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASPDAC.2017.7858394"},{"key":"e_1_3_1_14_2","first-page":"26","article-title":"[DL] A survey of FPGA-based neural network inference accelerators","volume":"12","author":"Guo Kaiyuan","year":"2019","unstructured":"Kaiyuan Guo, Shulin Zeng, Jincheng Yu, Yu Wang, and Huazhong Yang. 2019. [DL] A survey of FPGA-based neural network inference accelerators. ACM Transactions on Reconfigurable Technology and Systems 12, 1 (March 2019), Article 2, 26 pages.","journal-title":"ACM Transactions on Reconfigurable Technology and Systems"},{"key":"e_1_3_1_15_2","article-title":"A survey on methods and theories of quantized neural networks","volume":"1808","author":"Guo Yunhui","year":"2018","unstructured":"Yunhui Guo. 2018. A survey on methods and theories of quantized neural networks. arXiv abs\/1808.04752 (2018).","journal-title":"arXiv"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3007787.3001163"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_18_2","article-title":"Distilling the knowledge in a neural network","author":"Hinton Geoffrey","year":"2015","unstructured":"Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).","journal-title":"arXiv preprint arXiv:1503.02531"},{"key":"e_1_3_1_19_2","article-title":"MobileNets: Efficient convolutional neural networks for mobile vision applications","author":"Howard Andrew G.","year":"2017","unstructured":"Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017).","journal-title":"arXiv preprint arXiv:1704.04861"},{"key":"e_1_3_1_20_2","unstructured":"Intel. 2021. Intel\u00aeArchitecture Instruction Set Extensions and Future Features 2021. Intel."},{"key":"e_1_3_1_21_2","first-page":"1","volume-title":"Proceedings of the 2022 14th International Conference on Computer Research and Development (ICCRD\u201922)","author":"Jiang Hangyang","year":"2022","unstructured":"Hangyang Jiang, Quande Li, and Yanteng Li. 2022. Post training quantization after neural network. In Proceedings of the 2022 14th International Conference on Computer Research and Development (ICCRD\u201922). IEEE, Los Alamitos, CA, 1\u20136."},{"key":"e_1_3_1_22_2","first-page":"1369","volume-title":"Proceedings of the IEEE Conference on Computer Communications","author":"Jiang Shuang","year":"2020","unstructured":"Shuang Jiang, Zhiyao Ma, Xiao Zeng, Chenren Xu, Mi Zhang, Chen Zhang, and Yunxin Liu. 2020. SCYLLA: QoE-aware continuous mobile vision with FPGA-based dynamic deep neural network reconfiguration. In Proceedings of the IEEE Conference on Computer Communications(IEEE INFOCOM\u201920). 1369\u20131378."},{"key":"e_1_3_1_23_2","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/DSD53832.2021.00012","volume-title":"Proceedings of the 2021 24th Euromicro Conference on Digital System Design (DSD\u201921)","author":"Kalali Ercan","year":"2021","unstructured":"Ercan Kalali and Rene van Leuken. 2021. A power-efficient parameter quantization technique for CNN accelerators. In Proceedings of the 2021 24th Euromicro Conference on Digital System Design (DSD\u201921). IEEE, Los Alamitos, CA, 18\u201323."},{"key":"e_1_3_1_24_2","unstructured":"Kprobes. 2022. Kernel Probes (Kprobes)\u2014The Linux Kernel Documentation. (Oct. 2022). Retrieved October 1 2022 from https:\/\/docs.kernel.org\/trace\/kprobes.html"},{"key":"e_1_3_1_25_2","unstructured":"Perf Wiki. 2022. Linux Perf. (April 2022). Retrieved April 1 2023 from https:\/\/perf.wiki.kernel.org\/index.php\/Main_Page"},{"key":"e_1_3_1_26_2","unstructured":"GitHub. 2022. Xilinx\/linux-xlnx: The Official Linux Kernel from Xilinx. (Oct. 2022). Retrieved October 1 2022 from https:\/\/github.com\/Xilinx\/linux-xlnx\/blob\/master\/drivers\/misc\/xlnx_dpu.c"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i04.5963"},{"key":"e_1_3_1_28_2","first-page":"1","volume-title":"Proceedings of the 2016 26th International Conference on Field Programmable Logic and Applications (FPL\u201916)","author":"Nurvitadhi Eriko","year":"2016","unstructured":"Eriko Nurvitadhi, Jaewoong Sim, David Sheffield, Asit Mishra, Srivatsan Krishnan, and Debbie Marr. 2016. Accelerating recurrent neural networks in analytics servers: Comparison of FPGA, CPU, GPU, and ASIC. In Proceedings of the 2016 26th International Conference on Field Programmable Logic and Applications (FPL\u201916). 1\u20134."},{"key":"e_1_3_1_29_2","unstructured":"NVDLA. 2022. NVDLA Home Page. (April 2022). Retrieved April 1 2023 from http:\/\/nvdla.org\/"},{"key":"e_1_3_1_30_2","unstructured":"OpenVINO. 2022. OpenVINO\u2122 Documentation\u2014Version (latest). (Oct. 2022). Retrieved October 1 2022 from https:\/\/docs.openvino.ai\/latest\/index.html"},{"key":"e_1_3_1_31_2","unstructured":"AMD. 2022. DPUCZDX8G for Zynq UltraScale+ MPSoCs Product Guide (PG338): Introduction. (Oct. 2022). Retrieved October 1 2022 from https:\/\/docs.xilinx.com\/r\/3.4-English\/pg338-dpu\/Introduction"},{"key":"e_1_3_1_32_2","first-page":"5142","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Phuong Mary","year":"2019","unstructured":"Mary Phuong and Christoph Lampert. 2019. Towards understanding knowledge distillation. In Proceedings of the International Conference on Machine Learning. 5142\u20135151."},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.sysarc.2020.101896"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICESS.2019.8782524"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/RTSS52674.2021.00047"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2020.3033730"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2890150"},{"key":"e_1_3_1_38_2","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).","journal-title":"arXiv preprint arXiv:1409.1556"},{"key":"e_1_3_1_39_2","unstructured":"NVIDIA. 2022. TensorRT | NVIDIA Developer. (Oct. 2022). Retrieved October 1 2022 from https:\/\/developer.nvidia.com\/tensorrt"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3020078.3021744"},{"key":"e_1_3_1_41_2","unstructured":"Xilinx. 2022. Vitis-AI. (Oct. 2022). Retrieved October 1 2022 from https:\/\/www.xilinx.com\/products\/design-tools\/vitis\/vitis-ai.html"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00748"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.15"},{"key":"e_1_3_1_44_2","first-page":"3869","volume-title":"Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision","author":"Yvinec Edouard","year":"2023","unstructured":"Edouard Yvinec, Arnaud Dapogny, Matthieu Cord, and Kevin Bailly. 2023. SPIQ: Data-free per-channel static input quantization. In Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision. 3869\u20133878."},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2988311"}],"container-title":["ACM Transactions on Embedded Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3611015","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3611015","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:50:52Z","timestamp":1750287052000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3611015"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,11]]},"references-count":44,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,5,31]]}},"alternative-id":["10.1145\/3611015"],"URL":"https:\/\/doi.org\/10.1145\/3611015","relation":{},"ISSN":["1539-9087","1558-3465"],"issn-type":[{"value":"1539-9087","type":"print"},{"value":"1558-3465","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,11]]},"assertion":[{"value":"2022-11-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-28","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-11","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}