{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,14]],"date-time":"2026-05-14T23:22:01Z","timestamp":1778800921354,"version":"3.51.4"},"reference-count":28,"publisher":"Association for Computing Machinery (ACM)","issue":"3","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62376226"],"award-info":[{"award-number":["62376226"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Natural Science Basic Research Program of Shaanxi","award":["2025JC-YBQN-888"],"award-info":[{"award-number":["2025JC-YBQN-888"]}]},{"name":"Technology Innovation Leading Program of Shaanxi","award":["2024QCY-KXJ-094"],"award-info":[{"award-number":["2024QCY-KXJ-094"]}]},{"name":"Xi\u2019an Key Technology Research Projects for Key Agricultural Industry Chains","award":["2024JH-NYZD-0027"],"award-info":[{"award-number":["2024JH-NYZD-0027"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Embed. Comput. Syst."],"published-print":{"date-parts":[[2026,5,31]]},"abstract":"<jats:p>As a new generation of edge devices, the integrated CPU\/GPU architecture has opened up new opportunities for deploying different scale vision models. In order to reduce models\u2019 inference time on the integrated devices, this article first compresses deep learning models using model quantization. The quantization process greatly reduces the computation requirements of a model, which enables its deployment on embedded development boards. However, quantization also leads to lower GPU resource utilization during inference on integrated devices. This insufficient utilization results in slower inference speed. To address this problem, this article first depicts the data flow of model inference within an integrated device. Secondly, this article implements a unified memory management between the CPU and GPU based on managed memory strategy. Finally, this article designs a thread-level stream scheduling method to improve GPU utilization and throughput during model inference in a pipeline way. Experimental results show that the proposed method achieves a 2x-10x improvement in throughput compared with the TensorRT\u2019s default scheduling method, which is crucial for realizing real-time inference tasks on edge devices.<\/jats:p>","DOI":"10.1145\/3771550","type":"journal-article","created":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T11:13:10Z","timestamp":1760008390000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["A Thread-Level Stream Scheduling Method for Accelerating LVMs\u2019 Inference on a Resource-Constrained Platform"],"prefix":"10.1145","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4791-9762","authenticated-orcid":false,"given":"Yijie","family":"Chen","sequence":"first","affiliation":[{"name":"Northwest A&F University","place":["Yangling, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-9533-433X","authenticated-orcid":false,"given":"Jiaqi","family":"Han","sequence":"additional","affiliation":[{"name":"Northwest A&F University","place":["Yangling, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9388-0198","authenticated-orcid":false,"given":"Bin","family":"Liu","sequence":"additional","affiliation":[{"name":"Northwest A&F University","place":["Yangling, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-1321-7288","authenticated-orcid":false,"given":"Xinzhe","family":"Zhang","sequence":"additional","affiliation":[{"name":"Northwest A&F University","place":["Yangling, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-1683-1676","authenticated-orcid":false,"given":"Zijian","family":"Hu","sequence":"additional","affiliation":[{"name":"Northwest A&F University","place":["Yangling, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-2844-2908","authenticated-orcid":false,"given":"Rongyu","family":"Dou","sequence":"additional","affiliation":[{"name":"Northwest A&F University","place":["Yangling, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5224-4048","authenticated-orcid":false,"given":"Keqin","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Computer Science, State University of New York at New Paltz","place":["New Paltz, United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,5,14]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"crossref","unstructured":"Siqi Wang Gayathri Ananthanarayanan Yifan Zeng Neeraj Goel Anuj Pathania and Tulika Mitra. 2019. High-throughput cnn inference on embedded arm big. little multicore processors. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39 10 (2019) 2254\u20132267.","DOI":"10.1109\/TCAD.2019.2944584"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/RTAS48715.2020.00007"},{"key":"e_1_3_1_4_2","first-page":"1","article-title":"YOLOv4-5D: An effective and efficient object detector for autonomous driving","volume":"70","author":"Cai Yingfeng","year":"2021","unstructured":"Yingfeng Cai, Tianyu Luan, Hongbo Gao, Hai Wang, Long Chen, Yicheng Li, Miguel Angel Sotelo, and Zhixiong Li. 2021. YOLOv4-5D: An effective and efficient object detector for autonomous driving. IEEE Transactions on Instrumentation and Measurement 70 (2021), 1\u201313.","journal-title":"IEEE Transactions on Instrumentation and Measurement"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCC.2021.3132929"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.sysarc.2023.102990"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.6018"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3092255.3092256"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW59228.2023.00444"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2023.3290092"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/MNET.011.2000639"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3498361.3538932"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.01286"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.3390\/computers12030060"},{"key":"e_1_3_1_15_2","volume-title":"Jetson Modules","year":"2024","unstructured":"NVIDIA. 2024. Jetson Modules. Retrieved from https:\/\/developer.nvidia.com\/embedded\/jetson-modules"},{"key":"e_1_3_1_16_2","volume-title":"Jetson Nano Developer Kit","year":"2024","unstructured":"NVIDIA. 2024. Jetson Nano Developer Kit. Retrieved from https:\/\/developer.nvidia.com\/embedded\/jetson-nano-developer-kit"},{"key":"e_1_3_1_17_2","volume-title":"Jetson Xavier NX Developer Kit","year":"2024","unstructured":"NVIDIA. 2024. Jetson Xavier NX Developer Kit. Retrieved from https:\/\/developer.nvidia.com\/embedded\/jetson-xavier-nx-devkit"},{"key":"e_1_3_1_18_2","volume-title":"GPU Technology Conference","author":"Peri Dheeraj","year":"2020","unstructured":"Dheeraj Peri, Jhalak Patel, and Josh Park. 2020. Deploying quantization-aware trained networks using TensorRT. In GPU Technology Conference."},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10586-022-03916-5"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/INMIC50486.2020.9318136"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/IOTM.001.2200152"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV56688.2023.00348"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3264828"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/AICAS48895.2020.9073977"},{"issue":"3","key":"e_1_3_1_25_2","first-page":"1275","article-title":"A co-scheduling framework for DNN models on mobile and edge devices with heterogeneous hardware","volume":"22","author":"Xu Zhiyuan","year":"2021","unstructured":"Zhiyuan Xu, Dejun Yang, Chengxiang Yin, Jian Tang, Yanzhi Wang, and Guoliang Xue. 2021. A co-scheduling framework for DNN models on mobile and edge devices with heterogeneous hardware. IEEE Transactions on Mobile Computing 22, 3 (2021), 1275\u20131288.","journal-title":"IEEE Transactions on Mobile Computing"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/GCWkshps58843.2023.10465019"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/TWC.2021.3088910"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2021.3068255"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCC-DSS-SmartCity-DependSys57074.2022.00299"}],"container-title":["ACM Transactions on Embedded Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3771550","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,14]],"date-time":"2026-05-14T11:41:51Z","timestamp":1778758911000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3771550"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,14]]},"references-count":28,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,5,31]]}},"alternative-id":["10.1145\/3771550"],"URL":"https:\/\/doi.org\/10.1145\/3771550","relation":{},"ISSN":["1539-9087","1558-3465"],"issn-type":[{"value":"1539-9087","type":"print"},{"value":"1558-3465","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,14]]},"assertion":[{"value":"2024-11-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-04","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-14","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}