{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,24]],"date-time":"2026-07-24T13:06:37Z","timestamp":1784898397967,"version":"3.55.0"},"reference-count":31,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2026,7,24]],"date-time":"2026-07-24T00:00:00Z","timestamp":1784851200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"name":"Key Research and Development and Promotion Program of Henan Province, China","award":["242102210184 and 252102210247"],"award-info":[{"award-number":["242102210184 and 252102210247"]}]},{"name":"Science and Technology Program of Nanyang City, China","award":["23JCQY2003"],"award-info":[{"award-number":["23JCQY2003"]}]},{"name":"Doctoral Research Foundation of Nanyang Normal University, China","award":["2023ZX008"],"award-info":[{"award-number":["2023ZX008"]}]},{"name":"Joint Fund for Science and Technology Research and Development of Henan Province, China","award":["245101610059"],"award-info":[{"award-number":["245101610059"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2026,9,30]]},"abstract":"<jats:p>In-kernel machine learning (in-kernel ML) enables low-overhead online decision-making for OS subsystems such as scheduling, storage, and security. However, in the current in-kernel-to-dGPU offloading path, inference data must traverse kernel space, user space, and device memory, making the software path and PCIe transfers the dominant costs in the small-batch regime and often erasing acceleration gains, rather than being limited by insufficient dGPU compute capability. To address this issue, this article proposes a hierarchical co-optimization scheme of data reduction, lightweight compression, and protocol optimization: it combines feature selection, mixed-precision quantization, and lightweight compression to reduce transfer load, and uses a state-aware dynamic scheduler to adapt the transfer strategy to changes in PCIe bandwidth, CPU load, and data sparsity. On the dGPU, decompression and dequantization are pipelined with computation to increase overlap. Across multiple in-kernel workloads, our scheme reduces transferred bytes by 50%\u201387.5%, lowers the dGPU break-even batch size from 256 to 64, reduces end-to-end inference latency by 3.2\u00d7\u20135.8\u00d7, and keeps the accuracy loss within 3%.<\/jats:p>","DOI":"10.1145\/3821578","type":"journal-article","created":{"date-parts":[[2026,6,23]],"date-time":"2026-06-23T21:17:51Z","timestamp":1782249471000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Protocol and Software Co-optimization: Overcoming the PCIe Transfer Bottleneck in dGPU Accelerated In-kernel ML"],"prefix":"10.1145","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6024-9698","authenticated-orcid":false,"given":"WenTao","family":"Li","sequence":"first","affiliation":[{"name":"School of Artificial Intelligence, Nanyang Normal University","place":["Nanyang, China"]},{"name":"Nanyang Explosion-proof Electric Apparatus Institute","place":["Nanyang, China"]},{"name":"Collaborative Innovation Center of Intelligent Explosion-proof Equipment","place":["Nanyang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-6570-8872","authenticated-orcid":false,"given":"SongQuan","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Nanyang Normal University","place":["Nanyang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-7568-8650","authenticated-orcid":false,"given":"Panpan","family":"Cuan","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Nanyang Normal University","place":["Nanyang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-1683-3267","authenticated-orcid":false,"given":"Zheng","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Nanyang Normal University","place":["Nanyang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-8731-1552","authenticated-orcid":false,"given":"Jie","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Nanyang Normal University","place":["Nanyang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0823-5846","authenticated-orcid":false,"given":"Qinglei","family":"Qi","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Nanyang Normal University","place":["Nanyang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,7,24]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3465332.3470875"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.52202\/079017-3180"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3669940.3707280"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3409963.3410492"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3575693.3575697"},{"key":"e_1_3_1_7_2","volume-title":"Proceedings of the 9th Workshop on Systems for Multi-core and Heterogeneous Architectures","author":"Gottschlag Mathias","year":"2019","unstructured":"Mathias Gottschlag and Frank Bellosa. 2019. Reducing AVX-Induced frequency variation with core specialization. In Proceedings of the 9th Workshop on Systems for Multi-core and Heterogeneous Architectures."},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383669.3398282"},{"issue":"156","key":"e_1_3_1_9_2","first-page":"2","article-title":"eCryptfs: A stacked cryptographic filesystem","volume":"2007","author":"Halcrow Mike","year":"2007","unstructured":"Mike Halcrow. 2007. eCryptfs: A stacked cryptographic filesystem. Linux Journal 2007, 156 (2007), 2.","journal-title":"Linux Journal"},{"key":"e_1_3_1_10_2","first-page":"173","volume-title":"Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201920)","author":"Hao Mingzhe","year":"2020","unstructured":"Mingzhe Hao, Levent Toksoz, Nanqinqin Li, Edward Edberg Halim, Henry Hoffmann, and Haryadi S. Gunawi. 2020. LinnOS: Predictability on unpredictable flash storage with a light neural network. In Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201920). 173\u2013190."},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.comnet.2024.110188"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.14778\/3749646.3749698"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/RTAS61025.2024.00037"},{"key":"e_1_3_1_14_2","first-page":"401","volume-title":"Proceedings of the 2012 USENIX Annual Technical Conference (USENIX ATC\u201912)","author":"Kato Shinpei","year":"2012","unstructured":"Shinpei Kato, Michael McThrow, Carlos Maltzahn, and Scott Brandt. 2012. Gdev: First-class GPU resource management in the operating system. In Proceedings of the 2012 USENIX Annual Technical Conference (USENIX ATC\u201912). 401\u2013412."},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3458817.3476224"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2019.2928289"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.14778\/3749646.3749710"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2025.3624576"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/2901318.2901326"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3736227.3736240"},{"key":"e_1_3_1_21_2","article-title":"NVIDIA Ada GPU Architecture","author":"Corporation NVIDIA","year":"2022","unstructured":"NVIDIA Corporation. 2022. NVIDIA Ada GPU Architecture. White Paper. Retrieved May 6, 2026 from https:\/\/images.nvidia.com\/aem-dam\/Solutions\/geforce\/ada\/nvidia-ada-gpu-architecture.pdf","journal-title":"White Paper"},{"key":"e_1_3_1_22_2","first-page":"803","volume-title":"Proceedings of the 19th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201925)","author":"Park Yeonhong","year":"2025","unstructured":"Yeonhong Park, Jake Hyun, Hojoon Kim, and Jae W. Lee. 2025. DecDEC: A systems approach to advancing low-bit LLM quantization. In Proceedings of the 19th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201925). 803\u2013819."},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.parco.2014.09.011"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.52202\/079017-2193"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2011.112"},{"key":"e_1_3_1_26_2","series-title":"Proceedings of Machine Learning Research","first-page":"38087","volume-title":"Proceedings of the 40th International Conference on Machine Learning","volume":"202","author":"Xiao Guangxuan","year":"2023","unstructured":"Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. SmoothQuant: Accurate and efficient post-training quantization for large language models. In Proceedings of the 40th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 202). PMLR, 38087\u201338099."},{"key":"e_1_3_1_27_2","first-page":"119","volume-title":"Proceedings of the 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201923)","author":"You Jie","year":"2023","unstructured":"Jie You, Jae-Won Chung, and Mosharaf Chowdhury. 2023. Zeus: Understanding and optimizing GPU energy consumption of DNN training. In Proceedings of the 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201923). 119\u2013139."},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.14778\/3717755.3717780"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3721145.3725743"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3352020.3352027"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.5555\/3767955.3767978"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3779212.3790181"}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3821578","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,24]],"date-time":"2026-07-24T12:57:38Z","timestamp":1784897858000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3821578"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,24]]},"references-count":31,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,9,30]]}},"alternative-id":["10.1145\/3821578"],"URL":"https:\/\/doi.org\/10.1145\/3821578","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"value":"1544-3566","type":"print"},{"value":"1544-3973","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,24]]},"assertion":[{"value":"2026-02-11","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-02","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-07-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}