{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,13]],"date-time":"2026-09-13T12:48:05Z","timestamp":1789303685975,"version":"build-2803163510"},"reference-count":53,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2024,1,18]],"date-time":"2024-01-18T00:00:00Z","timestamp":1705536000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"U.S. National Science Foundation","doi-asserted-by":"crossref","award":["CNS-CCF-1942182"],"award-info":[{"award-number":["CNS-CCF-1942182"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2024,3,31]]},"abstract":"<jats:p>The abstraction of a shared memory space over separate CPU and GPU memory domains has eased the burden of portability for many HPC codebases. However, users pay for ease of use provided by system-managed memory with a moderate-to-high performance overhead. NVIDIA Unified Virtual Memory (UVM) is currently the primary real-world implementation of such abstraction and offers a functionally equivalent testbed for in-depth performance study for both UVM and future Linux Heterogeneous Memory Management (HMM) compatible systems. The continued advocacy for UVM and HMM motivates improvement of the underlying system. We focus on UVM-based systems and investigate the root causes of UVM overhead, a non-trivial task due to complex interactions of multiple hardware and software constituents and the desired cost granularity.<\/jats:p>\n                  <jats:p>In our prior work, we delved deeply into UVM system architecture and showed internal behaviors of page fault servicing in batches. We provided quantitative evaluation of batch handling for various applications under different scenarios, including prefetching and oversubscription. We revealed that the driver workload depends on the interactions among application access patterns, GPU hardware constraints, and host OS components. Host OS components have significant overhead present across implementations, warranting close attention.<\/jats:p>\n                  <jats:p>This extension furthers our prior study in three aspects: fine-grain cost analysis and breakdown, extension to multiple GPUs, and investigation of platforms with different GPU-GPU interconnects. We take a top-down approach to quantitative batch analysis and uncover how constituent component costs accumulate and overlap, governed by synchronous and asynchronous operations. Our multi-GPU analysis shows reduced cost of GPU-GPU batch workloads compared to CPU-GPU workloads. We further demonstrate that while specialized interconnects, NVLink, can improve batch cost, their benefits are limited by host OS software overhead and GPU oversubscription. This study serves as a proxy for future shared memory systems, such as those that interface with HMM, and the development of interconnects.<\/jats:p>","DOI":"10.1145\/3632953","type":"journal-article","created":{"date-parts":[[2023,11,14]],"date-time":"2023-11-14T06:30:36Z","timestamp":1699943436000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Fine-grain Quantitative Analysis of Demand Paging in Unified Virtual Memory"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0253-372X","authenticated-orcid":false,"given":"Tyler","family":"Allen","sequence":"first","affiliation":[{"name":"University of North Carolina at Charlotte, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-0375-9418","authenticated-orcid":false,"given":"Bennett","family":"Cooper","sequence":"additional","affiliation":[{"name":"Clemson University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2218-3675","authenticated-orcid":false,"given":"Rong","family":"Ge","sequence":"additional","affiliation":[{"name":"Clemson University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,1,18]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"High Performance Geometric Multigrid. 2022. Retrieved July 13 2021 from https:\/\/crd.lbl.gov\/divisions\/amcr\/computer-science-amcr\/par\/research\/hpgmg\/"},{"key":"e_1_3_1_3_2","volume-title":"Holistic Performance Analysis and Optimization of Unified Virtual Memory","author":"Allen Tyler","year":"2022","unstructured":"Tyler Allen. 2022. Holistic Performance Analysis and Optimization of Unified Virtual Memory. Ph.D. Dissertation. Clemson University."},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS49936.2021.00023"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3458817.3480855"},{"key":"e_1_3_1_6_2","first-page":"27","volume-title":"2017 USENIX Annual Technical Conference (USENIX ATC\u201917)","author":"Amit Nadav","year":"2017","unstructured":"Nadav Amit. 2017. Optimizing the TLB shootdown algorithm with page access tracking. In 2017 USENIX Annual Technical Conference (USENIX ATC\u201917). USENIX Association, Santa Clara, CA, 27\u201339. https:\/\/www.usenix.org\/conference\/atc17\/technical-sessions\/presentation\/amit"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3123939.3123975"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","unstructured":"Trinayan Baruah Yifan Sun Ali Tolga Din\u00e7er Saiful A. Mojumder Jos\u00e9 L. Abell\u00e1n Yash Ukidave Ajay Joshi Norman Rubin John Kim and David Kaeli. 2020. Griffin: Hardware-software support for efficient page migration in multi-GPU systems. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE 596\u2013609. DOI:10.1109\/HPCA47549.2020.00055","DOI":"10.1109\/HPCA47549.2020.00055"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/ScalA51936.2020.00012"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/P3HPC49587.2019.00012"},{"key":"e_1_3_1_11_2","unstructured":"Manuel Birke Bobby Philip Zhen Wang and Mark Berrill. 2019. Block-relaxation Methods for 3D Constant-Coefficient Stencils on GPUs and Multicore CPUs. (2019). arxiv:cs.DC\/1208.1975"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2014.07.003"},{"key":"e_1_3_1_13_2","first-page":"1","volume-title":"Proceedings of the 14th ACM International Conference on Systems and Storage","author":"Chang Chia-Hao","year":"2021","unstructured":"Chia-Hao Chang, Adithya Kumar, and Anand Sivasubramaniam. 2021. To move or not to move? Page migration for irregular applications in over-subscribed GPU memory systems with DynaMap. In Proceedings of the 14th ACM International Conference on Systems and Storage. 1\u201312."},{"key":"e_1_3_1_14_2","article-title":"cuDNN: Efficient primitives for deep learning","volume":"1410","author":"Chetlur Sharan","year":"2014","unstructured":"Sharan Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, and Evan Shelhamer. 2014. cuDNN: Efficient primitives for deep learning. ArXiv abs\/1410.0759 (2014).","journal-title":"ArXiv"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/mchpc49590.2019.00014"},{"key":"e_1_3_1_16_2","unstructured":"Linux Kernel Development Community. Heterogeneous Memory Management (HMM). 2023. Retrieved May 25 2021 from https:\/\/www.kernel.org\/doc\/html\/latest\/mm\/hmm.html"},{"key":"e_1_3_1_17_2","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1007\/978-3-319-46079-6_34","volume-title":"High Performance Computing","author":"Deakin Tom","year":"2016","unstructured":"Tom Deakin, James Price, Matt Martineau, and Simon McIntosh-Smith. 2016. GPU-STREAM v2.0: Benchmarking the achievable memory bandwidth of many-core processors across diverse parallel programming models. In High Performance Computing, Michela Taufer, Bernd Mohr, and Julian M. Kunkel (Eds.). Springer International Publishing, Cham, 489\u2013507."},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1177\/1094342015593158"},{"key":"e_1_3_1_19_2","doi-asserted-by":"crossref","first-page":"1212","DOI":"10.23919\/DATE51398.2021.9473982","volume-title":"2021 Design, Automation & Test in Europe Conference & Exhibition (DATE\u201921)","author":"Ganguly Debashis","year":"2021","unstructured":"Debashis Ganguly, Rami Melhem, and Jun Yang. 2021. An adaptive framework for oversubscription management in CPU-GPU unified memory. In 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE\u201921). IEEE, 1212\u20131217."},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3307650.3322224"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ipdps47924.2020.00054"},{"key":"e_1_3_1_22_2","first-page":"41","volume-title":"2019 IEEE\/ACM Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS\u201919)","author":"Gayatri Rahulkumar","year":"2019","unstructured":"Rahulkumar Gayatri, Kevin Gott, and Jack Deslippe. 2019. Comparing managed memory and ATS with and without prefetching on NVIDIA volta GPUs. In 2019 IEEE\/ACM Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS\u201919). 41\u201346."},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.14778\/3384345.3384358"},{"key":"e_1_3_1_24_2","unstructured":"Yongbin Gu Wenxuan Wu Yunfan Li and Lizhong Chen. 2020. UVMBench: A Comprehensive Benchmark Suite for Researching Unified Virtual Memory in GPUs. (2020). arXiv:2007.09822."},{"key":"e_1_3_1_25_2","first-page":"637","volume-title":"Proceedings of the 23rd International Conference on Architectural Support for Programming Languages and Operating Systems","author":"Haria Swapnil","year":"2018","unstructured":"Swapnil Haria, Mark D. Hill, and Michael M. Swift. 2018. Devirtualizing memory in heterogeneous systems. In Proceedings of the 23rd International Conference on Architectural Support for Programming Languages and Operating Systems. 637\u2013650."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/1089014.1089021"},{"key":"e_1_3_1_27_2","unstructured":"John Hubbard and Jerome Glisee. 2017. GPUs: HMM: Heterogeneous Memory Management. https:\/\/www.redhat.com\/files\/summit\/session-assets\/2017\/S104078-hubbard.pdf"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3373376.3378529"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11227-019-02966-8"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPEC.2014.7040988"},{"key":"e_1_3_1_31_2","unstructured":"Michael Larabel. AMD Making Progress on HMM-based SVM Memory Manager for Open-Source Compute. 2021. https:\/\/www.phoronix.com\/news\/AMD-ROCm-HMM-SVM-Memory"},{"key":"e_1_3_1_32_2","volume-title":"2023 IEEE International Symposium on High Performance Computer Architecture (HPCA\u201923)","author":"Li Bingyao","year":"2023","unstructured":"Bingyao Li, Jieming Yin, Anup Holey, Youtao Zhang, Jun Yang, and Xulong Tang. 2023. Trans-FW: Short circuiting page table walk in multi-GPU systems via remote forwarding. In 2023 IEEE International Symposium on High Performance Computer Architecture (HPCA\u201923)."},{"key":"e_1_3_1_33_2","first-page":"49","volume-title":"Proceedings of the 24th International Conference on Architectural Support for Programming Languages and Operating Systems","author":"Li Chen","year":"2019","unstructured":"Chen Li, Rachata Ausavarungnirun, Christopher J. Rossbach, Youtao Zhang, Onur Mutlu, Yang Guo, and Jun Yang. 2019. A framework for memory oversubscription management in graphics processing units. In Proceedings of the 24th International Conference on Architectural Support for Programming Languages and Operating Systems. 49\u201363."},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.23919\/DATE.2018.8342166"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389705"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3300053.3319419"},{"key":"e_1_3_1_37_2","unstructured":"Seung Won Min Vikram Sharma Mailthody Zaid Qureshi Jinjun Xiong Eiman Ebrahimi and Wen mei Hwu. 2020. EMOGI: Efficient Memory-access for Out-of-memory Graph-traversal In GPUs. (2020). arXiv:2006.06890."},{"key":"e_1_3_1_38_2","unstructured":"Saiful A. Mojumder Yifan Sun Leila Delshadtehrani Yenai Ma Trinayan Baruah Jos\u00e9 L. Abell\u00e1n John Kim David Kaeli and Ajay Joshi. 2020. MGPU-TSM: A Multi-GPU System with Truly Shared Memory. (2020). arxiv:2008.02300."},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2015.2463813"},{"key":"e_1_3_1_40_2","unstructured":"NVIDIA. Open GPU Documentation. (????). Retrieved May 25 2021 from https:\/\/nvidia.github.io\/open-gpu-doc\/"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1006\/jcph.1995.1039"},{"key":"e_1_3_1_42_2","unstructured":"Steve Plimpton. 2017. FFTs for (Mostly) Particle Codes within the DOE Exascale Computing Program. https:\/\/www.osti.gov\/servlets\/purl\/1483229"},{"key":"e_1_3_1_43_2","doi-asserted-by":"crossref","unstructured":"Zaid Qureshi Vikram Sharma Mailthody Isaac Gelado Seung Won Min Amna Masood Jeongmin Park Jinjun Xiong C. J. Newburn Dmitri Vainbrand I. Chung Michael Garland William Dally and Wen-mei Hwu. 2023. GPU-Initiated on-demand high-throughput storage access in the bam system architecture. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS\u201923). Vol. 2 Association for Computing Machinery 325\u2013339.","DOI":"10.1145\/3575693.3575748"},{"key":"e_1_3_1_44_2","unstructured":"Nikolay Sakharnykh. 2016. High-performance Geometric Multi-Grid with GPU Acceleration. Retrieved May 25 2021 from https:\/\/developer.nvidia.com\/blog\/high-performance-geometric-multi-grid-gpu-acceleration\/"},{"key":"e_1_3_1_45_2","unstructured":"Nikolay Sakharnykh. 2019. Memory Management on Modern GPU Architectures. https:\/\/developer.download.nvidia.com\/video\/gputechconf\/gtc\/2019\/presentation\/s9727-memory-management-on-modern-gpu-architectures.pdf"},{"key":"e_1_3_1_46_2","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1145\/3489525.3511691","volume-title":"Proceedings of the 2022 ACM\/SPEC on International Conference on Performance Engineering","author":"Shao Chuanming","year":"2022","unstructured":"Chuanming Shao, Jinyang Guo, Pengyu Wang, Jing Wang, Chao Li, and Minyi Guo. 2022. Oversubscribing GPU unified virtual memory: Implications and suggestions. In Proceedings of the 2022 ACM\/SPEC on International Conference on Performance Engineering. 67\u201375."},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/1810085.1810120"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3503222.3507771"},{"key":"e_1_3_1_49_2","volume-title":"Evaluation and Design of FFT for Distributed Accelerated Systems","author":"Tomov Stanimire","year":"2018","unstructured":"Stanimire Tomov, Azzam Haidar, Daniel Schultz, and Jack Dongarra. 2018. Evaluation and Design of FFT for Distributed Accelerated Systems. ECP WBS 2.3.3.09 Milestone Report FFT-ECP ST-MS-10-1216. Innovative Computing Laboratory, University of Tennessee. Revision 10-2018."},{"key":"e_1_3_1_50_2","article-title":"Enabling large-reach TLBs for high-throughput processors by exploiting memory subregion contiguity","volume":"2110","author":"Yu Chao","year":"2021","unstructured":"Chao Yu, Yuebin Bai, and Rui Wang. 2021. Enabling large-reach TLBs for high-throughput processors by exploiting memory subregion contiguity. CoRR abs\/2110.08613 (2021). https:\/\/arxiv.org\/abs\/2110.08613","journal-title":"CoRR"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","unstructured":"Qi Yu Bruce Childers Libo Huang Cheng Qian Hui Guo and Zhiying Wang. 2020. Coordinated page prefetch and eviction for memory oversubscription management in GPUs. In International Parallel and Distributed Processing Symposium (IPDPS\u201920). IEEE 472\u2013482. DOI:10.1109\/IPDPS47924.2020.00056","DOI":"10.1109\/IPDPS47924.2020.00056"},{"key":"e_1_3_1_52_2","doi-asserted-by":"crossref","unstructured":"Qi Yu Bruce Childers Libo Huang Cheng Qian and Zhiying Wang. 2020. HPE: Hierarchical page eviction policy for unified memory in GPUs. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39 10 (2020) 2461\u20132474.","DOI":"10.1109\/TCAD.2019.2944790"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11227-019-03079-y"},{"key":"e_1_3_1_54_2","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1109\/HPCA.2016.7446077","volume-title":"2016 IEEE International Symposium on High Performance Computer Architecture (HPCA\u201916)","author":"Zheng Tianhao","year":"2016","unstructured":"Tianhao Zheng, David Nellans, Arslan Zulfiqar, Mark Stephenson, and Stephen W. Keckler. 2016. Towards high performance paged memory for GPUs. In 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA\u201916). 345\u2013357."}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3632953","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3632953","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T12:35:50Z","timestamp":1750163750000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3632953"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,18]]},"references-count":53,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,3,31]]}},"alternative-id":["10.1145\/3632953"],"URL":"https:\/\/doi.org\/10.1145\/3632953","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"value":"1544-3566","type":"print"},{"value":"1544-3973","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,18]]},"assertion":[{"value":"2023-03-31","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-10-23","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-01-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}