{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,27]],"date-time":"2025-07-27T07:52:35Z","timestamp":1753602755115,"version":"3.41.0"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"1-2","license":[{"start":{"date-parts":[[2024,2,13]],"date-time":"2024-02-13T00:00:00Z","timestamp":1707782400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2022YFB4500701"],"award-info":[{"award-number":["2022YFB4500701"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Science Foundation of China","doi-asserted-by":"crossref","award":["62032001, 62032008, 62372011"],"award-info":[{"award-number":["62032001, 62032008, 62372011"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Syst."],"published-print":{"date-parts":[[2024,5,31]]},"abstract":"<jats:p>The tiered-memory system can effectively expand the memory capacity for virtual machines (VMs). However, virtualization introduces new challenges specifically in enforcing performance isolation, minimizing context switching, and providing resource overcommit. None of the state-of-the-art designs consider virtualization and address these challenges; we observe that a VM with tiered memory incurs up to a 2\u00d7 slowdown compared to a DRAM-only VM.<\/jats:p>\n          <jats:p>\n            We propose\n            <jats:italic>vTMM<\/jats:italic>\n            , a hardware-software collaborative tiered-memory management framework for virtualization. A key insight in\n            <jats:italic>vTMM<\/jats:italic>\n            is to leverage the unique system features in virtualization to meet the above challenges.\n            <jats:italic>vTMM<\/jats:italic>\n            automatically determines page hotness and migrates pages between fast and slow memory to achieve better performance. Specially,\n            <jats:italic>vTMM<\/jats:italic>\n            optimizes page tracking and migration based on page-modification logging (PML), a hardware-assisted virtualization mechanism, and adaptively distinguishes hot\/cold pages through the page \u201ctemperature\u201d sorting.\n            <jats:italic>vTMM<\/jats:italic>\n            also dynamically adjusts fast memory among multi-VMs on demand by using a memory pool. Further,\n            <jats:italic>vTMM<\/jats:italic>\n            tracks huge pages at regular-page granularity in hardware and splits\/merges pages in software, realizing hybrid-grained page management and optimization. We implement and evaluate\n            <jats:italic>vTMM<\/jats:italic>\n            with single-grained page management on an Intel processor, and the hybrid-grained page management on a Sunway processor with hardware mode supporting hardware\/software co-designs. Experiments show that\n            <jats:italic>vTMM<\/jats:italic>\n            outperforms existing tiered-memory management designs in virtualization.\n          <\/jats:p>","DOI":"10.1145\/3639564","type":"journal-article","created":{"date-parts":[[2024,1,15]],"date-time":"2024-01-15T11:27:31Z","timestamp":1705318051000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Hardware-Software Collaborative Tiered-Memory Management Framework for Virtualization"],"prefix":"10.1145","volume":"42","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2506-7212","authenticated-orcid":false,"given":"Sai","family":"Sha","sequence":"first","affiliation":[{"name":"Peking University and Beijing Huawei Digital Technologies, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-7449-5105","authenticated-orcid":false,"given":"Chuandong","family":"Li","sequence":"additional","affiliation":[{"name":"Peking University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6951-1613","authenticated-orcid":false,"given":"Xiaolin","family":"Wang","sequence":"additional","affiliation":[{"name":"Peking University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0429-4371","authenticated-orcid":false,"given":"Zhenlin","family":"Wang","sequence":"additional","affiliation":[{"name":"Michigan Technological University, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7903-0717","authenticated-orcid":false,"given":"Yingwei","family":"Luo","sequence":"additional","affiliation":[{"name":"Peking University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,2,13]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"crossref","first-page":"631","DOI":"10.1145\/3037697.3037706","volume-title":"Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems","author":"Agarwal Neha","year":"2017","unstructured":"Neha Agarwal and Thomas F. Wenisch. 2017. Thermostat: Application-transparent page management for two-tiered main memory. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems. 631\u2013644."},{"key":"e_1_3_2_3_2","unstructured":"Alpha-Developers. 2020. PALcode for Alpha Microprocessors System Design Guide. Retrieved February 1 2020 from https:\/\/download.majix.org\/dec\/palcode_dsgn_gde.pdf"},{"key":"e_1_3_2_4_2","article-title":"The GAP benchmark suite","author":"Beamer S.","year":"2015","unstructured":"S. Beamer, K. Asanovi, and D. Patterson. 2015.The GAP benchmark suite. arXiv e-prints (2015). http:\/\/arxiv.org\/abs\/1508.03619","journal-title":"arXiv e-prints"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3453933.3454018"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.5555\/1077084"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/s002360050103"},{"key":"e_1_3_2_8_2","unstructured":"Linux community. 2021. Intel PML Buffer Flush. https:\/\/elixir.bootlin.com\/linux\/v5.4\/source\/arch\/x86\/kvm\/vmx\/vmx.c"},{"key":"e_1_3_2_9_2","unstructured":"Linux community. 2022. migrate_pages() Function of Linux Kernel. https:\/\/elixir.bootlin.com\/linux\/v5.4\/source\/mm\/migrate.c#L1399"},{"key":"e_1_3_2_10_2","unstructured":"Qemu community. 2021. QEMU. https:\/\/www.qemu.org\/"},{"key":"e_1_3_2_11_2","unstructured":"CXL Consortium.2022. CXL. https:\/\/www.computeexpresslink.org\/"},{"key":"e_1_3_2_12_2","volume-title":"Proceedings of the 1st ACM Symposium on Cloud Computing, SoCC 2010, Indianapolis, Indiana, USA, June 10-11, 2010","author":"Cooper B. F.","year":"2010","unstructured":"B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. 2010. Benchmarking cloud serving systems with YCSB. In Proceedings of the 1st ACM Symposium on Cloud Computing, SoCC 2010, Indianapolis, Indiana, USA, June 10-11, 2010."},{"key":"e_1_3_2_13_2","unstructured":"Jonathan Corbet. 2021. The multi-generational LRU. https:\/\/lwn.net\/Articles\/851184\/"},{"key":"e_1_3_2_14_2","unstructured":"Intel Corporation. 2022. Page Modification Logging for Virtual Machine Monitor White Paper. https:\/\/www.intel.com\/content\/dam\/www\/public\/us\/en\/documents\/white-papers\/page-modification-logging-vmm-white-paper.pdf"},{"key":"e_1_3_2_15_2","unstructured":"Standard Performance Evaluation Corporation. 2022. SPEC CPU 2006 benchmarks. http:\/\/www.spec.org\/cpu2006"},{"key":"e_1_3_2_16_2","unstructured":"Standard Performance Evaluation Corporation. 2022. SPEC CPU 2017 benchmarks. http:\/\/www.spec.org\/cpu2017"},{"key":"e_1_3_2_17_2","unstructured":"Sunway Processor R&D Department. 2017. Sunway 1621 Processor Software Interface Manual. Retrieved February 1 2021 from http:\/\/www.swcpu.cn\/uploadfile\/2018\/0709\/20180709030859351.pdf"},{"key":"e_1_3_2_18_2","unstructured":"Graph500 developers. 2022. Graph500. http:\/\/graph500.org\/"},{"key":"e_1_3_2_19_2","unstructured":"NUMA developers. 2022. A NUMA API for Linux. http:\/\/developer.amd.com\/wordpress\/media\/2012\/10\/LibNUMA-WP-fv1.pdf"},{"key":"e_1_3_2_20_2","volume-title":"IEEE Design Automation Conference","author":"Dhiman G.","year":"2009","unstructured":"G. Dhiman, R. Ayoub, and T. Rosing. 2009. PDRAM: A hybrid PRAM and DRAM main memory system. In IEEE Design Automation Conference."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3341301.3359637"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/2669594.2669599"},{"key":"e_1_3_2_23_2","first-page":"231","volume-title":"2014  \\(\\lbrace\\) USENIX \\(\\rbrace\\)  Annual Technical Conference ( \\(\\lbrace\\) USENIX \\(\\rbrace\\)  \\(\\lbrace\\) ATC \\(\\rbrace\\)  14)","author":"Gaud Fabien","year":"2014","unstructured":"Fabien Gaud, Baptiste Lepers, Jeremie Decouchant, Justin Funston, Alexandra Fedorova, and Vivien Qu\u00e9ma. 2014. Large pages may be harmful on NUMA systems. In 2014 \\(\\lbrace\\) USENIX \\(\\rbrace\\) Annual Technical Conference ( \\(\\lbrace\\) USENIX \\(\\rbrace\\) \\(\\lbrace\\) ATC \\(\\rbrace\\) 14). 231\u2013242."},{"key":"e_1_3_2_24_2","article-title":"Ginkgo: Automated, application-driven memory overcommitment for cloud computing","author":"Gordon Abel","year":"2011","unstructured":"Abel Gordon, Michael Hines, Dilma Da Silva, Muli Ben-Yehuda, Marcio Silva, and Gabriel Lizarraga. 2011. Ginkgo: Automated, application-driven memory overcommitment for cloud computing. Proc. RESoLVE (2011), 1\u20136.","journal-title":"Proc. RESoLVE"},{"key":"e_1_3_2_25_2","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1145\/2987550.2987570","volume-title":"Proceedings of the Seventh ACM Symposium on Cloud Computing","author":"Hirofuchi Takahiro","year":"2016","unstructured":"Takahiro Hirofuchi and Ryousei Takano. 2016. RAMinate: Hypervisor-based virtualization for hybrid main memory systems. In Proceedings of the Seventh ACM Symposium on Cloud Computing. 112\u2013125."},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240302.3240420"},{"key":"e_1_3_2_27_2","first-page":"351","volume-title":"2016 USENIX Annual Technical Conference (USENIX ATC 16)","author":"Hu Xiameng","year":"2016","unstructured":"Xiameng Hu, Xiaolin Wang, Lan Zhou, Yingwei Luo, Chen Ding, and Zhenlin Wang. 2016. Kinetic modeling of data eviction in cache. In 2016 USENIX Annual Technical Conference (USENIX ATC 16). 351\u2013364."},{"key":"e_1_3_2_28_2","unstructured":"Intel Inc.2021. Intel Optane DC Persistent Memory. https:\/\/www.intel.com\/content\/www\/us\/en\/architecture-and-technology\/optane-dc-persistent-memory.html"},{"key":"e_1_3_2_29_2","unstructured":"Intel Inc.2022. Intel 64 and IA-32 Architectures Software Developer\u2019s Manual Volume 3B. https:\/\/www.intel.cn\/content\/www\/cn\/zh\/architecture-and-technology\/64-ia-32-architectures-software-developer-vol-3b-part-2-manual.html"},{"key":"e_1_3_2_30_2","unstructured":"Intel Inc.2022. Intel 64 and IA-32 Architectures Software Developer\u2019s Manual Volume 3C. https:\/\/www.intel.cn\/content\/www\/cn\/zh\/architecture-and-technology\/64-ia-32-architectures-software-developer-vol-3c-part-3-manual.html"},{"key":"e_1_3_2_31_2","unstructured":"Intel Inc.2022. PerfMon Events. https:\/\/perfmon-events.intel.com\/#"},{"key":"e_1_3_2_32_2","doi-asserted-by":"crossref","unstructured":"RISC-V International. 2020. RISC-V: The Free and Open RISC Instruction Set Architecture. Retrieved February 1 2021 from https:\/\/riscv.org\/","DOI":"10.1016\/B978-0-12-820064-3.00024-6"},{"key":"e_1_3_2_33_2","unstructured":"Joseph Izraelevitz Jian Yang Lu Zhang Juno Kim Xiao Liu Amirsaman Memaripour Yun Joon Soh Zixuan Wang Yi Xu Subramanya R. Dulloor Jishen Zhao and Steven Swanson. 2019. Basic performance measurements of the Intel Optane DC persistent memory module. arXiv preprint arXiv:1903.05714 (2019)."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3079856.3080245"},{"key":"e_1_3_2_35_2","unstructured":"The kernel development community. 2022. DAMON. https:\/\/damonitor.github.io\/doc\/html\/latest-damon\/vm\/damon\/index.html"},{"key":"e_1_3_2_36_2","first-page":"715","volume-title":"2021 USENIX Annual Technical Conference, USENIX ATC 2021, July 14-16, 2021","author":"Kim Jonghyeon","year":"2021","unstructured":"Jonghyeon Kim, Wonkyo Choe, and Jeongseob Ahn. 2021. Exploring the design space of page management for multi-tiered memory systems. In 2021 USENIX Annual Technical Conference, USENIX ATC 2021, July 14-16, 2021, Irina Calciu and Geoff Kuenning (Eds.). USENIX Association, 715\u2013728. https:\/\/www.usenix.org\/conference\/atc21\/presentation\/kim-jonghyeon"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/mc.2003.1250880"},{"key":"e_1_3_2_38_2","unstructured":"Redis Ltd.2022. Redis. https:\/\/redis.io\/"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/2810103.2813694"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3582016.3582063"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/2892242.2892258"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3190508.3190537"},{"key":"e_1_3_2_43_2","unstructured":"Western Digital Corporation or its affiliates and other contributors. 2020. RISC-V Open Source Supervisor Binary Interface (OpenSBI). Retrieved February 1 2020 from https:\/\/github.com\/riscv\/opensbi"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3623652.3623668"},{"key":"e_1_3_2_45_2","first-page":"1","volume-title":"Proceedings of the 9th Asia-Pacific Workshop on Systems","author":"Pan Cheng","year":"2018","unstructured":"Cheng Pan, Xiameng Hu, Lan Zhou, Yingwei Luo, Xiaolin Wang, and Zhenlin Wang. 2018. PACE: Penalty aware cache modeling with enhanced AET. In Proceedings of the 9th Asia-Pacific Workshop on Systems. 1\u20138."},{"key":"e_1_3_2_46_2","article-title":"Scalable high performance main memory system using phase-change memory technology","author":"Qureshi Moinuddin K.","year":"2009","unstructured":"Moinuddin K. Qureshi, Vijayalakshmi Srinivasan, and Jude A. Rivers. 2009. Scalable high performance main memory system using phase-change memory technology. Computer Architecture News (2009).","journal-title":"Computer Architecture News"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477132.3483550"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3453933.3454012"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2022.3145671"},{"key":"e_1_3_2_50_2","unstructured":"Rik van Riel and Vinod Chegu. 2022. Automatic NUMA Balancing. https:\/\/www.redhat.com\/files\/summit\/2014\/summit2014_riel_chegu_w_0340_automatic_numa_balancing.pdf"},{"key":"e_1_3_2_51_2","article-title":"Big data oriented hybrid memory systems","author":"Wang X.","year":"2018","unstructured":"X. Wang, X. Liao, H. Liu, and H. Jin. 2018. Big data oriented hybrid memory systems. Big Data Research (2018).","journal-title":"Big Data Research"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3297858.3304024"},{"key":"e_1_3_2_53_2","first-page":"17","volume-title":"2020 USENIX Annual Technical Conference (USENIX ATC 20)","author":"Yao Ting","year":"2020","unstructured":"Ting Yao, Yiwen Zhang, Jiguang Wan, Qiu Cui, Liu Tang, Hong Jiang, Changsheng Xie, and Xubin He. 2020. atrixKV: Reducing write stalls and write amplification in LSM-tree based KV stores with matrix container in NVM. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). 17\u201331."}],"container-title":["ACM Transactions on Computer Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3639564","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3639564","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:56Z","timestamp":1750291436000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3639564"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,13]]},"references-count":52,"journal-issue":{"issue":"1-2","published-print":{"date-parts":[[2024,5,31]]}},"alternative-id":["10.1145\/3639564"],"URL":"https:\/\/doi.org\/10.1145\/3639564","relation":{},"ISSN":["0734-2071","1557-7333"],"issn-type":[{"type":"print","value":"0734-2071"},{"type":"electronic","value":"1557-7333"}],"subject":[],"published":{"date-parts":[[2024,2,13]]},"assertion":[{"value":"2023-08-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-13","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}