{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T00:50:00Z","timestamp":1775782200471,"version":"3.50.1"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,8,6]],"date-time":"2024-08-06T00:00:00Z","timestamp":1722902400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100016822","name":"Seagate Technology","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100016822","id-type":"DOI","asserted-by":"crossref"}]},{"name":"European Union\u2019s Horizon 2020"},{"name":"German Federal Ministry of Education and Research"},{"name":"EuroHPC Joint Undertaking (JU) IO-SEA","award":["955811"],"award-info":[{"award-number":["955811"]}]},{"name":"BMBF","award":["16HPC027"],"award-info":[{"award-number":["16HPC027"]}]},{"name":"National Research Data Infrastructure for and with Computer Science"},{"DOI":"10.13039\/501100001659","name":"German Research Foundation","doi-asserted-by":"crossref","award":["501930651"],"award-info":[{"award-number":["501930651"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Storage"],"published-print":{"date-parts":[[2024,11,30]]},"abstract":"<jats:p>Deduplication and compression are powerful techniques to reduce the ratio between the quantity of logical data stored and the physical amount of consumed storage. Deduplication can impose significant performance overheads, as duplicate detection for large systems induces random accesses to the backend storage. These random accesses have led to the concern that deduplication for primary storage and HDDs are not compatible. Most inline data reduction solutions are therefore optimized for SSDs and discourage their use for HDDs, even for sequential workloads.<\/jats:p>\n          <jats:p>In this work, we show that these concerns are valid if and only if the lessons learned from deduplication research are not applied. We have therefore investigated data reduction solutions for primary storage based on the RedHat Virtual Disk Optimizer (VDO) and show that directly applying them can decrease sequential write performance for HDDs by 36\u00d7. We then show that slight modifications to VDO plus the integration of a very small SSD area significantly improve performance even beyond the performance without data reduction enabled, making HDDs more cost-efficient for a wide range of mostly sequential cloud workloads than SSDs. Additionally, these VDO optimizations do not require to maintain different code bases for HDDs and SSDs, and we therefore provide the first data reduction solution applicable to both storage media.<\/jats:p>","DOI":"10.1145\/3678250","type":"journal-article","created":{"date-parts":[[2024,7,23]],"date-time":"2024-07-23T11:58:29Z","timestamp":1721735909000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["From SSDs Back to HDDs: Optimizing VDO to Support Inline Deduplication and Compression for HDDs as Primary Storage Media"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1002-9734","authenticated-orcid":false,"given":"Patrick","family":"Raaf","sequence":"first","affiliation":[{"name":"Johannes Gutenberg-Universit\u00e4t Mainz, Mainz, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3083-2775","authenticated-orcid":false,"given":"Andr\u00e9","family":"Brinkmann","sequence":"additional","affiliation":[{"name":"Zentrum f\u00fcr Datenverarbeitung, Johannes Gutenberg-Universit\u00e4t Mainz, Mainz, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2378-3023","authenticated-orcid":false,"given":"Eric","family":"Borba","sequence":"additional","affiliation":[{"name":"Johannes Gutenberg-Universit\u00e4t Mainz, Mainz, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0264-3865","authenticated-orcid":false,"given":"Hossein","family":"Asadi","sequence":"additional","affiliation":[{"name":"Johannes Gutenberg-Universit\u00e4t Mainz, Mainz, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6448-7239","authenticated-orcid":false,"given":"Sai","family":"Narasimhamurthy","sequence":"additional","affiliation":[{"name":"ParTec, Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7887-0047","authenticated-orcid":false,"given":"John","family":"Bent","sequence":"additional","affiliation":[{"name":"Los Alamos National Laboratories, Los Alamos, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-2122-8990","authenticated-orcid":false,"given":"Mohamad","family":"El-Batal","sequence":"additional","affiliation":[{"name":"Seagate Technology, Oklahoma City, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3786-7102","authenticated-orcid":false,"given":"Reza","family":"Salkhordeh","sequence":"additional","affiliation":[{"name":"Johannes Gutenberg-Universit\u00e4t Mainz, Mainz, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,8,6]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3530896"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/2485732.2485740"},{"key":"e_1_3_2_4_2","volume-title":"Proceedings of the USENIX Annual Technical Conference (ATC\u201906). 85\u201390.","author":"Black John","year":"2006","unstructured":"John Black. 2006. Compare-by-hash: A reasoned analysis. In Proceedings of the USENIX Annual Technical Conference (ATC\u201906). 85\u201390."},{"key":"e_1_3_2_5_2","volume-title":"Proceedings of the 2nd USENIX Conference on File and Storage Technologies (FAST\u201903)\u2014Work in Progress Reports.","author":"Bonwick Jeff","year":"2003","unstructured":"Jeff Bonwick, Matt Ahrens, Val Henson, Mark Maybee, and Mark Shellenbaum. 2003. The Zettabyte file system. In Proceedings of the 2nd USENIX Conference on File and Storage Technologies (FAST\u201903)\u2014Work in Progress Reports."},{"key":"e_1_3_2_6_2","article-title":"Block I\/O layer tracing: blktrace","author":"Brunelle Alan D.","year":"2006","unstructured":"Alan D. Brunelle. 2006. Block I\/O layer tracing: blktrace. Presentation in Gelato-Cupertino, CA, USA.","journal-title":"Presentation in Gelato-Cupertino, CA, USA"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCD58817.2023.00050"},{"key":"e_1_3_2_8_2","volume-title":"Proceedings of the USENIX Annual Technical Conference (ATC\u201909)","author":"Clements Austin T.","year":"2009","unstructured":"Austin T. Clements, Irfan Ahmad, Murali Vilayannur, and Jinyuan Li. 2009. Decentralized deduplication in SAN cluster file systems. In Proceedings of the USENIX Annual Technical Conference (ATC\u201909)."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/DCC.2011.46"},{"key":"e_1_3_2_10_2","volume-title":"Proceedings of the USENIX Annual Technical Conference (ATC\u201910)","author":"Debnath Biplob K.","year":"2010","unstructured":"Biplob K. Debnath, Sudipta Sengupta, and Jin Li. 2010. ChunkStash: Speeding up inline storage deduplication using flash memory. In Proceedings of the USENIX Annual Technical Conference (ATC\u201910)."},{"key":"e_1_3_2_11_2","first-page":"285","volume-title":"Proceedings of the USENIX Annual Technical Conference (ATC\u201912)","author":"El-Shimi Ahmed","year":"2012","unstructured":"Ahmed El-Shimi, Ran Kalach, Ankit Kumar, Adi Ottean, Jin Li, and Sudipta Sengupta. 2012. Primary data deduplication\u2014Large scale study and system design. In Proceedings of the USENIX Annual Technical Conference (ATC\u201912). 285\u2013296."},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/1516046.1516059"},{"key":"e_1_3_2_13_2","volume-title":"Proceedings of the 9th Workshop on Hot Topics in Operating Systems (HotOS\u201903). 13\u201318.","author":"Henson Val","year":"2003","unstructured":"Val Henson. 2003. An analysis of compare-by-hash. In Proceedings of the 9th Workshop on Hot Topics in Operating Systems (HotOS\u201903). 13\u201318."},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3620665.3640353"},{"key":"e_1_3_2_15_2","volume-title":"Proceedings of the Flash Memory Summit","author":"Imershein Louis","year":"2018","unstructured":"Louis Imershein. 2018. Open source data reduction for high performance flash storage. In Proceedings of the Flash Memory Summit. https:\/\/www.flashmemorysummit.com\/English\/Collaterals\/Proceedings\/2018\/20180808_SOFT-202-1_Imershein.pdf"},{"key":"e_1_3_2_16_2","volume-title":"Open Cache Acceleration Software","year":"2022","unstructured":"Intel. 2022. Open Cache Acceleration Software. Retrieved January 20, 2022 from https:\/\/open-cas.github.io\/"},{"key":"e_1_3_2_17_2","volume-title":"ISA-L Performance Report\u2014Release 2.19","author":"Corp. Intel","year":"2017","unstructured":"Intel Corp.2017. ISA-L Performance Report\u2014Release 2.19. Technical Report. Intel Corp. https:\/\/01.org\/sites\/default\/files\/documentation\/intel_isa-l_2.19_performance_report_0.pdf"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/1534530.1534540"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/2367589.2367600"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/2757667.2757678"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/1837915.1837921"},{"key":"e_1_3_2_22_2","first-page":"501","volume-title":"Proceedings of the USENIX Annual Technical Conference (ATC\u201914)","author":"Li Cheng","year":"2014","unstructured":"Cheng Li, Philip Shilane, Fred Douglis, Hyong Shim, Stephen Smaldone, and Grant Wallace. 2014. Nitro: A capacity-optimized SSD cache for primary storage. In Proceedings of the USENIX Annual Technical Conference (ATC\u201914). 501\u2013512."},{"key":"e_1_3_2_23_2","volume-title":"Proceedings of the 1st Workshop on I\/O Virtualization (WIOV\u201908)","author":"Liguori Anthony","year":"2008","unstructured":"Anthony Liguori and Eric Van Hensbergen. 2008. Experiences with content addressable storage and virtual disks. In Proceedings of the 1st Workshop on I\/O Virtualization (WIOV\u201908)."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.5555\/2591272.2591292"},{"key":"e_1_3_2_25_2","first-page":"111","volume-title":"Proceedings of the 7th USENIX Conference on File and Storage Technologies (FAST\u201909)","author":"Lillibridge Mark","year":"2009","unstructured":"Mark Lillibridge, Kave Eshghi, Deepavali Bhagwat, Vinay Deolalikar, Greg Trezis, and Peter Camble. 2009. Sparse indexing: Large scale, inline deduplication using sampling and locality. In Proceedings of the 7th USENIX Conference on File and Storage Technologies (FAST\u201909). 111\u2013123."},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSST.2014.6855536"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/2367589.2367606"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/2512348"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2014.84"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/1534530.1534541"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSST.2010.5496992"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.5555\/2591272.2591291"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.5555\/1960475.1960476"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/502034.502052"},{"key":"e_1_3_2_35_2","volume-title":"Use Deduplication, Data Compression, and Data Compaction to Increase Storage Efficiency","author":"Inc. NetApp","year":"2022","unstructured":"NetApp Inc.2022. Use Deduplication, Data Compression, and Data Compaction to Increase Storage Efficiency. Technical Report. NetApp Inc. https:\/\/docs.netapp.com\/us-en\/ontap\/pdfs\/sidebar\/Use_deduplication__data_compression__and_data_compaction_to_increase_storage_efficiency.pdf"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25821-3_5"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2018.00106"},{"key":"e_1_3_2_38_2","volume-title":"Dell EMC Unity: Data Reduction","author":"Poulin Ryan","year":"2021","unstructured":"Ryan Poulin. 2021. Dell EMC Unity: Data Reduction. Technical Report. Dell. https:\/\/www.delltechnologies.com\/asset\/en-us\/products\/storage\/industry-market\/h16870-dell_emc_unity-data_reduction.pdf"},{"key":"e_1_3_2_39_2","volume-title":"How Flash Changes Everything","author":"Inc. Pure Storage","year":"2013","unstructured":"Pure Storage Inc.2013. How Flash Changes Everything. Technical Report. Pure Storage Inc. https:\/\/www.purestorage.com\/content\/dam\/purestorage\/pdf\/whitepapers\/Pure_Storage_Whitepaper_How_Flash_Changes_Everything.pdf"},{"key":"e_1_3_2_40_2","unstructured":"Pure Storage Inc.2015. Modeling Workload IO Size Mixes with Oracle\u2019s vdbench Tool. Retrieved July 26 2024 from https:\/\/blog.purestorage.com\/purely-technical\/modeling-io-size-mixes-with-vdbench\/"},{"key":"e_1_3_2_41_2","unstructured":"Pure Storage Inc.2018. Data Reduction Efficiency: All Storage Is NOT Created Equal! Retrieved July 26 2024 from https:\/\/blog.purestorage.com\/products\/"},{"key":"e_1_3_2_42_2","first-page":"89","volume-title":"Proceedings of the Conference on File and Storage Technologies (FAST\u201902)","author":"Quinlan Sean","year":"2002","unstructured":"Sean Quinlan and Sean Dorward. 2002. Venti: A new approach to archival storage. In Proceedings of the Conference on File and Storage Technologies (FAST\u201902). 89\u2013101."},{"key":"e_1_3_2_43_2","unstructured":"RedHat. 2022. Deduplicating and Compressing Logical Volumes on RHEL Red Hat Enterprise Linux 9 Using VDO to Increase LVM Storage Capacity. Retrieved September 9 2023 from https:\/\/access.redhat.com\/documentation\/en-us\/red_hat_enterprise_linux\/9"},{"key":"e_1_3_2_44_2","volume-title":"The Digitization of the World from Edge to Core","author":"Reinsel David","year":"2018","unstructured":"David Reinsel, John Gantz, and John Rydning. 2018. The Digitization of the World from Edge to Core. Technical Report US44413318. International Data Corporation (IDC)."},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/2501620.2501623"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2018.2796100"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3470496.3527442"},{"key":"e_1_3_2_48_2","volume-title":"Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST\u201912)","author":"Srinivasan Kiran","year":"2012","unstructured":"Kiran Srinivasan, Timothy Bisson, Garth R. Goodson, and Kaladhar Voruganti. 2012. iDedup: Latency-aware, inline data deduplication for primary storage. In Proceedings of the 10th USENIX Conference on File and Storage Technologies (FAST\u201912)."},{"key":"e_1_3_2_49_2","article-title":"Bcache: Caching beyond Just RAM","author":"Stearns William","year":"2010","unstructured":"William Stearns and Kent Overstreet. 2010. Bcache: Caching beyond Just RAM. LWN.net.","journal-title":"LWN.net"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSST.2016.7897080"},{"key":"e_1_3_2_51_2","first-page":"83","volume-title":"Proceedings of the Ottawa Linux Symposium (OLS\u201914)","author":"Tarasov Vasily","year":"2014","unstructured":"Vasily Tarasov, Deepak Jain, Geoff Kuenning, Sonam Mandal, Karthikeyani Palanisami, Philip Shilane, Sagar Trehan, and Erez Zadok. 2014. Dmdedup: Device mapper target for data deduplication. In Proceedings of the Ottawa Linux Symposium (OLS\u201914). 83\u201395."},{"key":"e_1_3_2_52_2","unstructured":"Henk Vandenbergh. 2012. Vdbench Users Guide\u2014Version: 5.03. Retrieved July 26 2024 from https:\/\/www.oracle.com\/technetwork\/server-storage\/vdbench-1901683.pdf"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/NAS.2019.8834729"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3465410"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2013.6544846"},{"key":"e_1_3_2_56_2","volume-title":"Proceedings of the 33rd International Conference on Massive Storage Systems and Technology (MSST\u201917)","author":"Wu Huijun","year":"2017","unstructured":"Huijun Wu, Chen Wang, Yinjin Fu, Sherif Sakr, Liming Zhu, and Kai Lu. 2017. HPDedup: A hybrid prioritized data deduplication mechanism for primary storage in the cloud. In Proceedings of the 33rd International Conference on Massive Storage Systems and Technology (MSST\u201917)."},{"key":"e_1_3_2_57_2","volume-title":"Proceedings of the 19th USENIX Conference on File and Storage Technologies (FAST\u201921)","author":"Wu Kan","year":"2021","unstructured":"Kan Wu, Zhihan Guo, Guanzhou Hu, Kaiwei Tu, Ramnatthan Alagappan, Rathijit Sen, Kwanghyun Park, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2021. The storage hierarchy is not a hierarchy: Optimizing caching on modern storage devices with Orthus. In Proceedings of the 19th USENIX Conference on File and Storage Technologies (FAST\u201921). 307\u2013323. https:\/\/www.usenix.org\/conference\/fast21\/presentation\/wu-kan"},{"key":"e_1_3_2_58_2","volume-title":"Proceedings of the USENIX Annual Technical Conference (ATC\u201911)","author":"Xia Wen","year":"2011","unstructured":"Wen Xia, Hong Jiang, Dan Feng, and Yu Hua. 2011. SiLo: A similarity-locality based near-exact deduplication scheme with low RAM overhead and high throughput. In Proceedings of the USENIX Annual Technical Conference (ATC\u201911)."},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2013.6704674"},{"key":"e_1_3_2_60_2","first-page":"269","volume-title":"Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST\u201908)","author":"Zhu Benjamin","year":"2008","unstructured":"Benjamin Zhu, Kai Li, and R. Hugo Patterson. 2008. Avoiding the disk bottleneck in the data domain deduplication file system. In Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST\u201908). 269\u2013282."},{"key":"e_1_3_2_61_2","first-page":"171","volume-title":"Proceedings of the 19th USENIX Conference on File and Storage Technologies (FAST\u201921)","author":"Zou Xiangyu","year":"2021","unstructured":"Xiangyu Zou, Jingsong Yuan, Philip Shilane, Wen Xia, Haijun Zhang, and Xuan Wang. 2021. The dilemma between deduplication and locality: Can both be achieved? In Proceedings of the 19th USENIX Conference on File and Storage Technologies (FAST\u201921). 171\u2013185."}],"container-title":["ACM Transactions on Storage"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3678250","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3678250","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:54:09Z","timestamp":1750287249000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3678250"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,6]]},"references-count":60,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,11,30]]}},"alternative-id":["10.1145\/3678250"],"URL":"https:\/\/doi.org\/10.1145\/3678250","relation":{},"ISSN":["1553-3077","1553-3093"],"issn-type":[{"value":"1553-3077","type":"print"},{"value":"1553-3093","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,6]]},"assertion":[{"value":"2023-11-17","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-10","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-06","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}