{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,4]],"date-time":"2026-07-04T17:09:19Z","timestamp":1783184959034,"version":"3.54.6"},"reference-count":100,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2021,10,15]],"date-time":"2021-10-15T00:00:00Z","timestamp":1634256000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Storage"],"published-print":{"date-parts":[[2021,11,30]]},"abstract":"<jats:p>This article is an eight-year retrospective on development priorities for RocksDB, a key-value store developed at Facebook that targets large-scale distributed systems and that is optimized for Solid State Drives (SSDs). We describe how the priorities evolved over time as a result of hardware trends and extensive experiences running RocksDB at scale in production at a number of organizations: from optimizing write amplification, to space amplification, to CPU utilization. We describe lessons from running large-scale applications, including that resource allocation needs to be managed across different RocksDB instances, that data formats need to remain backward- and forward-compatible to allow incremental software rollouts, and that appropriate support for database replication and backups are needed. Lessons from failure handling taught us that data corruption errors needed to be detected earlier and that data integrity protection mechanisms are needed at every layer of the system. We describe improvements to the key-value interface. We describe a number of efforts that in retrospect proved to be misguided. Finally, we describe a number of open problems that could benefit from future research.<\/jats:p>","DOI":"10.1145\/3483840","type":"journal-article","created":{"date-parts":[[2021,10,17]],"date-time":"2021-10-17T01:27:42Z","timestamp":1634434062000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":181,"title":["RocksDB: Evolution of Development Priorities in a Key-value Store Serving Large-scale Applications"],"prefix":"10.1145","volume":"17","author":[{"given":"Siying","family":"Dong","sequence":"first","affiliation":[{"name":"Facebook Inc., 1 Hacker Way, Menlo Park, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Kryczka","sequence":"additional","affiliation":[{"name":"Facebook Inc., 1 Hacker Way, Menlo Park, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanqin","family":"Jin","sequence":"additional","affiliation":[{"name":"Facebook Inc., 1 Hacker Way, Menlo Park, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Stumm","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Toronto, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,10,15]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/1376616.1376712"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3341301.3359656"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2015.2435779"},{"key":"e_1_3_3_5_2","first-page":"461","volume-title":"Proceedings of the International Conference on Extending Database Technology","author":"Athanassoulis Manos","year":"2016","unstructured":"Manos Athanassoulis, Michael S. Kester, Lukas M. Maas, Radu Stoica, Stratos Idreos, Anastasia Ailamaki, and Mark Callaghan. 2016. Designing access methods: The RUM conjecture. In Proceedings of the International Conference on Extending Database Technology (EDBT'16). 461\u2013466."},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.5555\/3154690.3154725"},{"key":"e_1_3_3_7_2","volume-title":"Proceedings of the Usenix Linux Storage and Filesystems Conference","author":"Bj\u00f8rling Matias","year":"2020","unstructured":"Matias Bj\u00f8rling. 2020. Zone Append: A new way of writing to zoned storage. In Proceedings of the Usenix Linux Storage and Filesystems Conference (VAULT'20)."},{"key":"e_1_3_3_8_2","unstructured":"Dhruba Borthakur. 2008. HDFS architecture guide. Hadoop Apache Project . Retrieved from http:\/\/hadoop.apache.org\/docs\/stable1\/hdfs_design.pdf."},{"key":"e_1_3_3_9_2","unstructured":"Mark Callaghan. 2016. MongoRocks and WiredTiger versus LinkBench on a small server . Retrieved from http:\/\/smalldatum.blogspot.com\/2016\/10\/mongorocks-and-wiredtiger-versus.html."},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.5555\/3386691.3386712"},{"issue":"4","key":"e_1_3_3_11_2","article-title":"Apache Flink: Stream and batch processing in a single engine","volume":"36","author":"Carbone Paris","year":"2015","unstructured":"Paris Carbone, Asterios Katsifodimos, Stephan Ewen, Volker Markl, Seif Haridi, and Kostas Tzoumas. 2015. Apache Flink: Stream and batch processing in a single engine. Bull. IEEE Comput. Societ. Technic. Commit. Data Eng. 36, 4 (2015).","journal-title":"Bull. IEEE Comput. Societ. Technic. Commit. Data Eng."},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/1365815.1365816"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.sse.2016.07.006"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2904441"},{"key":"e_1_3_3_15_2","unstructured":"Wikipedia contributors. 2020. Data Integrity Field . Retrieved from https:\/\/en.wikipedia.org\/wiki\/Data_Integrity_Field."},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/2491245"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3064054"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2463710"},{"key":"e_1_3_3_19_2","article-title":"Silent data corruptions at scale","author":"Dixit Harish Dattatraya","year":"2021","unstructured":"Harish Dattatraya Dixit, Sneha Pendharkar, Matt Beadon, Chris Mason, Tejasvi Chakravarthy, Bharath Muthiah, and Sriram Sankar. 2021. Silent data corruptions at scale. arXiv preprint arXiv:2102.11245 (2021).","journal-title":"arXiv preprint arXiv:2102.11245"},{"key":"e_1_3_3_20_2","volume-title":"Proceedings of the Conference on Innovative Data Systems Research","author":"Dong Siying","year":"2017","unstructured":"Siying Dong, Mark Callaghan, Leonidas Galanis, Dhruba Borthakur, Tony Savor, and Michael Stumm. 2017. Optimizing space amplification in RocksDB. In Proceedings of the Conference on Innovative Data Systems Research (CIDR'17)."},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3483840"},{"key":"e_1_3_3_22_2","unstructured":"Jason Evans. 2019. Jemalloc Memory Allocator . Retrieved from http:\/\/jemalloc.net."},{"key":"e_1_3_3_23_2","unstructured":"Facebook. 2021. Column families. RocksDB documentation wiki . Retrieved from https:\/\/github.com\/facebook\/rocksdb\/wiki\/Column-Families."},{"key":"e_1_3_3_24_2","unstructured":"Facebook. 2021. Compaction. RocksDB documentation wiki . Retrieved from https:\/\/github.com\/facebook\/rocksdb\/wiki\/Compaction."},{"key":"e_1_3_3_25_2","unstructured":"Facebook. 2021. Compaction filter. RocksDB documentation wiki . Retrieved from https:\/\/github.com\/facebook\/rocksdb\/wiki\/Compaction-Filter."},{"key":"e_1_3_3_26_2","unstructured":"Facebook. 2021. Merge operator. RocksDB documentation wiki . Retrieved from https:\/\/github.com\/facebook\/rocksdb\/wiki\/Merge-Operator."},{"key":"e_1_3_3_27_2","unstructured":"Facebook. 2020. Universal compaction. RocksDB documentation wiki . Retrieved from https:\/\/github.com\/facebook\/rocksdb\/wiki\/Universal-Compaction."},{"key":"e_1_3_3_28_2","unstructured":"Facebook. 2019. User Timestamps (Experimental). RocksDB documentation wiki . Retrieved from https:\/\/github.com\/facebook\/rocksdb\/wiki\/User-Timestamp-%28Experimental%29."},{"key":"e_1_3_3_29_2","unstructured":"Facebook. 2020. SST File Manger. RocksDB documentation wiki . Retrieved from https:\/\/github.com\/facebook\/rocksdb\/wiki\/SST-File-Manager."},{"key":"e_1_3_3_30_2","unstructured":"Facebook. 2020. Write buffer manager. RocksDB documentation wiki . Retrieved from https:\/\/github.com\/facebook\/rocksdb\/wiki\/Write-Buffer-Manager."},{"key":"e_1_3_3_31_2","unstructured":"Facebook. 2021. MemTable. RocksDB documentation wiki . Retrieved from https:\/\/github.com\/facebook\/rocksdb\/wiki\/MemTable#comparison."},{"key":"e_1_3_3_32_2","unstructured":"Facebook. 2021. Rate limiter. RocksDB documentation wiki . Retrieved from https:\/\/github.com\/facebook\/rocksdb\/wiki\/Rate-Limiter."},{"key":"e_1_3_3_33_2","unstructured":"Facebook. 2021. Write Ahead Log. RocksDB documentation wiki . Retrieved from https:\/\/github.com\/facebook\/rocksdb\/wiki\/Write-Ahead-Log."},{"key":"e_1_3_3_34_2","volume-title":"Proceedings of the 18th International Workshop on High Performance Transaction Systems","author":"Faleiro Jose","year":"2019","unstructured":"Jose Faleiro. 2019. The dangers of logical replication and a practical solution. In Proceedings of the 18th International Workshop on High Performance Transaction Systems (HPTS'19)."},{"key":"e_1_3_3_35_2","unstructured":"Apache Software Foundation. 2021. Apache Cassandra . Retrieved from https:\/\/cassandra.apache.org\/."},{"key":"e_1_3_3_36_2","unstructured":"Apache Software Foundation. 2021. Apache HBase . Retrieved from https:\/\/hbase.apache.org\/."},{"key":"e_1_3_3_37_2","unstructured":"Apache Software Foundation. 2016. Compaction. Apache Cassandra Documentation . Retrieved from https:\/\/murukeshm.github.io\/cassandra\/trunk\/operating\/compaction.html?highlight=tiered%20compaction#size-tiered-compaction-strategy."},{"key":"e_1_3_3_38_2","unstructured":"Apache Software Foundation. 2015. HBase: Add Compaction Policy that Explores More Storefile Groups . Retrieved from https:\/\/issues.apache.org\/jira\/browse\/HBASE-7842."},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/2184512.2184527"},{"key":"e_1_3_3_40_2","unstructured":"S. Ghemawat and J. Dean. 2011. LevelDB . Retrieved from https:\/\/github.com\/google\/leveldb."},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/945445.945450"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/2741948.2741973"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3337821.3337836"},{"key":"e_1_3_3_44_2","unstructured":"Google. 2021. Protobuf . Retrieved from https:\/\/opensource.google\/projects\/protobuf."},{"key":"e_1_3_3_45_2","unstructured":"Dikang Gu. 2018. Open-sourcing a 10x reduction in Apache Cassandra tail latency. Instagram Engineering Blog . Retrieved from https:\/\/instagram-engineering.com\/open-sourcing-a-10x-reduction-in-apache-cassandra-tail-latency-d64f86b43589."},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3458336.3465297"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415535"},{"key":"e_1_3_3_48_2","unstructured":"Intel. 2021. Memory optimized for data-centric workloads . Retrieved from https:\/\/www.intel.ca\/content\/www\/ca\/en\/architecture-and-technology\/optane-dc-persistent-memory.html."},{"key":"e_1_3_3_49_2","unstructured":"Intel. 2021. TRIM Overview . Retrieved from https:\/\/www.intel.com\/content\/www\/us\/en\/support\/articles\/000016148\/memory-and-storage.html."},{"key":"e_1_3_3_50_2","unstructured":"Iron. 2021. Simple flexible reliable serverless tools . Retrieved from https:\/\/www.iron.io."},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2746480"},{"key":"e_1_3_3_52_2","unstructured":"Jay Kreps. 2016. Introducing Kafka Streams: Stream processing made simple. Confluent . Retrieved from https:\/\/www.confluent.io\/blog\/introducing-kafka-streams-stream-processing-made-simple\/."},{"key":"e_1_3_3_54_2","unstructured":"Chuck Lever. 2012. End-to-end data integrity requirements For NFS. Oracle Corp. slide deck . Retrieved from https:\/\/datatracker.ietf.org\/meeting\/83\/materials\/slides-83-nfsv4-2."},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/2043556.2043558"},{"key":"e_1_3_3_56_2","unstructured":"Mike Lin. 2019. Faster BAM Sorting with Samtools and RocksDB. Dnanexus Developer Blog . Retrieved from https:\/\/devblog.dnanexus.com\/faster-bam-sorting-with-samtools-and-rocksdb\/."},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3033273"},{"key":"e_1_3_3_58_2","unstructured":"Scott Mansfield Vu Tuan Nguyen Sridhar Enugula and Shashi Madappa. 2016. Application data caching using SSDs: The Moneta project: Next generation EVCache for better cost optimization. Netflix Technology Blog . Retrieved from https:\/\/netflixtechblog.com\/application-data-caching-using-ssds-5bf25df851ef."},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-020-00638-1"},{"key":"e_1_3_3_60_2","unstructured":"Mark Marchukov. 2017. LogDevice: A distributed data store for logs. Facebook Engineering Blog . Retrieved from https:\/\/engineering.fb.com\/core-data\/logdevice-a-distributed-data-store-for-logs\/."},{"key":"e_1_3_3_61_2","unstructured":"Yoshinori Matsunobu. 2017. Migrating a database from InnoDB to MyRock. Facebook Engineering Blog . Retrieved from https:\/\/engineering.fb.com\/core-data\/migrating-a-database-from-innodb-to-myrocks\/."},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415546"},{"key":"e_1_3_3_63_2","unstructured":"Microsoft. 2021. Microsoft SQL Server . Retrieved from https:\/\/www.microsoft.com\/en-us\/sql-server\/."},{"key":"e_1_3_3_64_2","unstructured":"Domas Mituzas. 2013. Flashcache at Facebook: From 2010 to 2013 and beyond . Retrieved from https:\/\/engineering.fb.com\/2013\/10\/09\/core-data\/flashcache-at-facebook-from-2010-to-2013-and-beyond\/."},{"key":"e_1_3_3_65_2","unstructured":"MongoDB. 2021. WiredTiger Storage Engine . Retrieved from https:\/\/docs.mongodb.com\/manual\/core\/wiredtiger\/."},{"key":"e_1_3_3_66_2","unstructured":"MongoDB. 2021. The Database for Modern Applications . Retrieved from https:\/\/www.mongodb.com."},{"key":"e_1_3_3_67_2","unstructured":"MySQL. 2021. Introduction to InnodeDB . Retrieved from https:\/\/dev.mysql.com\/doc\/refman\/5.6\/en\/innodb-introduction.html."},{"key":"e_1_3_3_68_2","unstructured":"MySQL. 2021. MySQL . Retrieved from https:\/\/www.mysql.com\/."},{"key":"e_1_3_3_69_2","unstructured":"Xu Ning and Maxim Fateev. 2016. Cherami: Uber Engineering's durable and scalable task queue in Go. Uber Engineering Blog . Retrieved from https:\/\/eng.uber.com\/cherami-message-queue-system\/."},{"key":"e_1_3_3_70_2","doi-asserted-by":"publisher","DOI":"10.14778\/3137765.3137770"},{"key":"e_1_3_3_71_2","unstructured":"University of Wisconsin Computer Sciences Department. 2012. SHORE Storage Manager: The multi-threaded version . Retrieved from https:\/\/research.cs.wisc.edu\/shore-mt."},{"key":"e_1_3_3_72_2","doi-asserted-by":"publisher","DOI":"10.5555\/1268708.1268751"},{"key":"e_1_3_3_73_2","doi-asserted-by":"publisher","DOI":"10.1007\/s002360050048"},{"key":"e_1_3_3_74_2","unstructured":"Oracle. 2021. Oracle Database . Retrieved from https:\/\/www.oracle.com\/database\/."},{"key":"e_1_3_3_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/3093241.3093278"},{"key":"e_1_3_3_76_2","doi-asserted-by":"publisher","DOI":"10.5555\/1137835"},{"key":"e_1_3_3_77_2","unstructured":"Percona. 2021. MongoRocks. Percona server for MongoDB documentation . Retrieved from https:\/\/www.percona.com\/doc\/percona-server-for-mongodb\/3.4\/mongorocks.html."},{"key":"e_1_3_3_78_2","volume-title":"Proceedings of the Linux Symposium","volume":"4","author":"Petersen Martin K.","year":"2008","unstructured":"Martin K. Petersen. 2008. Linux data integrity extensions. In Proceedings of the Linux Symposium, Vol. 4."},{"key":"e_1_3_3_79_2","unstructured":"Martin K. Petersen and Sergio Leunissen. 2010. Eliminating silent data corruption with Oracle Linux. Oracle Corp. Slide Deck . Retrieved from https:\/\/oss.oracle.com\/mkp\/docs\/data-integrity-webcast.pdf."},{"key":"e_1_3_3_80_2","volume-title":"Proceedings of the Conference on Innovative Data Systems Research","author":"Picoli Ivan Luiz","year":"2020","unstructured":"Ivan Luiz Picoli, Niclas Hedam, Philippe Bonnet, and Pinar T\u00f6z\u00fcn. 2020. Open-channel SSD (What is it good for). In Proceedings of the Conference on Innovative Data Systems Research (CIDR'20)."},{"key":"e_1_3_3_81_2","unstructured":"Qihoo. 2021. Pika . Retrieved from https:\/\/github.com\/Qihoo360\/pika."},{"key":"e_1_3_3_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132747.3132765"},{"key":"e_1_3_3_83_2","doi-asserted-by":"publisher","DOI":"10.14778\/3151106.3151108"},{"key":"e_1_3_3_84_2","doi-asserted-by":"publisher","DOI":"10.1145\/357401.357402"},{"key":"e_1_3_3_85_2","doi-asserted-by":"publisher","DOI":"10.1145\/2889160.2889223"},{"key":"e_1_3_3_86_2","doi-asserted-by":"publisher","DOI":"10.1145\/2213836.2213862"},{"key":"e_1_3_3_87_2","unstructured":"Arun Sharma. 2016. Dragon: A distributed graph query engine. Facebook Engineering Blog . Retrieved from https:\/\/engineering.fb.com\/2016\/03\/18\/data-infrastructure\/dragon-a-distributed-graph-query-engine\/."},{"key":"e_1_3_3_88_2","unstructured":"Arun Sharma. 2019. How we use RocksDB at Rockset . Retrieved from https:\/\/rockset.com\/blog\/how-we-use-rocksdb-at-rockset\/."},{"key":"e_1_3_3_89_2","doi-asserted-by":"publisher","DOI":"10.5555\/2591272.2591275"},{"key":"e_1_3_3_91_2","unstructured":"Mark Slee Aditya Agarwal and Marc Kwiatkowski. 2007. Thrift: Scalable cross-language services implementation. Facebook White Paper . Retrieved from https:\/\/thrift.apache.org\/static\/files\/thrift-20070401.pdf."},{"key":"e_1_3_3_92_2","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3386134"},{"key":"e_1_3_3_93_2","doi-asserted-by":"publisher","DOI":"10.5555\/3358807.3358891"},{"key":"e_1_3_3_94_2","unstructured":"Levi Tamasi. 2021. Integrated BlobDB. Facebook Engineering Blog . Retrieved from https:\/\/rocksdb.org\/blog\/2021\/05\/26\/integrated-blob-db.html."},{"key":"e_1_3_3_95_2","unstructured":"Facebook RocksDB team. 2021. A Persistent Key-value Store for Fast Storage Environments . Retrieved from https:\/\/rocksdb.org."},{"key":"e_1_3_3_96_2","first-page":"457","volume-title":"Proceedings of the 21st International Conference on Extending Database Technology","author":"Vin\u00e7on Tobias","year":"2018","unstructured":"Tobias Vin\u00e7on, Sergej Hardock, Christian Riegger, Julian Oppermann, Andreas Koch, and Ilia Petrov. 2018. NoFTL-KV: Tackling write-amplification on KV-Stores with native storage management. In Proceedings of the 21st International Conference on Extending Database Technology (EDBT'18). 457\u2013460."},{"key":"e_1_3_3_97_2","doi-asserted-by":"publisher","DOI":"10.1145\/2592798.2592804"},{"key":"e_1_3_3_98_2","unstructured":"Qi Wang. 2018. Jemalloc tuning . Retrieved from https:\/\/github.com\/jemalloc\/jemalloc\/blob\/dev\/TUNING.md."},{"key":"e_1_3_3_99_2","unstructured":"Tao Xu. 2015. RocksDB for personalized search at Airbnb. Video talk . Retrieved from https:\/\/www.codechannels.com\/rocksdb-meetup-tao-xu-airbnb-rocksdb-for-personalized-search-at-airbnb\/."},{"key":"e_1_3_3_100_2","volume-title":"Proceedings of the Non-volatile Memories Workshop","author":"Yang Fei","year":"2015","unstructured":"Fei Yang, K. Dou, S. Chen, J. U. Kang, and S. Cho. 2015. Multi-streaming RocksDB. In Proceedings of the Non-volatile Memories Workshop."},{"key":"e_1_3_3_101_2","doi-asserted-by":"publisher","DOI":"10.1145\/3126545"},{"key":"e_1_3_3_102_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSST.2013.6558423"},{"key":"e_1_3_3_103_2","doi-asserted-by":"publisher","DOI":"10.5555\/1855511.1855514"}],"container-title":["ACM Transactions on Storage"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3483840","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3483840","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:19:03Z","timestamp":1750191543000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3483840"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,15]]},"references-count":100,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,11,30]]}},"alternative-id":["10.1145\/3483840"],"URL":"https:\/\/doi.org\/10.1145\/3483840","relation":{},"ISSN":["1553-3077","1553-3093"],"issn-type":[{"value":"1553-3077","type":"print"},{"value":"1553-3093","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,15]]},"assertion":[{"value":"2021-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-10-15","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}