{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T09:50:36Z","timestamp":1773481836897,"version":"3.50.1"},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"9","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2023,5]]},"abstract":"<jats:p>Database systems are paying more attention to data security in recent years. Immutable systems such as blockchains, verifiable databases, and ledger databases are equipped with various verifiability mechanisms to protect data. Such systems often adopt different threat models, and techniques, therefore, have different performance implications compared to traditional database systems. So far, there is no uniform benchmarking tool for evaluating the performance of these systems, especially at the level of verification functions. In this paper, we first survey the design space of the<jats:italic>verifiability-enabled database systems<\/jats:italic>along five dimensions: threat model, authenticated data structure (ADS), query processing, verification, and auditing. Based on this survey, we design and implement VeriBench, a benchmark framework for<jats:italic>verifiability-enabled database systems.<\/jats:italic>VeriBench enables a fair comparison of systems designed with different underlying technologies that share the client-side verification scheme, and focuses on design space exploration to provide a deeper understanding of different system design choices. VeriBench incorporates micro- and macro-benchmarks to provide a comprehensive evaluation. Further, VeriBench is designed to enable easy extension for benchmarking new systems and workloads. We run VeriBench to conduct a comprehensive analysis of state-of-the-art systems comprising blockchains, ledger databases, and log transparency technologies. The results expose the weaknesses and strengths of each underlying design choice, and the insights should serve as guidance for future development.<\/jats:p>","DOI":"10.14778\/3598581.3598588","type":"journal-article","created":{"date-parts":[[2023,7,10]],"date-time":"2023-07-10T22:19:06Z","timestamp":1689027546000},"page":"2145-2157","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["VeriBench: Analyzing the Performance of Database Systems with Verifiability"],"prefix":"10.14778","volume":"16","author":[{"given":"Cong","family":"Yue","sequence":"first","affiliation":[{"name":"National University of Singapore"}]},{"given":"Meihui","family":"Zhang","sequence":"additional","affiliation":[{"name":"Beijing Institute of Technology"}]},{"given":"Changhao","family":"Zhu","sequence":"additional","affiliation":[{"name":"Beijing Institute of Technology"}]},{"given":"Gang","family":"Chen","sequence":"additional","affiliation":[{"name":"Zhejiang University"}]},{"given":"Dumitrel","family":"Loghin","sequence":"additional","affiliation":[{"name":"National University of Singapore"}]},{"given":"Beng Chin","family":"Ooi","sequence":"additional","affiliation":[{"name":"National University of Singapore"}]}],"member":"320","published-online":{"date-parts":[[2023,7,10]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"AlibabaCloud. 2022. LedgerDB. https:\/\/www.alibabacloud.com\/product\/ledgerdb AlibabaCloud. 2022. LedgerDB. https:\/\/www.alibabacloud.com\/product\/ledgerdb"},{"key":"e_1_2_1_2_1","volume-title":"Veritas: Shared Verifiable Databases and Tables in the Cloud. In CIDR. 1--9.","author":"Lindsey Allen","year":"2019","unstructured":"Lindsey Allen et al. 2019 . Veritas: Shared Verifiable Databases and Tables in the Cloud. In CIDR. 1--9. Lindsey Allen et al. 2019. Veritas: Shared Verifiable Databases and Tables in the Cloud. In CIDR. 1--9."},{"key":"e_1_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Mohammad Alomari Michael Cahill AlanFekete and Uwe Rohm. 2008. The Cost of Serializability on Platforms That Use Snapshot Isolation. In ICDE. 576--585. Mohammad Alomari Michael Cahill AlanFekete and Uwe Rohm. 2008. The Cost of Serializability on Platforms That Use Snapshot Isolation. In ICDE. 576--585.","DOI":"10.1109\/ICDE.2008.4497466"},{"key":"e_1_2_1_4_1","unstructured":"Amazon. 2019. Amazon Quantum Ledger Database. https:\/\/aws.amazon.com\/qldb\/ Amazon. 2019. Amazon Quantum Ledger Database. https:\/\/aws.amazon.com\/qldb\/"},{"key":"e_1_2_1_5_1","volume-title":"David Enyeart, Christopher Ferris, Gennady Laventman, Yacov Manevich, et al.","author":"Androulaki Elli","year":"2018","unstructured":"Elli Androulaki , Artem Barger , Vita Bortnikov , Christian Cachin , Konstantinos Christidis , Angelo De Caro , David Enyeart, Christopher Ferris, Gennady Laventman, Yacov Manevich, et al. 2018 . Hyperledger fabric: a distributed operating system for permissioned blockchains. In EuroSys . 1--15. Elli Androulaki, Artem Barger, Vita Bortnikov, Christian Cachin, Konstantinos Christidis, Angelo De Caro, David Enyeart, Christopher Ferris, Gennady Laventman, Yacov Manevich, et al. 2018. Hyperledger fabric: a distributed operating system for permissioned blockchains. In EuroSys. 1--15."},{"key":"e_1_2_1_6_1","volume-title":"Reilly Wong, Jason Anderson, and Jakub Szymaszek.","author":"Antonopoulos Panagiotis","year":"2021","unstructured":"Panagiotis Antonopoulos , Raghav Kaushik , Hanuma Kodavalla , Sergio Rosales Aceves , Reilly Wong, Jason Anderson, and Jakub Szymaszek. 2021 . SQL Ledger: Cryptographically Verifiable Data in Azure SQL Database. In SIGMOD. 2437--2449. Panagiotis Antonopoulos, Raghav Kaushik, Hanuma Kodavalla, Sergio Rosales Aceves, Reilly Wong, Jason Anderson, and Jakub Szymaszek. 2021. SQL Ledger: Cryptographically Verifiable Data in Azure SQL Database. In SIGMOD. 2437--2449."},{"key":"e_1_2_1_7_1","volume-title":"Concerto: A High Concurrency Key-Value Store with Integrity. In SIGMOD. 251--266.","author":"Arasu Arvind","year":"2017","unstructured":"Arvind Arasu , Ken Eguro , Raghav Kaushik , Donald Kossmann , Pingfan Meng , Vineet Pandey , and Ravi Ramamurthy . 2017 . Concerto: A High Concurrency Key-Value Store with Integrity. In SIGMOD. 251--266. Arvind Arasu, Ken Eguro, Raghav Kaushik, Donald Kossmann, Pingfan Meng, Vineet Pandey, and Ravi Ramamurthy. 2017. Concerto: A High Concurrency Key-Value Store with Integrity. In SIGMOD. 251--266."},{"key":"e_1_2_1_8_1","doi-asserted-by":"crossref","unstructured":"Siavosh Benabbas Rosario Gennaro and Yevgeniy Vahlis. 2011. Verifiable Delegation of Computation over Large Datasets. In CRYPTO. 111--131. Siavosh Benabbas Rosario Gennaro and Yevgeniy Vahlis. 2011. Verifiable Delegation of Computation over Large Datasets. In CRYPTO. 111--131.","DOI":"10.1007\/978-3-642-22792-9_7"},{"key":"e_1_2_1_9_1","unstructured":"Miguel Castro and Barbara Liskov. 1999. Practical Byzantine Fault Tolerance. In OSDI. Miguel Castro and Barbara Liskov. 1999. Practical Byzantine Fault Tolerance. In OSDI."},{"key":"e_1_2_1_10_1","unstructured":"ConsenSys. 2020. ConsenSys\/quorum: A permissioned implementation of Ethereum supporting data privacy. https:\/\/github.com\/ConsenSys\/quorum ConsenSys. 2020. ConsenSys\/quorum: A permissioned implementation of Ethereum supporting data privacy. https:\/\/github.com\/ConsenSys\/quorum"},{"key":"e_1_2_1_11_1","doi-asserted-by":"crossref","unstructured":"Brian F. Cooper Adam Silberstein Erwin Tam Raghu Ramakrishnan and Russell Sears. 2010. Benchmarking Cloud Serving Systems with YCSB. In SoCC. 143--154. Brian F. Cooper Adam Silberstein Erwin Tam Raghu Ramakrishnan and Russell Sears. 2010. Benchmarking Cloud Serving Systems with YCSB. In SoCC. 143--154.","DOI":"10.1145\/1807128.1807152"},{"key":"e_1_2_1_12_1","doi-asserted-by":"crossref","unstructured":"Akon Dey Alan D. Fekete Raghunath Nambiar and Uwe R\u00f6hm. 2014. YCSB+T: Benchmarking web-scale transactional databases. In ICDE. 223--230. Akon Dey Alan D. Fekete Raghunath Nambiar and Uwe R\u00f6hm. 2014. YCSB+T: Benchmarking web-scale transactional databases. In ICDE. 223--230.","DOI":"10.1109\/ICDEW.2014.6818330"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732240.2732246"},{"key":"e_1_2_1_14_1","volume-title":"Beng Chin Ooi, and Kian-Lee Tan","author":"Anh Dinh Tien Tuan","year":"2017","unstructured":"Tien Tuan Anh Dinh , Ji Wang , Gang Chen , Rui Liu , Beng Chin Ooi, and Kian-Lee Tan . 2017 . BLOCKBENCH : A Framework for Analyzing Private Blockchains. In SIGMOD. 1085--1100. Tien Tuan Anh Dinh, Ji Wang, Gang Chen, Rui Liu, Beng Chin Ooi, and Kian-Lee Tan. 2017. BLOCKBENCH: A Framework for Analyzing Private Blockchains. In SIGMOD. 1085--1100."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.14778\/3342263.3342636"},{"key":"e_1_2_1_16_1","unstructured":"Ahmad Ghazal Tilmann Rabl Minqing Hu Francois Raab Meikel Poess Alain Crolotte and Hans-Arno Jacobsen. 2013. BigBench: towards an industry standard benchmark for big data analytics. In SIGMOD. 1197--1208. Ahmad Ghazal Tilmann Rabl Minqing Hu Francois Raab Meikel Poess Alain Crolotte and Hans-Arno Jacobsen. 2013. BigBench: towards an industry standard benchmark for big data analytics. In SIGMOD. 1197--1208."},{"key":"e_1_2_1_17_1","unstructured":"Google. 2020. Certificate Transparency. https:\/\/www.certificate-transparency.org\/. Google. 2020. Certificate Transparency. https:\/\/www.certificate-transparency.org\/."},{"key":"e_1_2_1_18_1","volume-title":"Seung Jin Yang, and Reluca Ada Popa","author":"Hu Yuncong","year":"2021","unstructured":"Yuncong Hu , Kian Hooshmand , Harika Kalidhindi , Seung Jin Yang, and Reluca Ada Popa . 2021 . Mekle2: a low-latency transparency log system. In SP. 285--303. Yuncong Hu, Kian Hooshmand, Harika Kalidhindi, Seung Jin Yang, and Reluca Ada Popa. 2021. Mekle2: a low-latency transparency log system. In SP. 285--303."},{"key":"e_1_2_1_19_1","unstructured":"Hyperledger. 2022. Hyperledger Caliper. https:\/\/www.hyperledger.org\/use\/caliper Hyperledger. 2022. Hyperledger Caliper. https:\/\/www.hyperledger.org\/use\/caliper"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.14778\/3007263.3007270"},{"key":"e_1_2_1_21_1","volume-title":"Leutenegger and Daniel Dias","author":"Scott","year":"1993","unstructured":"Scott T. Leutenegger and Daniel Dias . 1993 . A Modeling Study of the TPC-C Benchmark. In SIGMOD. 22--31. Scott T. Leutenegger and Daniel Dias. 1993. A Modeling Study of the TPC-C Benchmark. In SIGMOD. 22--31."},{"key":"e_1_2_1_22_1","volume-title":"Troy McConaghy, Greg McMullen, Ryan Henderson, Sylvain Bellemare, and Alberto Granzotto.","author":"McConaghy Trent","year":"2018","unstructured":"Trent McConaghy , Rodolphe Marques , Andreas M\u00fcller , Dimitri De Jonghe , Troy McConaghy, Greg McMullen, Ryan Henderson, Sylvain Bellemare, and Alberto Granzotto. 2018 . BigchainDB: A Scalable Blockchain Database . (2018). https:\/\/www.bigchaindb.com\/whitepaper\/bigchaindb-whitepaper.pdf Trent McConaghy, Rodolphe Marques, Andreas M\u00fcller, Dimitri De Jonghe, Troy McConaghy, Greg McMullen, Ryan Henderson, Sylvain Bellemare, and Alberto Granzotto. 2018. BigchainDB: A Scalable Blockchain Database. (2018). https:\/\/www.bigchaindb.com\/whitepaper\/bigchaindb-whitepaper.pdf"},{"key":"e_1_2_1_23_1","volume-title":"Freedman","author":"Melara Marcela S.","year":"2015","unstructured":"Marcela S. Melara , Aaron Blankstein , Joseph Bonneau , Edward W. Felten , and Michael J . Freedman . 2015 . CONIKS : Bringing Key Transparency to End Users. In Usenix Security . 383--398. Marcela S. Melara, Aaron Blankstein, Joseph Bonneau, Edward W. Felten, and Michael J. Freedman. 2015. CONIKS: Bringing Key Transparency to End Users. In Usenix Security. 383--398."},{"key":"e_1_2_1_24_1","doi-asserted-by":"crossref","unstructured":"Ralph C. Merkle. 1988. A Digital Signature Based on a Conventional Encryption Function. In Advances in Cryptology. 369--378. Ralph C. Merkle. 1988. A Digital Signature Based on a Conventional Encryption Function. In Advances in Cryptology. 369--378.","DOI":"10.1007\/3-540-48184-2_32"},{"key":"e_1_2_1_25_1","unstructured":"Microsoft. 2022. The Confidential Consortium Framework. https:\/\/github.com\/microsoft\/CCF Microsoft. 2022. The Confidential Consortium Framework. https:\/\/github.com\/microsoft\/CCF"},{"key":"e_1_2_1_26_1","unstructured":"Microsoft. 2022. Ledger overview. https:\/\/archive.ph\/mvXCt Microsoft. 2022. Ledger overview. https:\/\/archive.ph\/mvXCt"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.14778\/3342263.3342632"},{"key":"e_1_2_1_28_1","unstructured":"FISCO open-source working group. 2022. FISCO BCOS. http:\/\/www.fisco-bcos.org\/ FISCO open-source working group. 2022. FISCO BCOS. http:\/\/www.fisco-bcos.org\/"},{"key":"e_1_2_1_29_1","volume-title":"Pinocchio: Nearly Practical Verifiable Computation. In SP. 238--252.","author":"Parno Bryan","year":"2013","unstructured":"Bryan Parno , Jon Howell , Craig Gentry , and Mariana Raykova . 2013 . Pinocchio: Nearly Practical Verifiable Computation. In SP. 238--252. Bryan Parno, Jon Howell, Craig Gentry, and Mariana Raykova. 2013. Pinocchio: Nearly Practical Verifiable Computation. In SP. 238--252."},{"key":"e_1_2_1_30_1","doi-asserted-by":"crossref","unstructured":"Swapnil Patil Milo Polte Kai Ren Wittawat Tantisiriroj Lin Xiao Julio L\u00f3pez Garth Gibson Adam Fuchs and Billie Rinaldi. 2011. YCSB++ benchmarking and performance debugging advanced features in scalable table stores. In SOCC. 1--14. Swapnil Patil Milo Polte Kai Ren Wittawat Tantisiriroj Lin Xiao Julio L\u00f3pez Garth Gibson Adam Fuchs and Billie Rinaldi. 2011. YCSB++ benchmarking and performance debugging advanced features in scalable table stores. In SOCC. 1--14.","DOI":"10.1145\/2038916.2038925"},{"key":"e_1_2_1_31_1","volume-title":"Wintersteige","author":"Russinovich Mark","year":"2019","unstructured":"Mark Russinovich , Edward Ashton , Christine Avanessians , Miguel Castro , Amaury Chamayou , Sylvan Clebsch , Manuel Costa , Cedric Fournet , Matthew Kerner , Sid Krishna , Julien Maffre , Thomas Moscibroda , Kartik Nayak , Olga Ohrimenko , Felix Schuster , Roy Schuster , Alex Shamis , Olga Vrousgou , and Christoph M . Wintersteige . 2019 . CCF : A Framework for Building Confidential Verifiable Replicated Services . https:\/\/github.com\/microsoft\/CCF\/blob\/main\/CCF-TECHNICAL-REPORT.pdf Mark Russinovich, Edward Ashton, Christine Avanessians, Miguel Castro, Amaury Chamayou, Sylvan Clebsch, Manuel Costa, Cedric Fournet, Matthew Kerner, Sid Krishna, Julien Maffre, Thomas Moscibroda, Kartik Nayak, Olga Ohrimenko, Felix Schuster, Roy Schuster, Alex Shamis, Olga Vrousgou, and Christoph M. Wintersteige. 2019. CCF: A Framework for Building Confidential Verifiable Replicated Services. https:\/\/github.com\/microsoft\/CCF\/blob\/main\/CCF-TECHNICAL-REPORT.pdf"},{"key":"e_1_2_1_32_1","doi-asserted-by":"crossref","unstructured":"Mark Ryan. 2014. Enhanced certificate transparency and end-to-end encrypted mail. In NDSS. Mark Ryan. 2014. Enhanced certificate transparency and end-to-end encrypted mail. In NDSS.","DOI":"10.14722\/ndss.2014.23379"},{"key":"e_1_2_1_33_1","unstructured":"Felix Martin Schuhknecht Ankur Sharma Jens Dittrich and Divya Agrawal. 2019. ChainifyDB: How to Blockchainify any Data Management System. http:\/\/arxiv.org\/abs\/1912.04820 Felix Martin Schuhknecht Ankur Sharma Jens Dittrich and Divya Agrawal. 2019. ChainifyDB: How to Blockchainify any Data Management System. http:\/\/arxiv.org\/abs\/1912.04820"},{"key":"e_1_2_1_34_1","volume-title":"Narayanan Sundaram, Samuel Madden, and Michael Stonebraker.","author":"Taft Rebecca","year":"2014","unstructured":"Rebecca Taft , Manasi Vartak , Nadathur Rajagopalan Satish , Narayanan Sundaram, Samuel Madden, and Michael Stonebraker. 2014 . GenBase: a complex analytics genomics benchmark. In SIGMOD. 177--188. Rebecca Taft, Manasi Vartak, Nadathur Rajagopalan Satish, Narayanan Sundaram, Samuel Madden, and Michael Stonebraker. 2014. GenBase: a complex analytics genomics benchmark. In SIGMOD. 177--188."},{"key":"e_1_2_1_35_1","unstructured":"UCB. 2021. MerkleSquare. https:\/\/github.com\/ucbrise\/MerkleSquare UCB. 2021. MerkleSquare. https:\/\/github.com\/ucbrise\/MerkleSquare"},{"key":"e_1_2_1_36_1","doi-asserted-by":"crossref","unstructured":"M. Vieira and H. Madeira. 2005. Towards a security benchmark for database management systems. In DSN. 592--601. M. Vieira and H. Madeira. 2005. Towards a security benchmark for database management systems. In DSN. 592--601.","DOI":"10.1109\/DSN.2005.93"},{"key":"e_1_2_1_37_1","volume-title":"Ethereum: A secure decentralised generalised transaction ledger. Ethereum project yellow paper 151","author":"Gavin Wood","year":"2014","unstructured":"Gavin Wood et al. 2014 . Ethereum: A secure decentralised generalised transaction ledger. Ethereum project yellow paper 151 , 2014 (2014), 1--32. Gavin Wood et al. 2014. Ethereum: A secure decentralised generalised transaction ledger. Ethereum project yellow paper 151, 2014 (2014), 1--32."},{"key":"e_1_2_1_38_1","volume-title":"Litmus: Towards a Practical Database Management System with Verifiable ACID Properties and Transaction Correctness. In SIGMOD. 1478--1492.","author":"Xia Yu","year":"2022","unstructured":"Yu Xia , Xiangyao Yu , Matthew Butrovich , Andrew Pavlo , and Srinivas Devadas . 2022 . Litmus: Towards a Practical Database Management System with Verifiable ACID Properties and Transaction Correctness. In SIGMOD. 1478--1492. Yu Xia, Xiangyao Yu, Matthew Butrovich, Andrew Pavlo, and Srinivas Devadas. 2022. Litmus: Towards a Practical Database Management System with Verifiable ACID Properties and Transaction Correctness. In SIGMOD. 1478--1492."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415540"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.14778\/3583140.3583152"},{"key":"e_1_2_1_41_1","volume-title":"Sheng Wang, and Xiaokui Xiao.","author":"Yue Cong","year":"2020","unstructured":"Cong Yue , Zhongle Xie , Meihui Zhang , Gang Chen , Beng Chin Ooi , Sheng Wang, and Xiaokui Xiao. 2020 . Analysis of Indexing Structures for Immutable Data. In SIGMOD. 925--935. Cong Yue, Zhongle Xie, Meihui Zhang, Gang Chen, Beng Chin Ooi, Sheng Wang, and Xiaokui Xiao. 2020. Analysis of Indexing Structures for Immutable Data. In SIGMOD. 925--935."},{"key":"e_1_2_1_42_1","doi-asserted-by":"crossref","unstructured":"Yupeng Zhang Daniel Genkin Jonathan Katz Dimitrios Papadopoulos and Charalampos Papamanthou. 2017. vSQL: Verifying Arbitrary SQL Queries over Dynamic Outsourced Databases. In SP. 863--880. Yupeng Zhang Daniel Genkin Jonathan Katz Dimitrios Papadopoulos and Charalampos Papamanthou. 2017. vSQL: Verifying Arbitrary SQL Queries over Dynamic Outsourced Databases. In SP. 863--880.","DOI":"10.1109\/SP.2017.43"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1007\/s12652-018-0757-8"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3598581.3598588","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,23]],"date-time":"2024-10-23T22:48:37Z","timestamp":1729723717000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3598581.3598588"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5]]},"references-count":43,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2023,5]]}},"alternative-id":["10.14778\/3598581.3598588"],"URL":"https:\/\/doi.org\/10.14778\/3598581.3598588","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2023,5]]},"assertion":[{"value":"2023-07-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}