{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T23:51:30Z","timestamp":1781135490850,"version":"3.54.1"},"publisher-location":"New York, NY, USA","reference-count":18,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,6,12]],"date-time":"2022-06-12T00:00:00Z","timestamp":1654992000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,6,17]]},"DOI":"10.1145\/3531348.3532176","type":"proceedings-article","created":{"date-parts":[[2022,5,16]],"date-time":"2022-05-16T16:09:31Z","timestamp":1652717371000},"page":"1-9","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["DeepBench"],"prefix":"10.1145","author":[{"given":"Stefano","family":"Belloni","sequence":"first","affiliation":[{"name":"SAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Ritter","sequence":"additional","affiliation":[{"name":"SAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marco","family":"Schr\u00f6der","sequence":"additional","affiliation":[{"name":"SAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nils","family":"R\u00f6rup","sequence":"additional","affiliation":[{"name":"SAP"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,6,12]]},"reference":[{"key":"e_1_3_2_1_1_2","volume-title":"Research Directions for Principles of Data Management (Dagstuhl Perspectives Workshop 16151)","author":"Abiteboul Serge","year":"2018","unstructured":"Serge Abiteboul , Marcelo Arenas , Pablo Barcel\u00f3 , Meghyn Bienvenu , Diego Calvanese , Claire David , Richard Hull , Eyke H\u00fcllermeier , Benny Kimelfeld , Leonid Libkin , Wim Martens , Tova Milo , Filip Murlak , Frank Neven , Magdalena Ortiz , Thomas Schwentick , Julia Stoyanovich , Jianwen Su , Dan Suciu , Victor Vianu , and Ke Yi . 2018 . Research Directions for Principles of Data Management (Dagstuhl Perspectives Workshop 16151) . Dagstuhl Manifestos 7, 1 (2018), 1\u201329. Serge Abiteboul, Marcelo Arenas, Pablo Barcel\u00f3, Meghyn Bienvenu, Diego Calvanese, Claire David, Richard Hull, Eyke H\u00fcllermeier, Benny Kimelfeld, Leonid Libkin, Wim Martens, Tova Milo, Filip Murlak, Frank Neven, Magdalena Ortiz, Thomas Schwentick, Julia Stoyanovich, Jianwen Su, Dan Suciu, Victor Vianu, and Ke Yi. 2018. Research Directions for Principles of Data Management (Dagstuhl Perspectives Workshop 16151). Dagstuhl Manifestos 7, 1 (2018), 1\u201329."},{"key":"e_1_3_2_1_2_2","doi-asserted-by":"crossref","unstructured":"Tim Bray et al. 2014. The javascript object notation (json) data interchange format. (2014).  Tim Bray et al. 2014. The javascript object notation (json) data interchange format. (2014).","DOI":"10.17487\/rfc7159"},{"key":"e_1_3_2_1_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-13021-7_12"},{"key":"e_1_3_2_1_4_2","volume-title":"Thomas Neumann, Meikel Poess, Kai-Uwe Sattler, Michael Seibold, Eric Simon, and Florian Waas.","author":"Cole Richard L.","year":"2011","unstructured":"Richard L. Cole , Florian Funke , Leo Giakoumakis , Wey Guy , Alfons Kemper , Stefan Krompass , Harumi A. Kuno , Raghunath Othayoth Nambiar , Thomas Neumann, Meikel Poess, Kai-Uwe Sattler, Michael Seibold, Eric Simon, and Florian Waas. 2011 . The mixed workload CH-benCHmark. In DBTest, Goetz Graefe and Kenneth Salem (Eds.). ACM , 8. Richard L. Cole, Florian Funke, Leo Giakoumakis, Wey Guy, Alfons Kemper, Stefan Krompass, Harumi A. Kuno, Raghunath Othayoth Nambiar, Thomas Neumann, Meikel Poess, Kai-Uwe Sattler, Michael Seibold, Eric Simon, and Florian Waas. 2011. The mixed workload CH-benCHmark. In DBTest, Goetz Graefe and Kenneth Salem (Eds.). ACM, 8."},{"key":"e_1_3_2_1_5_2","doi-asserted-by":"crossref","unstructured":"Brian F. Cooper Adam Silberstein Erwin Tam Raghu Ramakrishnan and Russell Sears. 2010. Benchmarking cloud serving systems with YCSB. In SoCC. ACM 143\u2013154.  Brian F. Cooper Adam Silberstein Erwin Tam Raghu Ramakrishnan and Russell Sears. 2010. Benchmarking cloud serving systems with YCSB. In SoCC . ACM 143\u2013154.","DOI":"10.1145\/1807128.1807152"},{"key":"e_1_3_2_1_6_2","doi-asserted-by":"publisher","DOI":"10.14778\/2732240.2732246"},{"key":"e_1_3_2_1_7_2","volume-title":"Hassan Chafi, Andrey Gubichev, Arnau Prat-P\u00e9rez, Minh-Duc Pham, and Peter A. Boncz.","author":"Erling Orri","year":"2015","unstructured":"Orri Erling , Alex Averbuch , Josep Llu\u00eds Larriba-Pey , Hassan Chafi, Andrey Gubichev, Arnau Prat-P\u00e9rez, Minh-Duc Pham, and Peter A. Boncz. 2015 . The LDBC Social Network Benchmark: Interactive Workload . In SIGMOD. ACM , 619\u2013630. Orri Erling, Alex Averbuch, Josep Llu\u00eds Larriba-Pey, Hassan Chafi, Andrey Gubichev, Arnau Prat-P\u00e9rez, Minh-Duc Pham, and Peter A. Boncz. 2015. The LDBC Social Network Benchmark: Interactive Workload. In SIGMOD. ACM, 619\u2013630."},{"key":"e_1_3_2_1_8_2","volume-title":"The Benchmark Handbook for Database and Transaction Systems","author":"Gray Jim","unstructured":"Jim Gray . 1993. Database and Transaction Processing Performance Handbook . In The Benchmark Handbook for Database and Transaction Systems ( 2 nd Edition). Morgan Kaufmann . Jim Gray. 1993. Database and Transaction Processing Performance Handbook. In The Benchmark Handbook for Database and Transaction Systems (2nd Edition). Morgan Kaufmann.","edition":"2"},{"key":"e_1_3_2_1_9_2","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415551"},{"key":"e_1_3_2_1_10_2","unstructured":"Henrik Ingo and David Daly. 2020. Automated system performance testing at MongoDB. In DBTest@SIGMOD. 3:1\u20133:6.  Henrik Ingo and David Daly. 2020. Automated system performance testing at MongoDB. In DBTest@SIGMOD . 3:1\u20133:6."},{"key":"e_1_3_2_1_11_2","doi-asserted-by":"crossref","unstructured":"Shiva Jahangiri. 2021. Wisconsin Benchmark Data Generator: To JSON and Beyond. In SIGMOD Guoliang Li Zhanhuai Li Stratos Idreos and Divesh Srivastava (Eds.). ACM 2887\u20132889.  Shiva Jahangiri. 2021. Wisconsin Benchmark Data Generator: To JSON and Beyond. In SIGMOD Guoliang Li Zhanhuai Li Stratos Idreos and Divesh Srivastava (Eds.). ACM 2887\u20132889.","DOI":"10.1145\/3448016.3450577"},{"key":"e_1_3_2_1_12_2","doi-asserted-by":"publisher","DOI":"10.14778\/3352063.3352140"},{"key":"e_1_3_2_1_13_2","first-page":"387","article-title":"DeWitt clauses: Can we protect purchasers without hurting","volume":"25","author":"Read Anthony G","year":"2006","unstructured":"Anthony G Read . 2006 . DeWitt clauses: Can we protect purchasers without hurting Microsoft. Rev. Litig. 25 (2006), 387 . Anthony G Read. 2006. DeWitt clauses: Can we protect purchasers without hurting Microsoft. Rev. Litig. 25 (2006), 387.","journal-title":"Microsoft. Rev. Litig."},{"key":"e_1_3_2_1_14_2","unstructured":"Manuel Rigger and Zhendong Su. 2020. Testing Database Engines via Pivoted Query Synthesis. (2020) 667\u2013682.  Manuel Rigger and Zhendong Su. 2020. Testing Database Engines via Pivoted Query Synthesis. (2020) 667\u2013682."},{"key":"e_1_3_2_1_15_2","doi-asserted-by":"crossref","unstructured":"Daniel Ritter Norman May Kai Sachs and Stefanie Rinderle-Ma. 2016. Benchmarking integration pattern implementations. In DEBS. ACM 125\u2013136.  Daniel Ritter Norman May Kai Sachs and Stefanie Rinderle-Ma. 2016. Benchmarking integration pattern implementations. In DEBS . ACM 125\u2013136.","DOI":"10.1145\/2933267.2933269"},{"key":"e_1_3_2_1_16_2","unstructured":"Andreas Seltenreich Bo Tang and Sjoerd Mullender. 2016. SQLSmith. https:\/\/github.com\/anse1\/sqlsmith.  Andreas Seltenreich Bo Tang and Sjoerd Mullender. 2016. SQLSmith. https:\/\/github.com\/anse1\/sqlsmith."},{"key":"e_1_3_2_1_17_2","first-page":"1","article-title":"Get Real: How Benchmarks Fail to Represent the Real World. In DBTest@SIGMOD, Alexander B\u00f6hm and Tilmann Rabl (Eds.)","volume":"1","author":"Vogelsgesang Adrian","year":"2018","unstructured":"Adrian Vogelsgesang , Michael Haubenschild , Jan Finis , Alfons Kemper , Viktor Leis , Tobias M\u00fchlbauer , Thomas Neumann , and Manuel Then . 2018 . Get Real: How Benchmarks Fail to Represent the Real World. In DBTest@SIGMOD, Alexander B\u00f6hm and Tilmann Rabl (Eds.) . ACM , 1 : 1 \u2013 1 :6. Adrian Vogelsgesang, Michael Haubenschild, Jan Finis, Alfons Kemper, Viktor Leis, Tobias M\u00fchlbauer, Thomas Neumann, and Manuel Then. 2018. Get Real: How Benchmarks Fail to Represent the Real World. In DBTest@SIGMOD, Alexander B\u00f6hm and Tilmann Rabl (Eds.). ACM, 1:1\u20131:6.","journal-title":"ACM"},{"key":"e_1_3_2_1_18_2","volume-title":"SQUIRREL: Testing Database Management Systems with Language Validity and Coverage Feedback.","author":"Zhong Rui","year":"2020","unstructured":"Rui Zhong , Yongheng Chen , Hong Hu , Hangfan Zhang , Wenke Lee , and Dinghao Wu . 2020 . SQUIRREL: Testing Database Management Systems with Language Validity and Coverage Feedback. (2020), 955\u2013970. Rui Zhong, Yongheng Chen, Hong Hu, Hangfan Zhang, Wenke Lee, and Dinghao Wu. 2020. SQUIRREL: Testing Database Management Systems with Language Validity and Coverage Feedback. (2020), 955\u2013970."}],"event":{"name":"SIGMOD\/PODS '22: International Conference on Management of Data","location":"Philadelphia PA USA","acronym":"SIGMOD\/PODS '22","sponsor":["SIGMOD ACM Special Interest Group on Management of Data"]},"container-title":["Proceedings of the 2022 workshop on 9th International Workshop of Testing Database Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3531348.3532176","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3531348.3532176","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:09:21Z","timestamp":1750183761000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3531348.3532176"}},"subtitle":["Benchmarking JSON Document Stores"],"short-title":[],"issued":{"date-parts":[[2022,6,12]]},"references-count":18,"alternative-id":["10.1145\/3531348.3532176","10.1145\/3531348"],"URL":"https:\/\/doi.org\/10.1145\/3531348.3532176","relation":{},"subject":[],"published":{"date-parts":[[2022,6,12]]},"assertion":[{"value":"2022-06-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}