{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,18]],"date-time":"2026-04-18T12:43:56Z","timestamp":1776516236774,"version":"3.51.2"},"reference-count":119,"publisher":"Association for Computing Machinery (ACM)","issue":"10","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62302370, 62272369, and 62372352"],"award-info":[{"award-number":["62302370, 62272369, and 62372352"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2026,7,31]]},"abstract":"<jats:p>Database Management System (DBMS) fuzzing is an automated testing technique aimed at detecting errors and vulnerabilities in DBMSs by generating, mutating, and executing test cases. It not only reduces the time and cost of manual testing but also enhances detection coverage, providing valuable assistance in developing commercial DBMSs. Existing fuzzing surveys mainly focus on general-purpose software. However, DBMSs are different from them in terms of internal structure, input\/output, and test objectives, requiring specialized fuzzing strategies. Therefore, this article focuses on DBMS fuzzing and provides a comprehensive review and comparison of the methods in this field. We first introduce the fundamental concepts. Then, we systematically define a general fuzzing procedure and decompose and categorize existing methods. Furthermore, we classify existing methods from the testing objective perspective, covering various components in DBMSs. For representative works, more detailed descriptions are provided to analyze their strengths and limitations. Additionally, we review recent fuzzing methods for non-relational DBMSs as well as emerging techniques. To objectively evaluate the performance of each method, we present an open-source DBMS fuzzing toolkit, OpenDBFuzz. Based on this toolkit, we conduct a detailed experimental comparative analysis of existing methods and finally discuss future research directions.<\/jats:p>\n                  <jats:p\/>","DOI":"10.1145\/3799227","type":"journal-article","created":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T20:51:56Z","timestamp":1772830316000},"page":"1-36","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["A Comprehensive Survey on Database Management System Fuzzing: Techniques, Taxonomy and Evaluation"],"prefix":"10.1145","volume":"58","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4370-6954","authenticated-orcid":false,"given":"Xiyue","family":"Gao","sequence":"first","affiliation":[{"name":"School of computer science and technology, Xidian University","place":["Xi\u2019an, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-3939-2986","authenticated-orcid":false,"given":"Zhuang","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Xidian University","place":["Xi\u2019an, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5569-0780","authenticated-orcid":false,"given":"Jiangtao","family":"Cui","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Xidian University","place":["Xi\u2019an, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2382-6289","authenticated-orcid":false,"given":"Hui","family":"Li","sequence":"additional","affiliation":[{"name":"Xidian University","place":["Xi\u2019an, China"]},{"name":"Shanghai Yunxi Technology Co., Ltd.","place":["Xi\u2019an, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-3729-8514","authenticated-orcid":false,"given":"Hui","family":"Zhang","sequence":"additional","affiliation":[{"name":"Shanghai Yunxi Technology Co., Ltd.","place":["Shanghai, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-7287-6075","authenticated-orcid":false,"given":"Kewei","family":"Wei","sequence":"additional","affiliation":[{"name":"Shanghai Yunxi Technology Co., Ltd.","place":["Shanghai, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7438-4535","authenticated-orcid":false,"given":"Kankan","family":"Zhao","sequence":"additional","affiliation":[{"name":"Shanghai Yunxi Technology Co., Ltd.","place":["Shanghai, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,4,18]]},"reference":[{"key":"e_1_3_3_2_2","unstructured":"1996. TPC-H. Retrieved March 20 2026 from https:\/\/www.tpc.org"},{"key":"e_1_3_3_3_2","unstructured":"2011. Data science revealed: A data-driven glimpse into the burgeoning new field. Retrieved March 20 2026 from https:\/\/www.predictiveanalyticsworld.com\/machinelearningtimes\/august12\/emc-data-science-study.pdf"},{"key":"e_1_3_3_4_2","unstructured":"2015. AFL: American Fuzzy Lop. Retrieved March 20 2026 from http:\/\/lcamtuf.coredump.cx\/afl\/"},{"key":"e_1_3_3_5_2","unstructured":"2015. SQLsmith. Retrieved March 20 2026 from https:\/\/github.com\/anse1\/sqlsmith"},{"key":"e_1_3_3_6_2","unstructured":"2016. Honggfuzz. Retrieved March 20 2026 from https:\/\/google.github.io\/honggfuzz\/"},{"key":"e_1_3_3_7_2","unstructured":"2016. OSS-Fuzz: Continuous Fuzzing for Open Source Software. Retrieved March 20 2026 from https:\/\/github.com\/google\/oss-fuzz"},{"key":"e_1_3_3_8_2","unstructured":"2017. LibFuzzer - A Library For Coverage-guided Fuzz Testing. Retrieved March 20 2026 from http:\/\/llvm.org\/docs\/LibFuzzer.html"},{"key":"e_1_3_3_9_2","unstructured":"2021. CockroachDB Performance Bug Reports. Retrieved March 20 2026 from https:\/\/github.com\/cockroachdb\/cockroach\/issues?q=performance"},{"key":"e_1_3_3_10_2","unstructured":"2021. PostgreSQL Performance Bug Reports. Retrieved March 20 2026 from https:\/\/www.postgresql.org\/search\/?m=1&q=performance&l=8&d=-1&s=r"},{"key":"e_1_3_3_11_2","unstructured":"2021. SCOTT schema. Retrieved March 20 2026 from https:\/\/www.orafaq.com\/wiki\/SCOTT"},{"key":"e_1_3_3_12_2","unstructured":"2023. MySQL Bugs Home. Retrieved March 20 2026 from https:\/\/bugs.mysql.com\/"},{"key":"e_1_3_3_13_2","first-page":"329","volume-title":"Proceedings of the International Conference on Automated Software Engineering","author":"Khalek Shadi Abdul","year":"2010","unstructured":"Shadi Abdul Khalek and Sarfraz Khurshid. 2010. Automated SQL query generation for systematic testing of database engines. In Proceedings of the International Conference on Automated Software Engineering. ACM, 329\u2013332."},{"key":"e_1_3_3_14_2","first-page":"2060","volume-title":"Proceedings of the International Conference on Software Engineering","author":"Ba Jinsheng","year":"2023","unstructured":"Jinsheng Ba and Manuel Rigger. 2023. Testing database engines via query plan guidance. In Proceedings of the International Conference on Software Engineering. ACM, 2060\u20132071."},{"key":"e_1_3_3_15_2","first-page":"1","volume-title":"Proceedings of the International Conference on Software Engineering","author":"Ba Jinsheng","year":"2024","unstructured":"Jinsheng Ba and Manuel Rigger. 2024. Cert: Finding performance issues in database systems through the lens of cardinality estimation. In Proceedings of the International Conference on Software Engineering. 1\u201313."},{"key":"e_1_3_3_16_2","doi-asserted-by":"crossref","unstructured":"Jinsheng Ba and Manuel Rigger. 2024. Keep it simple: Testing databases via differential query plans. Proceedings of the ACM on Management of Data 2 3 (2024) 1\u201326.","DOI":"10.1145\/3654991"},{"key":"e_1_3_3_17_2","first-page":"1243","volume-title":"Proceedings of the International Conference on Very Large Data Bases","author":"Bati Hardik","year":"2007","unstructured":"Hardik Bati, Leo Giakoumakis, Steve Herbert, and Aleksandras Surna. 2007. A genetic approach for random testing of database systems. In Proceedings of the International Conference on Very Large Data Bases. VLDB Endowment, 1243\u20131251."},{"key":"e_1_3_3_18_2","doi-asserted-by":"crossref","unstructured":"B. Beizer. 1992. Software testing techniques. Software Testing Verification and Reliability 2 4 (1992) 215\u2013216.","DOI":"10.1002\/stvr.4370020406"},{"key":"e_1_3_3_19_2","unstructured":"Donald A. Berry and Bert Fristedt. 1985. Bandit problems: Sequential allocation of experiments (Monographs on statistics and applied probability). London: Chapman and Hall 5 71-87 (1985) 7\u20137."},{"key":"e_1_3_3_20_2","first-page":"1985","volume-title":"Proceedings of the USENIX Security Symposium","author":"Blazytko Tim","year":"2019","unstructured":"Tim Blazytko, Matt Bishop, Cornelius Aschermann, Justin Cappos, Moritz Schl\u00f6gel, Nadia Korshun, Ali Abbasi, Marco Schweighauser, Sebastian Schinzel, Sergej Schumilo, et\u00a0al. 2019. GRIMOIRE: Synthesizing structure while fuzzing. In Proceedings of the USENIX Security Symposium. USENIX Association, 1985\u20132002."},{"key":"e_1_3_3_21_2","first-page":"1097","volume-title":"Proceedings of the International Conference on Very Large Data Bases","author":"Bruno Nicolas","year":"2005","unstructured":"Nicolas Bruno and Surajit Chaudhuri. 2005. Flexible database generators. In Proceedings of the International Conference on Very Large Data Bases. VLDB Endowment, 1097\u20131107."},{"key":"e_1_3_3_22_2","doi-asserted-by":"crossref","unstructured":"N. Bruno S. Chaudhuri and D. Thomas. 2006. Generating queries with cardinality constraints for DBMS testing. IEEE Transactions on Knowledge and Data Engineering 18 12 (2006) 1721\u20131725.","DOI":"10.1109\/TKDE.2006.190"},{"key":"e_1_3_3_23_2","first-page":"2329","volume-title":"Proceedings of the Conference on Computer and Communications Security","author":"B\u00f6hme Marcel","year":"2017","unstructured":"Marcel B\u00f6hme, Van-Thuan Pham, Manh-Dung Nguyen, and Abhik Roychoudhury. 2017. Directed greybox fuzzing. In Proceedings of the Conference on Computer and Communications Security. ACM, 2329\u20132344."},{"key":"e_1_3_3_24_2","first-page":"1032","volume-title":"Proceedings of the Conference on Computer and Communications Security","author":"B\u00f6hme Marcel","year":"2016","unstructured":"Marcel B\u00f6hme, Van-Thuan Pham, and Abhik Roychoudhury. 2016. Coverage-based greybox fuzzing as markov chain. In Proceedings of the Conference on Computer and Communications Security. ACM, 1032\u20131043."},{"key":"e_1_3_3_25_2","unstructured":"Tsong Y. Chen Shing C. Cheung and Shiu Ming Yiu. 2020. Metamorphic testing: A new approach for generating next test cases. arXiv:2002.12543. Retrieved from https:\/\/arxiv.org\/abs\/2002.12543"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3395032.3395322"},{"key":"e_1_3_3_27_2","volume-title":"Proceedings of the Conference on Innovative Data Systems Research","author":"Chu Shumo","year":"2017","unstructured":"Shumo Chu, Chenglong Wang, Konstantin Weitz, and Alvin Cheung. 2017. Cosette: An automated prover for SQL. In Proceedings of the Conference on Innovative Data Systems Research."},{"key":"e_1_3_3_28_2","first-page":"1","volume-title":"Proceedings of the International Conference on Automated Software Engineering","author":"Cui Ziyu","year":"2022","unstructured":"Ziyu Cui, Wensheng Dou, Qianwang Dai, Jiansen Song, Wei Wang, Jun Wei, and Dan Ye. 2022. Differentially testing database transactions for fun and profit. In Proceedings of the International Conference on Automated Software Engineering. ACM, 1\u201312."},{"key":"e_1_3_3_29_2","doi-asserted-by":"crossref","unstructured":"Ziyu Cui Wensheng Dou Yu Gao Rui Yang Yingying Zheng Jiansen Song Yuan Feng and Jun Wei. 2025. Simple testing can expose most critical transaction bugs: Understanding and detecting write-specific serializability violations in database systems. In Proceedings of the VLDB Endowment.","DOI":"10.14778\/3742728.3742747"},{"key":"e_1_3_3_30_2","first-page":"667","volume-title":"Proceedings of the International Conference on Software Engineering","author":"Deng Wenqian","year":"2025","unstructured":"Wenqian Deng, Jie Liang, Zhiyong Wu, Jingzhou Fu, Mingzhe Wang, and Yu Jiang. 2025. CONI: Detecting database connector bugs via state-aware test case generation. In Proceedings of the International Conference on Software Engineering. 667\u2013678."},{"key":"e_1_3_3_31_2","doi-asserted-by":"crossref","unstructured":"Wenjing Deng Qiuyang Mang Chengyu Zhang and Manuel Rigger. 2024. Finding logic bugs in spatial database engines via affine equivalent inputs. Proceedings of the ACM on Management of Data 2 6 (2024) 1\u201326.","DOI":"10.1145\/3698810"},{"key":"e_1_3_3_32_2","volume-title":"Proceedings of the International Conference on Software Engineering","author":"Dou Wensheng","year":"2023","unstructured":"Wensheng Dou, Ziyu Cui, Qianwang Dai, Jiansen Song, Dong Wang, Yu Gao, Wei Wang, Jun Wei, Lei Chen, Hanmo Wang, et\u00a0al. 2023. Detecting isolation bugs via transaction oracle construction. In Proceedings of the International Conference on Software Engineering. IEEE."},{"key":"e_1_3_3_33_2","doi-asserted-by":"crossref","unstructured":"Cundi Fang Jie Liang Zhiyong Wu Jingzhou Fu Zhouyang Jia Chun Huang Yu Jiang and Shanshan Li. 2025. DepState: Detecting synchronization failure bugs in distributed database management systems. Proceedings of the ACM on Software Engineering 2 ISSTA (2025) 2001\u20132022.","DOI":"10.1145\/3728965"},{"key":"e_1_3_3_34_2","first-page":"1","volume-title":"Proceedings of the International Conference on Automated Software Engineering","author":"Fu Jingzhou","year":"2022","unstructured":"Jingzhou Fu, Jie Liang, Zhiyong Wu, Mingzhe Wang, and Yu Jiang. 2022. Griffin: Grammar-free DBMS fuzzing. In Proceedings of the International Conference on Automated Software Engineering. ACM, 1\u201312."},{"key":"e_1_3_3_35_2","first-page":"1061","volume-title":"Proceedings of the European Conference on Computer Systems","author":"Fu Jingzhou","year":"2025","unstructured":"Jingzhou Fu, Jie Liang, Zhiyong Wu, Yanyang Zhao, Shanshan Li, and Yu Jiang. 2025. Understanding and detecting sql function bugs: Using simple boundary arguments to trigger hundreds of DBMS bugs. In Proceedings of the European Conference on Computer Systems. 1061\u20131076."},{"key":"e_1_3_3_36_2","first-page":"844","volume-title":"Proceedings of the International Conference on Management of Data","author":"Galindo-Legaria C\u00e9sar A.","year":"2004","unstructured":"C\u00e9sar A. Galindo-Legaria, Stefano Stefani, and Florian Waas. 2004. Query processing for SQL updates. In Proceedings of the International Conference on Management of Data. ACM, 844\u2013849."},{"key":"e_1_3_3_37_2","doi-asserted-by":"crossref","unstructured":"Xiyue Gao Zhuang Liu Yiran Shen Hui Li Yingfan Liu Hongjun Xiao Yanguo Peng and Jiangtao Cui. 2025. Fucci: Database transaction fuzzing via random conflict construction and multilevel constraint solving. Proceedings of the VLDB Endowment 18 6 (2025) 1879\u20131891.","DOI":"10.14778\/3725688.3725713"},{"key":"e_1_3_3_38_2","unstructured":"Leo Giakoumakis and C\u00e9sar A Galindo-Legaria. 2008. Testing SQL server\u2019s query optimizer: Challenges techniques and experiences. Data Engineering Bulletin 31 1 (2008) 36\u201343."},{"key":"e_1_3_3_39_2","first-page":"206","volume-title":"Proceedings of the Conference on Programming Language Design and Implementation","author":"Godefroid Patrice","year":"2008","unstructured":"Patrice Godefroid, Adam Kiezun, and Michael Y. Levin. 2008. Grammar-based whitebox fuzzing. In Proceedings of the Conference on Programming Language Design and Implementation. ACM, 206\u2013215."},{"key":"e_1_3_3_40_2","doi-asserted-by":"crossref","unstructured":"Eugene Goldberg and Yakov Novikov. 2007. BerkMin: A fast and robust SAT-solver. Discrete Applied Mathematics 155 12 (2007) 1549\u20131561.","DOI":"10.1016\/j.dam.2006.10.007"},{"key":"e_1_3_3_41_2","doi-asserted-by":"crossref","unstructured":"Goetz Graefe. 1993. Query evaluation techniques for large databases. ACM Computing Surveys 25 2 (1993) 73\u2013169.","DOI":"10.1145\/152610.152611"},{"key":"e_1_3_3_42_2","unstructured":"Goetz Graefe. 1995. The cascades framework for query optimization. Data Engineering Bulletin 18 3 (1995) 19\u201329."},{"key":"e_1_3_3_43_2","first-page":"243","volume-title":"Proceedings of the International Conference on Management of Data","author":"Gray Jim","year":"1994","unstructured":"Jim Gray, Prakash Sundaresan, Susanne Englert, Ken Baclawski, and Peter J. Weinberger. 1994. Quickly generating billion-record synthetic databases. In Proceedings of the International Conference on Management of Data. ACM, 243\u2013252."},{"key":"e_1_3_3_44_2","first-page":"1","volume-title":"Proceedings of the Workshop on Testing Database Systems","author":"Gu Zhongxian","year":"2012","unstructured":"Zhongxian Gu, Mohamed A. Soliman, and Florian M. Waas. 2012. Testing the accuracy of query optimizers. In Proceedings of the Workshop on Testing Database Systems. ACM, 1\u20136."},{"key":"e_1_3_3_45_2","first-page":"345","volume-title":"Proceedings of the USENIX Annual Technical Conference","author":"Hao Zongyin","year":"2023","unstructured":"Zongyin Hao, Quanfeng Huang, Chengpeng Wang, Jianfeng Wang, Yushan Zhang, Rongxin Wu, and Charles Zhang. 2023. Pinolo: Detecting logical bugs in database management systems with approximate query synthesis. In Proceedings of the USENIX Annual Technical Conference. 345\u2013358."},{"key":"e_1_3_3_46_2","first-page":"1243","volume-title":"Proceedings of the International Conference on Very Large Data Bases","author":"Houkj\u00e6r Kenneth","year":"2006","unstructured":"Kenneth Houkj\u00e6r, Kristian Torp, and Rico Wind. 2006. Simple and realistic data generation. In Proceedings of the International Conference on Very Large Data Bases. VLDB Endowment, 1243\u20131246."},{"key":"e_1_3_3_47_2","doi-asserted-by":"crossref","unstructured":"William E. Howden. 1978. Theoretical and empirical studies of program testing. IEEE Transactions on Software Engineering SE-4 4 (1978) 293\u2013298.","DOI":"10.1109\/TSE.1978.231514"},{"key":"e_1_3_3_48_2","doi-asserted-by":"crossref","unstructured":"Jiaxin Hu and Rongxin Wu. 2025. SQLaw: Detecting bugs in GPU database management systems via rule-based differential execution. IEEE Transactions on Software Engineering 51 7 (2025) 2144\u20132160.","DOI":"10.1109\/TSE.2025.3574328"},{"key":"e_1_3_3_49_2","doi-asserted-by":"crossref","unstructured":"Toshihide Ibaraki and Tiko Kameda. 1984. On the optimal nesting order for computing N-Relational joins. ACM Transactions on Database Systems 9 3 (1984) 482\u2013502.","DOI":"10.1145\/1270.1498"},{"key":"e_1_3_3_50_2","doi-asserted-by":"crossref","unstructured":"Daniel Jackson. 2002. Alloy: A lightweight object modelling notation. ACM Transactions on Software Engineering and Methodology 11 2 (2002) 256\u2013290.","DOI":"10.1145\/505145.505149"},{"key":"e_1_3_3_51_2","first-page":"492","volume-title":"Proceedings of the ACM International Conference on the Foundations of Software Engineering","author":"Jiang Yu","year":"2024","unstructured":"Yu Jiang, Jie Liang, Fuchen Ma, Yuanliang Chen, Chijin Zhou, Yuheng Shen, Zhiyong Wu, Jingzhou Fu, Mingzhe Wang, Shanshan Li, et\u00a0al. 2024. When fuzzing meets llms: Challenges and opportunities. In Proceedings of the ACM International Conference on the Foundations of Software Engineering. 492\u2013496."},{"key":"e_1_3_3_52_2","first-page":"1","volume-title":"Proceedings of the International Conference on Software Engineering","author":"Jiang Yuancheng","year":"2024","unstructured":"Yuancheng Jiang, Jiahao Liu, Jinsheng Ba, Roland H. C. Yap, Zhenkai Liang, and Manuel Rigger. 2024. Detecting logic bugs in graph database management systems via injective and surjective graph query transformation. In Proceedings of the International Conference on Software Engineering. 1\u201312."},{"key":"e_1_3_3_53_2","unstructured":"Yuancheng Jiang Jianing Wang Chuqi Zhang Roland Yap Zhenkai Liang and Manuel Rigger. 2025. Enhanced differential testing in emerging database systems. arXiv:2501.01236. Retrieved from https:\/\/arxiv.org\/abs\/2501.01236"},{"key":"e_1_3_3_54_2","first-page":"4949","volume-title":"Proceedings of the USENIX Security Symposium","author":"Jiang Zu-Ming","year":"2023","unstructured":"Zu-Ming Jiang, Jia-Ju Bai, and Zhendong Su. 2023. DynSQL: Stateful fuzzing for database management systems with complex and valid SQL query generation. In Proceedings of the USENIX Security Symposium. USENIX Association, 4949\u20134965."},{"key":"e_1_3_3_55_2","first-page":"397","volume-title":"Proceedings of the USENIX Symposium on Operating Systems Design and Implementation","author":"Jiang Zu-Ming","year":"2023","unstructured":"Zu-Ming Jiang, Si Liu, Manuel Rigger, and Zhendong Su. 2023. Detecting transactional bugs in database engines via \\(\\lbrace\\) graph-based \\(\\rbrace\\) oracle construction. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation. 397\u2013417."},{"key":"e_1_3_3_56_2","first-page":"821","volume-title":"Proceedings of the USENIX Symposium on Operating Systems Design and Implementation","author":"Jiang Zu-Ming","year":"2024","unstructured":"Zu-Ming Jiang and Zhendong Su. 2024. Detecting logic bugs in database engines via equivalent expression transformation. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation. 821\u2013835."},{"key":"e_1_3_3_57_2","doi-asserted-by":"crossref","unstructured":"Jinho Jung Hong Hu Joy Arulraj Taesoo Kim and Woonhak Kang. 2019. APOLLO: Automatic detection and diagnosis of performance regressions in database systems. Very Large Data Bases Endowment 13 1 (2019) 57\u201370.","DOI":"10.14778\/3357377.3357382"},{"key":"e_1_3_3_58_2","volume-title":"Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis","author":"Kamm Matteo","year":"2023","unstructured":"Matteo Kamm, Manuel Rigger, Chengyu Zhang, and Zhendong Su. 2023. Testing graph database engines via query partitioning. In Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis."},{"key":"e_1_3_3_59_2","first-page":"238","volume-title":"Proceedings of the International Conference on Automated Software Engineering","author":"Khalek Shadi Abdul","year":"2008","unstructured":"Shadi Abdul Khalek, Bassem Elkarablieh, Yai O. Laleye, and Sarfraz Khurshid. 2008. Query-aware test generation using a relational constraint solver. In Proceedings of the International Conference on Automated Software Engineering. IEEE, 238\u2013247."},{"key":"e_1_3_3_60_2","doi-asserted-by":"crossref","unstructured":"Sarfraz Khurshid and Darko Marinov. 2004. TestEra: Specification-based testing of Java programs using SAT. Automated Software Engineering 11 4 (2004) 403\u2013434.","DOI":"10.1023\/B:AUSE.0000038938.10589.b9"},{"key":"e_1_3_3_61_2","doi-asserted-by":"crossref","unstructured":"Donghun Lee Sang K. Cha and Arthur H. Lee. 2011. A performance anomaly detection and analysis framework for DBMS development. IEEE Transactions on Knowledge and Data Engineering 24 8 (2011) 1345\u20131360.","DOI":"10.1109\/TKDE.2011.88"},{"key":"e_1_3_3_62_2","doi-asserted-by":"crossref","unstructured":"Viktor Leis Andrey Gubichev Atanas Mirchev Peter Boncz Alfons Kemper and Thomas Neumann. 2015. How good are query optimizers really? Very Large Data Bases Endowment 9 3 (2015) 204\u2013215.","DOI":"10.14778\/2850583.2850594"},{"key":"e_1_3_3_63_2","first-page":"475","volume-title":"Proceedings of the International Conference on Automated Software Engineering","author":"Lemieux Caroline","year":"2018","unstructured":"Caroline Lemieux and Koushik Sen. 2018. FairFuzz: A targeted mutation strategy for increasing greybox fuzz testing coverage. In Proceedings of the International Conference on Automated Software Engineering. ACM, 475\u2013485."},{"key":"e_1_3_3_64_2","unstructured":"Jiaqi Li Ke Wang Yaoguang Chen Yajin Zhou Lei Wu and Jiashui Wang. 2023. Detecting dbms bugs with context-sensitive instantiation and multi-plan execution. arXiv:2312.04941. Retrieved from https:\/\/arxiv.org\/abs\/2312.04941"},{"key":"e_1_3_3_65_2","doi-asserted-by":"crossref","unstructured":"Jiaqi Li Yajin Zhou and Lei Wu. 2024. Improving multitasking DBMS fuzzing With more accurate coverage and testcase trimming. IEEE Transactions on Dependable and Secure Computing 22 3 (2024) 2756\u20132770.","DOI":"10.1109\/TDSC.2024.3521591"},{"key":"e_1_3_3_66_2","first-page":"2155","volume-title":"Proceedings of the Information and Knowledge Management","author":"Li Zhan","year":"2016","unstructured":"Zhan Li, Olga Papaemmanouil, and Mitch Cherniack. 2016. Optmark: A Toolkit for Benchmarking Query Optimizers. In Proceedings of the Information and Knowledge Management. ACM, 2155\u20132160."},{"key":"e_1_3_3_67_2","volume-title":"Proceedings of the International Conference on Data Engineering","author":"Liang Jie","year":"2023","unstructured":"Jie Liang, Yaoguang Chen, Zhiyong Wu, Jingzhou Fu, Mingzhe Wang, Yu Jiang, Xiangdong Huang, Ting Chen, Jiashui Wang, and Jiajia Li. 2023. Sequence-oriented DBMS fuzzing. In Proceedings of the International Conference on Data Engineering. IEEE."},{"key":"e_1_3_3_68_2","first-page":"1","volume-title":"Proceedings of the International Conference on Software Engineering","author":"Liang Jie","year":"2024","unstructured":"Jie Liang, Zhiyong Wu, Jingzhou Fu, Mingzhe Wang, Chengnian Sun, and Yu Jiang. 2024. Mozi: Discovering dbms bugs via configuration-based equivalent transformation. In Proceedings of the International Conference on Software Engineering. 1\u201312."},{"key":"e_1_3_3_69_2","first-page":"4309","volume-title":"Proceedings of the USENIX Security Symposium","author":"Liang Yu","year":"2022","unstructured":"Yu Liang, Song Liu, and Hong Hu. 2022. Detecting logical bugs of DBMS with coverage-based guidance. In Proceedings of the USENIX Security Symposium. USENIX Association, 4309\u20134326."},{"key":"e_1_3_3_70_2","doi-asserted-by":"crossref","unstructured":"Li Lin Qinglin Zhu Hongqiao Chen Zhuangda Wang Rongxin Wu and Xiaoheng Xie. 2025. QTRAN: Extending metamorphic-oracle based logical bug detection techniques for multiple-DBMS dialect support. Proceedings of the ACM on Software Engineering 2 ISSTA (2025) 731\u2013752.","DOI":"10.1145\/3728908"},{"key":"e_1_3_3_71_2","doi-asserted-by":"crossref","unstructured":"Ziyue Hua Wei Lin Luyao Ren Zongyang Li Lu Zhang Wenpin Jiao and Tao Xie. 2023. GDsmith: Detecting bugs in cypher graph database engines. In Proceedings of the International Symposium on Software Testing and Analysis. 163\u2013174.","DOI":"10.1145\/3597926.3598046"},{"key":"e_1_3_3_72_2","first-page":"666","volume-title":"Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis","author":"Liu Shuang","year":"2024","unstructured":"Shuang Liu, Junhao Lan, Xiaoning Du, Jiyuan Li, Wei Lu, Jiajun Jiang, and Xiaoyong Du. 2024. Testing graph database systems with graph-state persistence oracle. In Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis. 666\u2013677."},{"key":"e_1_3_3_73_2","first-page":"225","volume-title":"Proceedings of the International Conference on Software Engineering","author":"Liu Xinyu","year":"2022","unstructured":"Xinyu Liu, Qi Zhou, Joy Arulraj, and Alessandro Orso. 2022. Automatic detection of performance bugs in database systems using equivalent queries. In Proceedings of the International Conference on Software Engineering. ACM, 225\u2013236."},{"key":"e_1_3_3_74_2","doi-asserted-by":"crossref","unstructured":"Eric Lo Carsten Binnig Donald Kossmann M. Tamer \u00d6zsu and Wing-Kai Hon. 2010. A framework for testing DBMS features. Very Large Data Bases Endowment 19 2 (2010) 203\u2013230.","DOI":"10.1007\/s00778-009-0157-y"},{"key":"e_1_3_3_75_2","doi-asserted-by":"crossref","unstructured":"Ce Lyu Minghao Zhao Yanhao Wang and Liang Jie. 2025. LLM-based dynamic differential testing for database connectors with reinforcement learning-guided prompt election. In Proceedings of the International Conference on Automated Software Engineering. ACM 3814\u20133818.","DOI":"10.1109\/ASE63991.2025.00322"},{"key":"e_1_3_3_76_2","doi-asserted-by":"crossref","unstructured":"Qiuyang Mang Jinsheng Ba Pinjia He and Manuel Rigger. 2025. Finding logic bugs in graph-processing systems via graph-cutting. Proceedings of the ACM on Management of Data 3 3 (2025) 1\u201327.","DOI":"10.1145\/3725300"},{"key":"e_1_3_3_77_2","volume-title":"Proceedings of the International Conference on Software Engineering","author":"Mang Qiuyang","year":"2024","unstructured":"Qiuyang Mang, Aoyang Fang, Boxi Yu, Hanfei Chen, and Pinjia He. 2024. Testing graph database systems via equivalent query rewriting. In Proceedings of the International Conference on Software Engineering."},{"key":"e_1_3_3_78_2","doi-asserted-by":"publisher","DOI":"10.1145\/3254363"},{"key":"e_1_3_3_79_2","unstructured":"William M. McKeeman. 1998. Differential testing for software. Digital Technical Journal 10 1 (1998) 100\u2013107."},{"key":"e_1_3_3_80_2","doi-asserted-by":"crossref","unstructured":"Phil McMinn Chris J. Wright and Gregory M. Kapfhammer. 2015. The effectiveness of test coverage criteria for relational database schema integrity constraints. ACMTransactions on Software Engineering and Methodology 25 1 (2015) 1\u201349.","DOI":"10.1145\/2818639"},{"key":"e_1_3_3_81_2","first-page":"74","volume-title":"Proceedings of the Benchmarking, Measuring, and Optimizing: 3rd BenchCouncil International Symposium","author":"Mi Kaiming","year":"2021","unstructured":"Kaiming Mi, Chunxi Zhang, Weining Qian, and Rong Zhang. 2021. Artemis: An automatic test suite generator for large scale OLAP database. In Proceedings of the Benchmarking, Measuring, and Optimizing: 3rd BenchCouncil International Symposium. Springer, 74\u201389."},{"key":"e_1_3_3_82_2","doi-asserted-by":"crossref","unstructured":"Barton P. Miller Lars Fredriksen and Bryan So. 1990. An empirical study of the reliability of UNIX utilities. Communications of the ACM 33 12 (1990) 32\u201344.","DOI":"10.1145\/96267.96279"},{"key":"e_1_3_3_83_2","first-page":"499","volume-title":"Proceedings of the International Conference on Management of Data","author":"Mishra Chaitanya","year":"2008","unstructured":"Chaitanya Mishra, Nick Koudas, and Calisto Zuzarte. 2008. Generating targeted queries for database testing. In Proceedings of the International Conference on Management of Data. ACM, 499\u2013510."},{"key":"e_1_3_3_84_2","first-page":"33","volume-title":"Proceedings of the USENIX Symposium on Operating Systems Design and Implementation","author":"Mohan Jayashree","year":"2018","unstructured":"Jayashree Mohan, Ashlie Martinez, Soujanya Ponnapalli, Pandian Raju, and Vijay Chidambaram. 2018. Finding crash-consistency bugs with bounded black-box crash testing. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation. USENIX Association, 33\u201350."},{"key":"e_1_3_3_85_2","unstructured":"MySQL. 2023. The MySQL Test Framework. Retrieved March 20 2026 from https:\/\/dev.mysql.com\/doc\/dev\/mysql-server\/latest\/PAGE_MYSQL_TEST_RUN.html"},{"key":"e_1_3_3_86_2","doi-asserted-by":"crossref","unstructured":"Andrea Neufeld Guido Moerkotte and Peter C Loekemann. 1993. Generating consistent test data: Restricting the search space by A generator formula. Very Large Data Bases Endowment 2 2 (1993) 173\u2013213.","DOI":"10.1007\/BF01232186"},{"key":"e_1_3_3_87_2","first-page":"75","volume-title":"Proceedings of the International Conference on Software Engineering","author":"Pacheco Carlos","year":"2007","unstructured":"Carlos Pacheco, Shuvendu K. Lahiri, Michael D. Ernst, and Thomas Ball. 2007. Feedback-directed random test generation. In Proceedings of the International Conference on Software Engineering. IEEE, 75\u201384."},{"key":"e_1_3_3_88_2","unstructured":"PingCap. 2023. go randgen. Retrieved March 20 2026 from https:\/\/github.com\/pingcap\/go-randgen"},{"key":"e_1_3_3_89_2","first-page":"1045","volume-title":"Proceedings of the International Conference on Very Large Data Bases","author":"Poess Meikel","year":"2004","unstructured":"Meikel Poess and John M. Stephens. 2004. Generating thousand benchmark queries in seconds. In Proceedings of the International Conference on Very Large Data Bases. VLDB Endowment, 1045\u20131053."},{"key":"e_1_3_3_90_2","unstructured":"PostgreSQL. 2023. Regression Tests. Retrieved March 20 2026 from https:\/\/www.postgresql.org\/docs\/current\/regress.html"},{"key":"e_1_3_3_91_2","volume-title":"Proceedings of the Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering","author":"Rigger Manuel","year":"2020","unstructured":"Manuel Rigger and Zhendong Su. 2020. Detecting optimization bugs in database engines via non-optimizing reference engine construction. In Proceedings of the Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM."},{"key":"e_1_3_3_92_2","doi-asserted-by":"crossref","unstructured":"Manuel Rigger and Zhendong Su. 2020. Finding bugs in database systems via query partitioning. Programming Languages 4 OOPSLA (2020) 1\u201330.","DOI":"10.1145\/3428279"},{"key":"e_1_3_3_93_2","first-page":"667","volume-title":"Proceedings of the USENIX Symposium on Operating Systems Design and Implementation","author":"Rigger Manuel","year":"2020","unstructured":"Manuel Rigger and Zhendong Su. 2020. Testing database engines via pivoted query synthesis. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation. USENIX Association, 667\u2013682."},{"key":"e_1_3_3_94_2","first-page":"128","volume-title":"Proceedings of the International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software","author":"Rigger Manuel","year":"2022","unstructured":"Manuel Rigger and Zhendong Su. 2022. Intramorphic testing: A new approach to the test oracle problem. In Proceedings of the International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software. ACM, 128\u2013136."},{"key":"e_1_3_3_95_2","volume-title":"Proceedings of the International Conference on Data Engineering","author":"Shah Shetal","year":"2011","unstructured":"Shetal Shah, S. Sudarshan, Suhas Kajbaje, Sandeep Patidar, Bhanu Pratap Gupta, and Devang Vira. 2011. Generating test data for killing SQL mutants: A constraint-based approach. In Proceedings of the International Conference on Data Engineering."},{"key":"e_1_3_3_96_2","first-page":"618","volume-title":"Proceedings of the International Conference on Very Large Data Bases","author":"Slutz Donald R.","year":"1998","unstructured":"Donald R. Slutz. 1998. Massive stochastic testing of SQL. In Proceedings of the International Conference on Very Large Data Bases. VLDB Endowment, 618\u2013622."},{"key":"e_1_3_3_97_2","volume-title":"Proceedings of the International Conference on Software Engineering","author":"Song Jiansen","year":"2023","unstructured":"Jiansen Song, Wensheng Dou, Ziyu Cui, Qianwang Dai, Wei Wang, Jun Wei, Hua Zhong, and Tao Huang. 2023. Testing database systems via differential query execution. In Proceedings of the International Conference on Software Engineering."},{"key":"e_1_3_3_98_2","doi-asserted-by":"crossref","unstructured":"Jiansen Song Wensheng Dou Yu Gao Ziyu Cui Yingying Zheng Dong Wang Wei Wang Jun Wei and Tao Huang. 2024. Detecting metadata-related logic bugs in database systems via raw database construction. Proceedings of the VLDB Endowment 17 8 (2024) 1884\u20131897.","DOI":"10.14778\/3659437.3659445"},{"key":"e_1_3_3_99_2","doi-asserted-by":"crossref","unstructured":"Jiansen Song Wensheng Dou Yingying Zheng Yu Gao Ziyu Cui Wei Wang and Jun Wei. 2025. Detecting schema-related logic bugs in relational dbmss via equivalent database construction. Proceedings of the VLDB Endowment 18 7 (2025) 2281\u20132294.","DOI":"10.14778\/3734839.3734861"},{"key":"e_1_3_3_100_2","unstructured":"SQLite. 2023. How SQLite Is Tested. Retrieved March 20 2026 from https:\/\/sqlite.org\/testing.html"},{"key":"e_1_3_3_101_2","doi-asserted-by":"crossref","unstructured":"Keith Stobie. 2005. Too darned big to test: Testing large systems is a daunting task but there are steps we can take to ease the pain. Queue 3 1 (2005) 30\u201337.","DOI":"10.1145\/1046931.1046944"},{"key":"e_1_3_3_102_2","doi-asserted-by":"crossref","unstructured":"Xiu Tang Sai Wu Dongxiang Zhang Feifei Li and Gang Chen. 2023. Detecting logic bugs of join optimizations in DBMS. International Conference on Management of Data 1 1 (2023) 1\u201326.","DOI":"10.1145\/3588909"},{"key":"e_1_3_3_103_2","doi-asserted-by":"crossref","unstructured":"Patrick Valduriez and Scott Danforth. 1992. Functional SQL (FSQL) an SQL upward-compatible database programming language. Information Sciences 62 3 (1992) 183\u2013203.","DOI":"10.1016\/0020-0255(92)90015-Z"},{"key":"e_1_3_3_104_2","first-page":"724","volume-title":"Proceedings of the International Conference on Software Engineering","author":"Wang Junjie","year":"2019","unstructured":"Junjie Wang, Bihuan Chen, Lei Wei, and Yang Liu. 2019. Superion: Grammar-aware greybox fuzzing. In Proceedings of the International Conference on Software Engineering. IEEE, 724\u2013735."},{"key":"e_1_3_3_105_2","first-page":"328","volume-title":"Proceedings of the International Conference on Software Engineering: Software Engineering in Practice","author":"Wang Mingzhe","year":"2021","unstructured":"Mingzhe Wang, Zhiyong Wu, Xinyi Xu, Jie Liang, Chijin Zhou, Huafeng Zhang, and Yu Jiang. 2021. Industry practice of coverage-guided enterprise-level DBMS fuzzing. In Proceedings of the International Conference on Software Engineering: Software Engineering in Practice. IEEE, 328\u2013337."},{"key":"e_1_3_3_106_2","doi-asserted-by":"crossref","unstructured":"Shihao Wen Peng Jia Pin Yang and Chi Hu. 2023. Squill: Testing DBMS with correctness feedback and accurate instantiation. Applied Sciences 13 4 (2023) 2519.","DOI":"10.3390\/app13042519"},{"key":"e_1_3_3_107_2","first-page":"679","volume-title":"Proceedings of the International Conference on Software Engineering","author":"Wu Zhiyong","year":"2025","unstructured":"Zhiyong Wu, Jie Liang, Jingzhou Fu, Mingzhe Wang, and Yu Jiang. 2025. Puppy: Finding performance degradation bugs in dbmss via limited-optimization plan construction. In Proceedings of the International Conference on Software Engineering. 679\u2013690."},{"key":"e_1_3_3_108_2","volume-title":"Proceedings of the International Symposium on Software Testing and Analysis","author":"Wu Zhiyong","year":"2022","unstructured":"Zhiyong Wu, Jie Liang, Mingzhe Wang, Chijin Zhou, and Yu Jiang. 2022. Unicorn: Detect runtime errors in time-series databases with hybrid input synthesis. In Proceedings of the International Symposium on Software Testing and Analysis."},{"key":"e_1_3_3_109_2","first-page":"1","volume-title":"Proceedings of the Workshop on Testing Database Systems","author":"Yan Jiaqi","year":"2018","unstructured":"Jiaqi Yan, Qiuye Jin, Shrainik Jain, Stratis D. Viglas, and Allison Lee. 2018. Snowtrail: Testing with production queries on a cloud database. In Proceedings of the Workshop on Testing Database Systems. ACM, 1\u20136."},{"key":"e_1_3_3_110_2","first-page":"136","volume-title":"International Conference on Software Engineering: Companion Proceedings","author":"Yang Rui","year":"2023","unstructured":"Rui Yang, Yingying Zheng, Lei Tang, Wensheng Dou, Wei Wang, and Jun Wei. 2023. Randomized differential testing of RDF stores. In International Conference on Software Engineering: Companion Proceedings. IEEE, 136\u2013140."},{"key":"e_1_3_3_111_2","first-page":"901","volume-title":"Proceedings of the USENIX Security Symposium","author":"Yang Yupeng","year":"2024","unstructured":"Yupeng Yang, Yongheng Chen, Rui Zhong, Jizhou Chen, and Wenke Lee. 2024. Towards generic database management system fuzzing. In Proceedings of the USENIX Security Symposium. 901\u2013918."},{"key":"e_1_3_3_112_2","doi-asserted-by":"crossref","unstructured":"Zijing Yin Si Liu and David Basin. 2025. Testing graph databases with synthesized queries. Proceedings of the ACM on Management of Data 3 4 (2025) 1\u201326.","DOI":"10.1145\/3749186"},{"key":"e_1_3_3_113_2","doi-asserted-by":"crossref","unstructured":"Chi Zhang and Manuel Rigger. 2025. Constant optimization driven database system testing. Proceedings of the ACM on Management of Data 3 1 (2025) 1\u201324.","DOI":"10.1145\/3709674"},{"key":"e_1_3_3_114_2","unstructured":"Yushan Zhang Peisen Yao Rongxin Wu and Charles Zhang. 2021. Duplicate-sensitivity guided transformation synthesis for DBMS correctness bug detection. arXiv:2107.03660. Retrieved from https:\/\/arxiv.org\/abs\/2107.03660"},{"key":"e_1_3_3_115_2","doi-asserted-by":"crossref","first-page":"1695","DOI":"10.1145\/3650212.3680392","volume-title":"Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis","author":"Zheng Yingying","year":"2024","unstructured":"Yingying Zheng, Wensheng Dou, Lei Tang, Ziyu Cui, Yu Gao, Jiansen Song, Liang Xu, Jiaxin Zhu, Wei Wang, Jun Wei, et\u00a0al. 2024. Testing gremlin-based graph database systems via query disassembling. In Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis. 1695\u20131707."},{"key":"e_1_3_3_116_2","first-page":"302","volume-title":"Proceedings of the International Symposium on Software Testing and Analysis","author":"Zheng Yingying","year":"2022","unstructured":"Yingying Zheng, Wensheng Dou, Yicheng Wang, Zheng Qin, Lei Tang, Yu Gao, Dong Wang, Wei Wang, and Jun Wei. 2022. Finding bugs in gremlin-based graph database systems via randomized differential testing. In Proceedings of the International Symposium on Software Testing and Analysis. ACM, 302\u2013313."},{"key":"e_1_3_3_117_2","first-page":"955","volume-title":"Proceedings of the Conference on Computer and Communications Security","author":"Zhong Rui","year":"2020","unstructured":"Rui Zhong, Yongheng Chen, Hong Hu, Hangfan Zhang, Wenke Lee, and Dinghao Wu. 2020. SQUIRREL: Testing Database Management Systems with Language Validity and Coverage Feedback. In Proceedings of the Conference on Computer and Communications Security. ACM, 955\u2013970."},{"key":"e_1_3_3_118_2","unstructured":"Suyang Zhong and Manuel Rigger. 2025. Testing Database Systems with Large Language Model Synthesized Fragments. arXiv:2505.02012. Retrieved from https:\/\/arxiv.org\/abs\/2505.02012"},{"key":"e_1_3_3_119_2","first-page":"69","volume-title":"Proceedings of the International Workshop on Metamorphic Testing","author":"Zhou Zenghui","year":"2021","unstructured":"Zenghui Zhou, Zheng Zheng, Tsong Yueh Chen, Jinyi Zhou, and Kun Qiu. 2021. Follow-up test cases are better than source test cases in metamorphic testing: A preliminary study. In Proceedings of the International Workshop on Metamorphic Testing. IEEE, 69\u201374."},{"key":"e_1_3_3_120_2","doi-asserted-by":"crossref","unstructured":"Zeyang Zhuang Penghui Li Pingchuan Ma Wei Meng and Shuai Wang. 2023. Testing graph database systems via graph-aware metamorphic relations. Proceedings of the VLDB Endowment 17 4 (2023) 836\u2013848.","DOI":"10.14778\/3636218.3636236"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3799227","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,18]],"date-time":"2026-04-18T11:49:04Z","timestamp":1776512944000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3799227"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,18]]},"references-count":119,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2026,7,31]]}},"alternative-id":["10.1145\/3799227"],"URL":"https:\/\/doi.org\/10.1145\/3799227","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,18]]},"assertion":[{"value":"2023-11-17","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-02-10","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}