{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T11:34:42Z","timestamp":1791459282975,"version":"4.3.4"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"ISSTA","license":[{"start":{"date-parts":[[2025,6,22]],"date-time":"2025-06-22T00:00:00Z","timestamp":1750550400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62272400"],"award-info":[{"award-number":["62272400"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2025,6,22]]},"abstract":"<jats:p>\n                    Metamorphic testing is a widely used method to detect logical bugs in Database Management Systems (DBMSs), referred to herein as\n                    <jats:bold>MOLT<\/jats:bold>\n                    (\n                    <jats:bold>M<\/jats:bold>\n                    etamorphic-\n                    <jats:bold>O<\/jats:bold>\n                    racle based\n                    <jats:bold>L<\/jats:bold>\n                    ogical Bug Detection\n                    <jats:bold>T<\/jats:bold>\n                    echnique). This technique involves constructing SQL statement pairs, including original and mutated queries, and assessing whether the execution results conform to predefined metamorphic relations to detect logical bugs. However, current MOLTs rely heavily on specific DBMS grammar to generate valid SQL statement pairs, which makes it challenging to adapt these techniques to various DBMSs with different grammatical structures. As a result, only a few popular DBMSs, such as PostgreSQL, MySQL, and MariaDB, are supported by existing MOLTs, with extensive manual effort required to expand to other DBMSs. Given that many DBMSs remain inadequately tested, there is a pressing need for a method that enables effortless extension of MOLTs across diverse DBMSs.\n                  <\/jats:p>\n                  <jats:p>In this paper, we propose QTRAN, a novel LLM-powered approach that automatically extends existing MOLTs to various DBMSs. Our key insight is to translate SQL statement pairs to target DBMSs for metamorphic testing from existing MOLTs using LLMs. To address the challenges of LLMs\u2019 limited understanding of dialect differences and metamorphic mechanisms, we propose a two-phase approach comprising the transfer and mutation phases. QTRAN tackles these challenges by drawing inspiration from the developer\u2019s process of creating a MOLT, which includes understanding the grammar of the target DBMS to generate original queries and employing a mutator for customized mutations. The transfer phase is designed to identify potential dialects and leverage information from SQL documents to enhance query retrieval, enabling LLMs to translate original queries across different DBMSs accurately. During the mutation phase, we gather SQL statement pairs from existing MOLTs to fine-tune the pretrained model, tailoring it specifically for mutation tasks. Then we employ the customized LLM to mutate the translated original queries, preserving the defined relationships necessary for metamorphic testing.<\/jats:p>\n                  <jats:p>We implement our approach as a tool and apply it to extend four state-of-the-art MOLTs for eight DBMSs: MySQL, MariaDB, TiDB, PostgreSQL, SQLite, MonetDB, DuckDB, and ClickHouse. The evaluation results show that over 99% of the SQL statement pairs transferred by QTRAN satisfy the metamorphic relations required for testing. Furthermore, we have detected 24 logical bugs among these DBMSs, with 16 confirmed as unique and previously unknown bugs. We believe that the generality of QTRAN can significantly enhance the reliability of DBMSs.<\/jats:p>","DOI":"10.1145\/3728908","type":"journal-article","created":{"date-parts":[[2025,6,22]],"date-time":"2025-06-22T10:53:21Z","timestamp":1750589601000},"page":"731-752","source":"Crossref","is-referenced-by-count":8,"title":["QTRAN: Extending Metamorphic-Oracle Based Logical Bug Detection Techniques for Multiple-DBMS Dialect Support"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-9895-8360","authenticated-orcid":false,"given":"Li","family":"Lin","sequence":"first","affiliation":[{"name":"Xiamen University, School of Informatics, XiaMen, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-7759-9431","authenticated-orcid":false,"given":"Qinglin","family":"Zhu","sequence":"additional","affiliation":[{"name":"Xiamen University, School of Informatics, XiaMen, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-8604-0996","authenticated-orcid":false,"given":"Hongqiao","family":"Chen","sequence":"additional","affiliation":[{"name":"Xiamen University, School of Informatics, XiaMen, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-7628-2208","authenticated-orcid":false,"given":"Zhuangda","family":"Wang","sequence":"additional","affiliation":[{"name":"Xiamen University, School of Informatics, XiaMen, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4648-3795","authenticated-orcid":false,"given":"Rongxin","family":"Wu","sequence":"additional","affiliation":[{"name":"Xiamen University, School of Informatics, XiaMen, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-9700-0542","authenticated-orcid":false,"given":"Xiaoheng","family":"Xie","sequence":"additional","affiliation":[{"name":"Ant Group, ShenZhen, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,6,22]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"2015. SQLsmith. https:\/\/github.com\/anse1\/sqlsmith. Accessed: 2024-10-24."},{"key":"e_1_3_1_3_2","unstructured":"2022. ANSI Standard. https:\/\/www.ansi.org\/. Accessed: 2024-10-24."},{"key":"e_1_3_1_4_2","unstructured":"2023. SQLancer. https:\/\/github.com\/sqlancer\/sqlancer. Accessed: 2024-10-24."},{"key":"e_1_3_1_5_2","unstructured":"2024. C to Go Translator. https:\/\/github.com\/gotranspile\/cxgo. Accessed: 2024-10-13."},{"key":"e_1_3_1_6_2","unstructured":"2024. C2Rust Transpiler. https:\/\/github.com\/immunant\/c2rust. Accessed: 2024-10-13."},{"key":"e_1_3_1_7_2","unstructured":"2024. Java 2 CSharp Translator for Eclipse. https:\/\/sourceforge.net\/projects\/j2cstranslator\/. Accessed: 2024-10-13."},{"key":"e_1_3_1_8_2","unstructured":"2024. Sharpen - Automated Java to C# Conversion. https:\/\/github.com\/mono\/sharpen. Accessed: 2024-10-13."},{"key":"e_1_3_1_9_2","unstructured":"Atul Adya. 1999. Weak consistency: a generalized theory and optimistic implementations for distributed transactions. (1999)."},{"key":"e_1_3_1_10_2","unstructured":"Hardik Bati Leo Giakoumakis Steve Herbert and Aleksandras Surna. 2007. A genetic approach for random testing of database systems. In Proceedings of the 33rd international conference on Very large data bases. 1243\u20131251."},{"key":"e_1_3_1_11_2","unstructured":"Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)."},{"issue":"240","key":"e_1_3_1_12_2","first-page":"1","article-title":"Palm: Scaling language modeling with pathways","volume":"24","author":"Chowdhery Aakanksha","year":"2023","unstructured":"Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research 24, 240 (2023), 1\u2013113.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_1_13_2","unstructured":"Karl Cobbe Vineet Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun Lukasz Kaiser Matthias Plappert Jerry Tworek Jacob Hilton Reiichiro Nakano et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 (2021)."},{"key":"e_1_3_1_14_2","doi-asserted-by":"crossref","unstructured":"Ziyu Cui Wensheng Dou Qianwang Dai Jiansen Song Wei Wang Jun Wei and Dan Ye. 2022. Differentially testing database transactions for fun and profit. In Proceedings of the 37th IEEE\/ACM International Conference on Automated Software Engineering. 1\u201312.","DOI":"10.1145\/3551349.3556924"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2024.107468"},{"key":"e_1_3_1_16_2","unstructured":"Victor Dantas. 2023. Large Language Model Powered Test Case Generation for Software Applications. (2023)."},{"key":"e_1_3_1_17_2","doi-asserted-by":"crossref","unstructured":"Yinlin Deng Chunqiu Steven Xia Haoran Peng Chenyuan Yang and Lingming Zhang. 2023. Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models. In Proceedings of the 32nd ACM SIGSOFT international symposium on software testing and analysis. 423\u2013435.","DOI":"10.1145\/3597926.3598067"},{"key":"e_1_3_1_18_2","doi-asserted-by":"crossref","unstructured":"Yinlin Deng Chunqiu Steven Xia Chenyuan Yang Shizhuo Dylan Zhang Shujing Yang and Lingming Zhang. 2023. Large language models are edge-case fuzzers: Testing deep learning libraries via fuzzgpt. arXiv preprint arXiv:2304.02014 (2023).","DOI":"10.1145\/3597926.3598067"},{"key":"e_1_3_1_19_2","unstructured":"Lukas Eder. 2024. JOOQ: Fluent API for Typesafe SQL Query Construction and Translation. https:\/\/www.jooq.org\/. Accessed: 2024-12-29."},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/290593.290605"},{"key":"e_1_3_1_21_2","unstructured":"Zongyin Hao Quanfeng Huang Chengpeng Wang Jianfeng Wang Yushan Zhang Rongxin Wu and Charles Zhang. 2023. Pinolo: Detecting Logical Bugs in Database Management Systems with Approximate Query Synthesis. In 2023 USENIX Annual Technical Conference (USENIX ATC 23). 345\u2013358."},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1978.231514"},{"key":"e_1_3_1_23_2","unstructured":"Ali Reza Ibrahimzada Yang Chen Ryan Rong and Reyhaneh Jabbarvand. 2023. Automated bug generation in the era of large language models. arXiv preprint arXiv:2310.02407 (2023)."},{"key":"e_1_3_1_24_2","doi-asserted-by":"crossref","unstructured":"Yu Jiang Jie Liang Fuchen Ma Yuanliang Chen Chijin Zhou Yuheng Shen Zhiyong Wu Jingzhou Fu Mingzhe Wang Shanshan Li et al. 2024. When fuzzing meets llms: Challenges and opportunities. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering. 492\u2013496.","DOI":"10.1145\/3663529.3663784"},{"key":"e_1_3_1_25_2","unstructured":"Zu-Ming Jiang Jia-Ju Bai and Zhendong Su. 2023. {DynSQL}: Stateful Fuzzing for Database Management Systems with Complex and Valid {SQL} Query Generation. In 32nd USENIX Security Symposium (USENIX Security 23). 4949\u20134965."},{"key":"e_1_3_1_26_2","unstructured":"Zu-Ming Jiang and Zhendong Su. 2024. Detecting Logic Bugs in Database Engines via Equivalent Expression Transformation. In 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). 821\u2013835."},{"key":"e_1_3_1_27_2","doi-asserted-by":"crossref","first-page":"57","DOI":"10.14778\/3357377.3357382","article-title":"Apollo: Automatic detection and diagnosis of performance regressions in database systems","volume":"1","author":"Jung Jinho","year":"2019","unstructured":"Jinho Jung, Hong Hu, Joy Arulraj, Taesoo Kim, and Woonhak Kang. 2019. Apollo: Automatic detection and diagnosis of performance regressions in database systems. Proceedings of the VLDB Endowment 13, 1 (2019), 57\u201370.","journal-title":"Proceedings of the VLDB Endowment 13"},{"key":"e_1_3_1_28_2","unstructured":"Zhenwen Li and Tao Xie. 2024. Using LLM to select the right SQL Query from candidates. arXiv preprint arXiv:2401.02115 (2024)."},{"key":"e_1_3_1_29_2","unstructured":"Yu Liang Song Liu and Hong Hu. 2022. Detecting logical bugs of {DBMS} with coverage-based guidance. In 31st USENIX Security Symposium (USENIX Security 22). 4309\u20134326."},{"key":"e_1_3_1_30_2","first-page":"4309","volume-title":"31st USENIX Security Symposium (USENIX Security 22)","author":"Liang Yu","year":"2022","unstructured":"Yu Liang, Song Liu, and Hong Hu. 2022. Detecting Logical Bugs of DBMS with Coverage-based Guidance. In 31st USENIX Security Symposium (USENIX Security 22). USENIX Association, Boston, MA, 4309\u20134326. https:\/\/www.usenix.org\/conference\/usenixsecurity22\/presentation\/liang"},{"key":"e_1_3_1_31_2","doi-asserted-by":"crossref","unstructured":"Li Lin Zongyin Hao Chengpeng Wang Zhuangda Wang Rongxin Wu and Gang Fan. 2024. SQLess: Dialect-Agnostic SQL Query Simplification. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 743\u2013754.","DOI":"10.1145\/3650212.3680317"},{"key":"e_1_3_1_32_2","unstructured":"Yunlong Lyu Yuxuan Xie Peng Chen and Hao Chen. 2023. Prompt Fuzzing for Fuzz Driver Generation. arXiv preprint arXiv:2312.17677 (2023)."},{"key":"e_1_3_1_33_2","unstructured":"Toby Mao. 2024. SQLGlot: Python SQL Parser and Translator. https:\/\/github.com\/tobymao\/sqlglot. Accessed: 2024-12-29."},{"key":"e_1_3_1_34_2","unstructured":"OpenAI. 2024. GPT-4 Turbo Fine-Tuning. https:\/\/openai.com\/index\/gpt-4o-fine-tuning\/. Accessed: 2024-10-30."},{"key":"e_1_3_1_35_2","unstructured":"Terence Parr. 2013. The Definitive ANTLR 4 Reference. Pragmatic Bookshelf Chapter 9."},{"key":"e_1_3_1_36_2","unstructured":"PingCap. 2023. go randgen. https:\/\/github.com\/pingcap\/go-randgen. [Online; accessed 2-October-2024]."},{"key":"e_1_3_1_37_2","unstructured":"Alec Radford. 2018. Improving language understanding by generative pre-training. (2018)."},{"key":"e_1_3_1_38_2","doi-asserted-by":"crossref","unstructured":"Manuel Rigger and Zhendong Su. 2020. Detecting optimization bugs in database engines via non-optimizing reference engine construction. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1140\u20131152.","DOI":"10.1145\/3368089.3409710"},{"key":"e_1_3_1_39_2","doi-asserted-by":"crossref","unstructured":"Manuel Rigger and Zhendong Su. 2020. Finding bugs in database systems via query partitioning. Proceedings of the ACM on Programming Languages 4 OOPSLA (2020) 1\u201330.","DOI":"10.1145\/3428279"},{"key":"e_1_3_1_40_2","unstructured":"Manuel Rigger and Zhendong Su. 2020. Testing database engines via pivoted query synthesis. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). 667\u2013682."},{"key":"e_1_3_1_41_2","doi-asserted-by":"crossref","unstructured":"Manuel Rigger and Zhendong Su. 2022. Intramorphic testing: A new approach to the test oracle problem. In Proceedings of the 2022 ACM SIGPLAN International Symposium on New Ideas New Paradigms and Reflections on Programming and Software. 128\u2013136.","DOI":"10.1145\/3563835.3567662"},{"key":"e_1_3_1_42_2","first-page":"31693","volume-title":"International Conference on Machine Learning","author":"Shrivastava Disha","year":"2023","unstructured":"Disha Shrivastava, Hugo Larochelle, and Daniel Tarlow. 2023. Repository-level prompt generation for large language models of code. In International Conference on Machine Learning. PMLR, 31693\u201331715."},{"key":"e_1_3_1_43_2","first-page":"618","volume-title":"VLDB","author":"Slutz Donald R","year":"1998","unstructured":"Donald R Slutz. 1998. Massive stochastic testing of SQL. In VLDB, Vol. 98. Citeseer, 618\u2013622."},{"key":"e_1_3_1_44_2","unstructured":"Solid IT. 2024. DB-Engines Ranking. https:\/\/db-engines.com\/en\/ranking. Accessed: September 3 2024."},{"key":"e_1_3_1_45_2","doi-asserted-by":"crossref","unstructured":"Jiansen Song Wensheng Dou Ziyu Cui Qianwang Dai Wei Wang Jun Wei Hua Zhong and Tao Huang. 2023. Testing database systems via differential query execution. In 2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE). IEEE 2072\u20132084.","DOI":"10.1109\/ICSE48619.2023.00175"},{"key":"e_1_3_1_46_2","unstructured":"QTRAN Team. 2024. QTRANBugReports. https:\/\/github.com\/QTRANll\/QTRAN-BUGLIST."},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.52202\/068431-1800"},{"key":"e_1_3_1_48_2","doi-asserted-by":"crossref","unstructured":"Chunqiu Steven Xia Matteo Paltenghi Jia Le Tian Michael Pradel and Lingming Zhang. 2024. Fuzz4all: Universal fuzzing with large language models. In Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering. 1\u201313.","DOI":"10.1145\/3597503.3639121"},{"key":"e_1_3_1_49_2","unstructured":"Congying Xu Songqiang Chen Jiarong Wu Shing-Chi Cheung Valerio Terragni Hengcheng Zhu and Jialun Cao. 2024. MR-Adopt: Automatic Deduction of Input Transformation Function for Metamorphic Testing. arXiv preprint arXiv:2408.15815 (2024)."},{"key":"e_1_3_1_50_2","doi-asserted-by":"crossref","unstructured":"Guixin Ye Zhanyong Tang Shin Hwei Tan Songfang Huang Dingyi Fang Xiaoyang Sun Lizhong Bian Haibo Wang and Zheng Wang. 2021. Automated conformance testing for JavaScript engines via deep compiler fuzzing. In Proceedings of the 42nd ACM SIGPLAN international conference on programming language design and implementation. 435\u2013450.","DOI":"10.1145\/3453483.3454054"},{"key":"e_1_3_1_51_2","unstructured":"Cen Zhang Mingqiang Bai Yaowen Zheng Yeting Li Wei Ma Xiaofei Xie Yuekang Li Limin Sun and Yang Liu. 2023. Understanding large language model based fuzz driver generation. arXiv e-priynts (2023) arXiv-2307."},{"key":"e_1_3_1_52_2","unstructured":"Qinggang Zhang Junnan Dong Hao Chen Wentao Li Feiran Huang and Xiao Huang. 2024. Structure guided large language model for sql generation. arXiv preprint arXiv:2402.13284 (2024)."},{"key":"e_1_3_1_53_2","first-page":"1096","article-title":"Database meets artificial intelligence: A survey","volume":"3","author":"Zhou Xuanhe","year":"2020","unstructured":"Xuanhe Zhou, Chengliang Chai, Guoliang Li, and Ji Sun. 2020. Database meets artificial intelligence: A survey. IEEE Transactions on Knowledge and Data Engineering 34, 3 (2020), 1096\u20131116.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3728908","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T13:55:40Z","timestamp":1787493340000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3728908"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,22]]},"references-count":52,"journal-issue":{"issue":"ISSTA","published-print":{"date-parts":[[2025,6,22]]}},"alternative-id":["10.1145\/3728908"],"URL":"https:\/\/doi.org\/10.1145\/3728908","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,22]]}}}