{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T16:41:02Z","timestamp":1774456862478,"version":"3.50.1"},"reference-count":76,"publisher":"Association for Computing Machinery (ACM)","issue":"4","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2024,12]]},"abstract":"<jats:p>\n            Translating natural language questions into SQL queries (NL2SQL) is a challenging task of great practical importance. Prior work has extensively studied how to address NL2SQL using Large Language Models (LLMs) with solutions ranging from careful prompt engineering, to fine-tuning existing LLMs, or even training custom models. However, a remaining challenging problem in NL2SQL is the inherent ambiguity in the natural language questions asked by users. In this paper, we introduce Sphinteract, a framework designed to assist LLMs in generating high-quality SQL answers that accurately reflect the user intent. Our key insight to resolve ambiguity is to take into account minimal user feedback interactively. We introduce the\n            <jats:italic>Summarize, Review, Ask<\/jats:italic>\n            (SRA) paradigm, which guides LLMs in identifying ambiguities in NL2SQL tasks and generates targeted questions for the user to answer. We propose three different methods of how to process user feedback and generate SQL queries based on user input. Our experiments on the challenging KaggleDBQA and BIRD benchmarks demonstrate that by means of asking clarification questions to the user, LLMs can efficiently incorporate the feedback, resulting in accuracy improvements of up to 42%.\n          <\/jats:p>","DOI":"10.14778\/3717755.3717772","type":"journal-article","created":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T15:51:49Z","timestamp":1747756309000},"page":"1145-1158","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Sphinteract: Resolving Ambiguities in NL2SQL through User Interaction"],"prefix":"10.14778","volume":"18","author":[{"given":"Fuheng","family":"Zhao","sequence":"first","affiliation":[{"name":"UC Santa Barbara"}]},{"given":"Shaleen","family":"Deep","sequence":"additional","affiliation":[{"name":"Microsoft"}]},{"given":"Fotis","family":"Psallidas","sequence":"additional","affiliation":[{"name":"Microsoft"}]},{"given":"Avrilia","family":"Floratou","sequence":"additional","affiliation":[{"name":"Microsoft"}]},{"given":"Divyakant","family":"Agrawal","sequence":"additional","affiliation":[{"name":"UC Santa Barbara"}]},{"given":"Amr El","family":"Abbadi","sequence":"additional","affiliation":[{"name":"UC Santa Barbara"}]}],"member":"320","published-online":{"date-parts":[[2025,5,20]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"[n.d.]. sphinteract_full.pdf. https:\/\/github.com\/ZhaoFuheng\/Sphinteract"},{"key":"e_1_2_1_2_1","volume-title":"Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.","author":"Achiam Josh","year":"2023","unstructured":"Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)."},{"key":"e_1_2_1_3_1","volume-title":"International Conference on Machine Learning. PMLR, 337\u2013371","author":"Aher Gati V","year":"2023","unstructured":"Gati V Aher, Rosa I Arriaga, and Adam Tauman Kalai. 2023. Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning. PMLR, 337\u2013371."},{"key":"e_1_2_1_4_1","volume-title":"Increasing the LLM Accuracy for Question Answering: Ontologies to the Rescue! arXiv preprint arXiv:2405.11706","author":"Allemang Dean","year":"2024","unstructured":"Dean Allemang and Juan Sequeda. 2024. Increasing the LLM Accuracy for Question Answering: Ontologies to the Rescue! arXiv preprint arXiv:2405.11706 (2024)."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3442188.3445922"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i16.29720"},{"key":"e_1_2_1_7_1","volume-title":"Using gpt for market research. Available at SSRN 4395751","author":"Brand James","year":"2023","unstructured":"James Brand, Ayelet Israeli, and Donald Ngwe. 2023. Using gpt for market research. Available at SSRN 4395751 (2023)."},{"key":"e_1_2_1_8_1","unstructured":"Christoph Brandt Nadja Geisler and Carsten Binnig. 2020. Towards robust and transparent natural language interfaces for databases. (2020)."},{"key":"e_1_2_1_9_1","unstructured":"Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020) 1877\u20131901."},{"key":"e_1_2_1_10_1","unstructured":"Xinyun Chen Maxwell Lin Nathanael Schaerli and Denny Zhou. 2023. Teaching Large Language Models to Self-Debug. (2023)."},{"key":"e_1_2_1_11_1","volume-title":"Seed: Simple, efficient, and effective data management via large language models. arXiv preprint arXiv:2310.00749","author":"Hen Zui","year":"2023","unstructured":"Zui CHen, Lei Cao, Sam Madden, Ju Fan, Nan Tang, Zihui Gu, Zeyuan Shang, Chunwei Liu, Michael Cafarella, and Tim Kraska. 2023. Seed: Simple, efficient, and effective data management via large language models. arXiv preprint arXiv:2310.00749 (2023)."},{"key":"e_1_2_1_12_1","volume-title":"Oreen Yousuf, Salomey Osei, Abigail Oppong, Iyanuoluwa Shode, Oluwabusayo Olufunke Awoyomi, and Chris Emezue.","author":"Dossou Bonaventure FP","year":"2022","unstructured":"Bonaventure FP Dossou, Atnafu Lambebo Tonja, Oreen Yousuf, Salomey Osei, Abigail Oppong, Iyanuoluwa Shode, Oluwabusayo Olufunke Awoyomi, and Chris Emezue. 2022. AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languages. (2022), 52\u201364."},{"key":"e_1_2_1_13_1","volume-title":"From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130","author":"Edge Darren","year":"2024","unstructured":"Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. 2024. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130 (2024)."},{"key":"e_1_2_1_14_1","doi-asserted-by":"crossref","unstructured":"Ahmed Elgohary Christopher Meek Matthew Richardson Adam Fourney Gonzalo Ramos and Ahmed Hassan. 2021. NL-EDIT: Correcting Semantic Parse Errors through Natural Language Interaction. (2021) 5599\u20135610.","DOI":"10.18653\/v1\/2021.naacl-main.444"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.14778\/3611479.3611527"},{"key":"e_1_2_1_16_1","volume-title":"Conference on Innovative Data Systems Research. https:\/\/api.semanticscholar.org\/CorpusID:266729311","author":"Floratou Avrilia","year":"2024","unstructured":"Avrilia Floratou, Fotis Psallidas, Fuheng Zhao, Shaleen Deep, Gunther Hagleither, Wangda Tan, Joyce Cahoon, Rana Alotaibi, Jordan Henkel, Abhik Singla, Alex Van Grootel, Brandon Chow, Kai Deng, Katherine Lin, Marcos Campos, K. Venkatesh Emani, Vivek Pandit, Victor Shnayder, Wenjing Wang, and Carlo Curino. 2024. NL2SQL is a solved problem... Not!. In Conference on Innovative Data Systems Research. https:\/\/api.semanticscholar.org\/CorpusID:266729311"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.14778\/3641204.3641221"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.14778\/3641204.3641221"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3452836"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.3115\/974194.974201"},{"key":"e_1_2_1_21_1","unstructured":"Jie Huang Xinyun Chen Swaroop Mishra Huaixiu Steven Zheng Adams Wei Yu Xinying Song and Denny Zhou. [n.d.]. Large Language Models Cannot Self-Correct Reasoning Yet. ([n. d.])."},{"key":"e_1_2_1_22_1","volume-title":"NeurIPS 2023 Second Table Representation Learning Workshop.","author":"Huang Zezhou","unstructured":"Zezhou Huang, Pavan Kalyan Damalapati, and Eugene Wu. [n.d.]. Data Ambiguity Strikes Back: How Documentation Improves GPT's Text-to-SQL. In NeurIPS 2023 Second Table Representation Learning Workshop."},{"key":"e_1_2_1_23_1","volume-title":"Improving text-to-sql with schema dependency learning. arXiv preprint arXiv:2103.04399","author":"Hui Binyuan","year":"2021","unstructured":"Binyuan Hui, Xiang Shi, Ruiying Geng, Binhua Li, Yongbin Li, Jian Sun, and Xiaodan Zhu. 2021. Improving text-to-sql with schema dependency learning. arXiv preprint arXiv:2103.04399 (2021)."},{"key":"e_1_2_1_24_1","unstructured":"Srinivasan Iyer Ioannis Konstas Alvin Cheung Jayant Krishnamurthy and Luke Zettlemoyer. 2017. Learning a Neural Semantic Parser from User Feedback. (2017) 963\u2013973."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.2307\/2981421"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-022-00776-8"},{"key":"e_1_2_1_27_1","volume-title":"Bert: Pre-training of deep bidirectional transformers for language understanding. 1, 2","author":"Ming-Wei Chang Jacob Devlin","year":"2019","unstructured":"Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. 1, 2 (2019)."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.14778\/3401960.3401970"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2018.2831214"},{"key":"e_1_2_1_30_1","unstructured":"Chia-Hsuan Lee Oleksandr Polozov and Matthew Richardson. 2021. KaggleDBQA: Realistic Evaluation of Text-to-SQL Parsers. (2021) 2261\u20132273."},{"key":"e_1_2_1_31_1","first-page":"15589","article-title":"Ehrsql: A practical text-to-sql benchmark for electronic health records","volume":"35","author":"Lee Gyubok","year":"2022","unstructured":"Gyubok Lee, Hyeonji Hwang, Seongsu Bae, Yeonsu Kwon, Woncheol Shin, Seongjun Yang, Minjoon Seo, Jong-Yeup Kim, and Edward Choi. 2022. Ehrsql: A practical text-to-sql benchmark for electronic health records. Advances in Neural Information Processing Systems 35 (2022), 15589\u201315601.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_32_1","volume-title":"The Dawn of Natural Language to SQL: Are We Fully Ready? arXiv preprint arXiv:2406.01265","author":"Li Boyan","year":"2024","unstructured":"Boyan Li, Yuyu Luo, Chengliang Chai, Guoliang Li, and Nan Tang. 2024. The Dawn of Natural Language to SQL: Are We Fully Ready? arXiv preprint arXiv:2406.01265 (2024)."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.14778\/2735461.2735468"},{"key":"e_1_2_1_34_1","unstructured":"Jinyang Li Binyuan Hui Ge Qu Jiaxi Yang Binhua Li Bowen Li Bailin Wang Bowen Qin Ruiying Geng Nan Huo et al. 2024. Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls. Advances in Neural Information Processing Systems 36 (2024)."},{"key":"e_1_2_1_35_1","volume-title":"A technique for the measurement of attitudes. Archives of Psychology","author":"Likert Rensis","year":"1932","unstructured":"Rensis Likert. 1932. A technique for the measurement of attitudes. Archives of Psychology (1932)."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.14778\/3685800.3685916"},{"key":"e_1_2_1_37_1","volume-title":"https:\/\/platform.openai.com\/examples\/default-sql-translate. Accessed","author":"Platform AI.","year":"2024","unstructured":"OpenAI. 2024. OpenAI Platform. https:\/\/platform.openai.com\/examples\/default-sql-translate. Accessed: 30 April 2024."},{"key":"e_1_2_1_38_1","volume-title":"Retrieved","year":"2024","unstructured":"openai.com. 2024. GPT-4 Turbo. Retrieved October 31, 2024 from https:\/\/platform.openai.com\/docs\/models#gpt-4-turbo-and-gpt-4"},{"key":"e_1_2_1_39_1","volume-title":"Retrieved","year":"2024","unstructured":"openai.com. 2024. GPT-4o. Retrieved October 31, 2024 from https:\/\/platform.openai.com\/docs\/models\/gpt-4o"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3586183.3606763"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.3115\/1220355.1220376"},{"key":"e_1_2_1_42_1","volume-title":"Din-sql: Decomposed in-context learning of text-to-sql with self-correction. Advances in Neural Information Processing Systems 36","author":"Pourreza Mohammadreza","year":"2024","unstructured":"Mohammadreza Pourreza and Davood Rafiei. 2024. Din-sql: Decomposed in-context learning of text-to-sql with self-correction. Advances in Neural Information Processing Systems 36 (2024)."},{"key":"e_1_2_1_43_1","doi-asserted-by":"crossref","unstructured":"Reid Pryzant Dan Iter Jerry Li Yin Lee Chenguang Zhu and Michael Zeng. 2023. Automatic Prompt Optimization with \"Gradient Descent\" and Beam Search. (2023) 7957\u20137968.","DOI":"10.18653\/v1\/2023.emnlp-main.494"},{"key":"e_1_2_1_44_1","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research 21, 140 (2020), 1\u201367.","journal-title":"Journal of machine learning research"},{"key":"e_1_2_1_45_1","volume-title":"Christopher Michael Rytting, and David Wingate","author":"Robinson Joshua","year":"2023","unstructured":"Joshua Robinson, Christopher Michael Rytting, and David Wingate. 2023. Leveraging Large Language Models for Multiple Choice Question Answering. arXiv:2210.12353 [cs.CL]"},{"key":"e_1_2_1_46_1","volume-title":"Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al.","author":"Romera-Paredes Bernardino","year":"2024","unstructured":"Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. 2024. Mathematical discoveries from program search with large language models. Nature 625, 7995 (2024), 468\u2013475."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.14778\/2994509.2994536"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3661304.3661901"},{"key":"e_1_2_1_49_1","volume-title":"A Survey on Employing Large Language Models for Text-to-SQL Tasks. arXiv preprint arXiv:2407.15186","author":"Shi Liang","year":"2024","unstructured":"Liang Shi, Zhengju Tang, and Zhi Yang. 2024. A Survey on Employing Large Language Models for Text-to-SQL Tasks. arXiv preprint arXiv:2407.15186 (2024)."},{"key":"e_1_2_1_50_1","volume-title":"Chess: Contextual harnessing for efficient sql synthesis. arXiv preprint arXiv:2405.16755","author":"Talaei Shayan","year":"2024","unstructured":"Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. 2024. Chess: Contextual harnessing for efficient sql synthesis. arXiv preprint arXiv:2405.16755 (2024)."},{"key":"e_1_2_1_51_1","doi-asserted-by":"crossref","unstructured":"Yuan Tian Zheng Zhang Zheng Ning Toby Li Jonathan K Kummerfeld and Tianyi Zhang. 2023. Interactive text-to-SQL generation via editable step-by-step explanations. (2023) 16149\u201316166.","DOI":"10.18653\/v1\/2023.emnlp-main.1004"},{"key":"e_1_2_1_52_1","volume-title":"Attention is all you need. Advances in neural information processing systems 30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)."},{"key":"e_1_2_1_53_1","doi-asserted-by":"crossref","unstructured":"Bailin Wang Richard Shin Xiaodong Liu Oleksandr Polozov and Matthew Richardson. 2020. RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers. (2020) 7567\u20137578.","DOI":"10.18653\/v1\/2020.acl-main.677"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3534678.3539305"},{"key":"e_1_2_1_55_1","volume-title":"Vincent Perot, Zifeng Wang, Lesly Miculicich, Yasuhisa Fujii, Jingbo Shang, Chen-Yu Lee, et al.","author":"Wang Zilong","year":"2024","unstructured":"Zilong Wang, Hao Zhang, Chun-Liang Li, Julian Martin Eisenschlos, Vincent Perot, Zifeng Wang, Lesly Miculicich, Yasuhisa Fujii, Jingbo Shang, Chen-Yu Lee, et al. 2024. Chain-of-table: Evolving tables in the reasoning chain for table understanding. arXiv preprint arXiv:2401.04398 (2024)."},{"key":"e_1_2_1_56_1","volume-title":"An efficient easily adaptable system for interpreting natural language queries. American journal of computational linguistics 8, 3-4","author":"Warren David HD","year":"1982","unstructured":"David HD Warren and Fernando CN Pereira. 1982. An efficient easily adaptable system for interpreting natural language queries. American journal of computational linguistics 8, 3-4 (1982), 110\u2013122."},{"key":"e_1_2_1_57_1","volume-title":"Denny Zhou, et al.","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824\u201324837."},{"key":"e_1_2_1_58_1","volume-title":"The Twelfth International Conference on Learning Representations.","author":"West Peter","year":"2023","unstructured":"Peter West, Ximing Lu, Nouha Dziri, Faeze Brahman, Linjie Li, Jena D Hwang, Liwei Jiang, Jillian Fisher, Abhilasha Ravichander, Khyathi Chandu, et al. 2023. THE GENERATIVE AI PARADOX: \"What It Can Create, It May Not Understand\". In The Twelfth International Conference on Learning Representations."},{"key":"e_1_2_1_59_1","volume-title":"Self-play preference optimization for language model alignment. arXiv preprint arXiv:2405.00675","author":"Wu Yue","year":"2024","unstructured":"Yue Wu, Zhiqing Sun, Huizhuo Yuan, Kaixuan Ji, Yiming Yang, and Quanquan Gu. 2024. Self-play preference optimization for language model alignment. arXiv preprint arXiv:2405.00675 (2024)."},{"key":"e_1_2_1_60_1","volume-title":"Db-gpt: Empowering database interactions with private large language models. arXiv preprint arXiv:2312.17449","author":"Xue Siqiao","year":"2023","unstructured":"Siqiao Xue, Caigao Jiang, Wenhui Shi, Fangyin Cheng, Keting Chen, Hongjun Yang, Zhiping Zhang, Jianshan He, Hongyang Zhang, Ganglin Wei, et al. 2023. Db-gpt: Empowering database interactions with private large language models. arXiv preprint arXiv:2312.17449 (2023)."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133887"},{"key":"e_1_2_1_62_1","volume-title":"Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems 36","author":"Yao Shunyu","year":"2024","unstructured":"Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems 36 (2024)."},{"key":"e_1_2_1_63_1","doi-asserted-by":"crossref","unstructured":"Ziyu Yao Yu Su Huan Sun and Wen-tau Yih. 2019. Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study. (2019) 5447\u20135458.","DOI":"10.18653\/v1\/D19-1547"},{"key":"e_1_2_1_64_1","volume-title":"Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task.","author":"Yu Tao","year":"2018","unstructured":"Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. 2018. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. (2018), 3911\u20133921."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1443"},{"key":"e_1_2_1_66_1","volume-title":"Proceedings of the national conference on artificial intelligence. 1050\u20131055","author":"Zelle John M","year":"1996","unstructured":"John M Zelle and Raymond J Mooney. 1996. Learning to parse database queries using inductive logic programming. In Proceedings of the national conference on artificial intelligence. 1050\u20131055."},{"key":"e_1_2_1_67_1","volume-title":"Differentiable prompt makes pre-trained language models better few-shot learners. arXiv preprint arXiv:2108.13161","author":"Zhang Ningyu","year":"2021","unstructured":"Ningyu Zhang, Luoqiu Li, Xiang Chen, Shumin Deng, Zhen Bi, Chuanqi Tan, Fei Huang, and Huajun Chen. 2021. Differentiable prompt makes pre-trained language models better few-shot learners. arXiv preprint arXiv:2108.13161 (2021)."},{"key":"e_1_2_1_68_1","volume-title":"William Yang Wang, and Linda Ruth Petzold","author":"Zhang Xinlu","year":"2023","unstructured":"Xinlu Zhang, Yujie Lu, Weizhi Wang, An Yan, Jun Yan, Lianke Qin, Heng Wang, Xifeng Yan, William Yang Wang, and Linda Ruth Petzold. 2023. Gpt-4v (ision) as a generalist evaluator for vision-language tasks. arXiv preprint arXiv:2311.01361 (2023)."},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.14778\/3636218.3636225"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE55515.2023.00123"},{"key":"e_1_2_1_71_1","unstructured":"Yunjia Zhang Jordan Henkel Avrilia Floratou Joyce Cahoon Shaleen Deep and Jignesh M Patel. [n.d.]. ReAcTable: Enhancing ReAct for Table Question Answering. ([n. d.])."},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.14778\/3659437.3659452"},{"key":"e_1_2_1_73_1","volume-title":"Hybrid Querying Over Relational Databases and Large Language Models. arXiv preprint arXiv:2408.00884","author":"Zhao Fuheng","year":"2024","unstructured":"Fuheng Zhao, Divyakant Agrawal, and Amr El Abbadi. 2024. Hybrid Querying Over Relational Databases and Large Language Models. arXiv preprint arXiv:2408.00884 (2024)."},{"key":"e_1_2_1_74_1","volume-title":"LLM-SQL-Solver: Can LLMs Determine SQL Equivalence? arXiv preprint arXiv:2312.10321","author":"Zhao Fuheng","year":"2023","unstructured":"Fuheng Zhao, Lawrence Lim, Ishtiyaque Ahmad, Divyakant Agrawal, and Amr El Abbadi. 2023. LLM-SQL-Solver: Can LLMs Determine SQL Equivalence? arXiv preprint arXiv:2312.10321 (2023)."},{"key":"e_1_2_1_75_1","volume-title":"Seq2sql: Generating structured queries from natural language using reinforcement learning. arXiv preprint arXiv:1709.00103","author":"Zhong Victor","year":"2017","unstructured":"Victor Zhong, Caiming Xiong, and Richard Socher. 2017. Seq2sql: Generating structured queries from natural language using reinforcement learning. arXiv preprint arXiv:1709.00103 (2017)."},{"key":"e_1_2_1_76_1","volume-title":"Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911","author":"Zhou Jeffrey","year":"2023","unstructured":"Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911 (2023)."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3717755.3717772","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T16:17:12Z","timestamp":1747757832000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3717755.3717772"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12]]},"references-count":76,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,12]]}},"alternative-id":["10.14778\/3717755.3717772"],"URL":"https:\/\/doi.org\/10.14778\/3717755.3717772","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2024,12]]},"assertion":[{"value":"2025-05-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}