{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T16:21:38Z","timestamp":1775665298619,"version":"3.50.1"},"reference-count":98,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2024,6,19]],"date-time":"2024-06-19T00:00:00Z","timestamp":1718755200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62372304, 62302375, 62192734"],"award-info":[{"award-number":["62372304, 62302375, 62192734"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100002858","name":"China Postdoctoral Science Foundation","doi-asserted-by":"crossref","award":["2023M723736"],"award-info":[{"award-number":["2023M723736"]}],"id":[{"id":"10.13039\/501100002858","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2024,8,31]]},"abstract":"<jats:p>Static analysis tools for capturing bugs and vulnerabilities in software programs are widely employed in practice, as they have the unique advantages of high coverage and independence from the execution environment. However, existing tools for analyzing large codebases often produce a great deal of false warnings over genuine bug reports. As a result, developers are required to manually inspect and confirm each warning, a challenging, time-consuming, and automation-essential task.<\/jats:p>\n          <jats:p>\n            This article advocates a fast, general, and easily extensible approach called\n            <jats:sc>Llm4sa<\/jats:sc>\n            that automatically inspects a sheer volume of static warnings by harnessing (some of) the powers of Large Language Models (LLMs). Our key insight is that LLMs have advanced program understanding capabilities, enabling them to effectively act as human experts in conducting manual inspections on bug warnings with their relevant code snippets. In this spirit, we propose a static analysis to effectively extract the relevant code snippets via program dependence traversal guided by the bug warning reports themselves. Then, by formulating customized questions that are enriched with domain knowledge and representative cases to query LLMs,\n            <jats:sc>Llm4sa<\/jats:sc>\n            can remove a great deal of false warnings and facilitate bug discovery significantly. Our experiments demonstrate that\n            <jats:sc>Llm4sa<\/jats:sc>\n            is practical in automatically inspecting thousands of static warnings from Juliet benchmark programs and 11 real-world C\/C++ projects, showcasing a high precision (81.13%) and a recall rate (94.64%) for a total of 9,547 bug warnings. Our research introduces new opportunities and methodologies for using the LLMs to reduce human labor costs, improve the precision of static analyzers, and ensure software trustworthiness\n          <\/jats:p>","DOI":"10.1145\/3653718","type":"journal-article","created":{"date-parts":[[2024,3,26]],"date-time":"2024-03-26T13:12:41Z","timestamp":1711458761000},"page":"1-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":35,"title":["Automatically Inspecting Thousands of Static Bug Warnings with Large Language Model: How Far Are We?"],"prefix":"10.1145","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1826-6213","authenticated-orcid":false,"given":"Cheng","family":"Wen","sequence":"first","affiliation":[{"name":"Guangzhou Institute of Technology &amp; ICTT and ISN Laboratory, Xidian University, Guangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6340-1416","authenticated-orcid":false,"given":"Yuandao","family":"Cai","sequence":"additional","affiliation":[{"name":"Fermat Labs, Huawei Technologies Co., Ltd, Hong Kong, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-1745-916X","authenticated-orcid":false,"given":"Bin","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5098-8040","authenticated-orcid":false,"given":"Jie","family":"Su","sequence":"additional","affiliation":[{"name":"Guangzhou Institute of Technology &amp; ICTT and ISN Laboratory, Xidian University, Guangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6727-440X","authenticated-orcid":false,"given":"Zhiwu","family":"Xu","sequence":"additional","affiliation":[{"name":"College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3612-709X","authenticated-orcid":false,"given":"Dugang","family":"Liu","sequence":"additional","affiliation":[{"name":"Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Shenzhen University, Shenzhen, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3028-8191","authenticated-orcid":false,"given":"Shengchao","family":"Qin","sequence":"additional","affiliation":[{"name":"Guangzhou Institute of Technology &amp; ICTT and ISN Laboratory, Xidian University, Guangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9310-3460","authenticated-orcid":false,"given":"Zhong","family":"Ming","sequence":"additional","affiliation":[{"name":"College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5429-4580","authenticated-orcid":false,"given":"Tian","family":"Cong","sequence":"additional","affiliation":[{"name":"ICTT and ISN Laboratory &amp; Guangzhou Institute of Technology, Xi'an, China"}]}],"member":"320","published-online":{"date-parts":[[2024,6,19]]},"reference":[{"key":"e_1_3_4_2_2","first-page":"1742","volume-title":"Proceedings of the 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE\u201923)","author":"Ahmed Toufique","year":"2023","unstructured":"Toufique Ahmed and Premkumar Devanbu. 2023. Better patching using LLM prompting, via self-consistency. In Proceedings of the 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE\u201923). IEEE, 1742\u20131746."},{"key":"e_1_3_4_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/567067.567085"},{"key":"e_1_3_4_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/2594291.2594299"},{"key":"e_1_3_4_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/1646353.1646374"},{"key":"e_1_3_4_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3276514"},{"key":"e_1_3_4_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3133956.3134020"},{"key":"e_1_3_4_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678572"},{"key":"e_1_3_4_9_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS\u201920)","author":"Brown Tom B.","year":"2020","unstructured":"Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS\u201920), Hugo Larochelle, Marc\u2019Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html"},{"key":"e_1_3_4_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533767.3534384"},{"key":"e_1_3_4_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533767.3534384"},{"key":"e_1_3_4_12_2","first-page":"209","volume-title":"Proceedings of the 8th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201908)","author":"Cadar Cristian","year":"2008","unstructured":"Cristian Cadar, Daniel Dunbar, and Dawson R. Engler. 2008. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the 8th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201908), Richard Draves and Robbert van Renesse (Eds.). USENIX Association, 209\u2013224. Retrieved from http:\/\/www.usenix.org\/events\/osdi08\/tech\/full_papers\/cadar\/cadar.pdf"},{"key":"e_1_3_4_13_2","volume-title":"Proceedings of the 33rd USENIX Security Symposium (USENIX Security\u201924)","author":"Cai Yuandao","year":"2024","unstructured":"Yuandao Cai, Yibo Jin, and Charles Zhang. 2024. Unleashing the power of type-based call graph construction by using regional pointer information. In Proceedings of the 33rd USENIX Security Symposium (USENIX Security\u201924), Davide Balzarotti and Wenyuan Xu (Eds.). USENIX Association. Retrieved from https:\/\/www.usenix.org\/system\/files\/sec23winter-prepub-350-cai.pdf"},{"key":"e_1_3_4_14_2","volume-title":"Proceedings of the 32nd USENIX Security Symposium (USENIX Security\u201923)","author":"Cai Yuandao","year":"2023","unstructured":"Yuandao Cai, Peisen Yao, Chengfeng Ye, and Charles Zhang. 2023. Place your locks well: Understanding and detecting lock misuse bugs. In Proceedings of the 32nd USENIX Security Symposium (USENIX Security\u201923), Joseph A. Calandrino and Carmela Troncoso (Eds.). USENIX Association. Retrieved from https:\/\/www.usenix.org\/conference\/usenixsecurity23\/presentation\/cai-yuandao"},{"key":"e_1_3_4_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3453483.3454099"},{"key":"e_1_3_4_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549110"},{"key":"e_1_3_4_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3622833"},{"key":"e_1_3_4_18_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-20398-5_33"},{"key":"e_1_3_4_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/1250734.1250789"},{"key":"e_1_3_4_20_2","article-title":"A survey of chain of thought reasoning: Advances, frontiers and future","author":"Chu Zheng","year":"2023","unstructured":"Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Tao He, Haotian Wang, Weihua Peng, Ming Liu, Bing Qin, and Ting Liu. 2023. A survey of chain of thought reasoning: Advances, frontiers and future. arXiv preprint arXiv:2309.15402 (2023).","journal-title":"arXiv preprint arXiv:2309.15402"},{"key":"e_1_3_4_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598067"},{"key":"e_1_3_4_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338112"},{"key":"e_1_3_4_23_2","article-title":"Llama 2: Open foundation and fine-tuned chat models","volume":"2307","author":"Scialom. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas","year":"2023","unstructured":"Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom.2023. Llama 2: Open foundation and fine-tuned chat models. CoRR abs\/2307.09288 (2023).","journal-title":"CoRR"},{"key":"e_1_3_4_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00025"},{"key":"e_1_3_4_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00128"},{"key":"e_1_3_4_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/24039.24041"},{"key":"e_1_3_4_27_2","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2018.23326"},{"key":"e_1_3_4_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/1480881.1480911"},{"key":"e_1_3_4_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00034"},{"key":"e_1_3_4_30_2","article-title":"Large language models for software engineering: A systematic literature review","volume":"2308","author":"Hou Xinyi","year":"2023","unstructured":"Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John C. Grundy, and Haoyu Wang. 2023. Large language models for software engineering: A systematic literature review. CoRR abs\/2308.10620 (2023).","journal-title":"CoRR"},{"key":"e_1_3_4_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP46214.2022.9833751"},{"key":"e_1_3_4_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2019.00040"},{"key":"e_1_3_4_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2013.6606613"},{"key":"e_1_3_4_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460319.3464832"},{"key":"e_1_3_4_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460319.3464832"},{"key":"e_1_3_4_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510214"},{"key":"e_1_3_4_37_2","article-title":"Evaluating diverse large language models for automatic and general bug reproduction","author":"Kang Sungmin","year":"2023","unstructured":"Sungmin Kang, Juyeon Yoon, Nargiz Askarbekkyzy, and Shin Yoo. 2023. Evaluating diverse large language models for automatic and general bug reproduction. arXiv preprint arXiv:2311.04532 (2023).","journal-title":"arXiv preprint arXiv:2311.04532"},{"key":"e_1_3_4_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00194"},{"key":"e_1_3_4_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510153"},{"key":"e_1_3_4_40_2","doi-asserted-by":"crossref","unstructured":"J. Koco\u0144 I. Cichecki O. Kaszyca M. Kochanek D. Szyd\u0142o J. Baran J. Bielaniewicz M. Gruza A. Janz K. Kanclerz A. Koco\u0144 B. Koptyra W. Mieleszczenko-Kowszewicz P. Mi\u0142kowski M. Oleksy M. Piasecki \u0141. Radli\u0144ski K. Wojtasik S. Wo\u017aniak P. Kazienko ChatGPT: Jack of all trades master of none. Information Fusion. 99 (2023) 101861.","DOI":"10.1016\/j.inffus.2023.101861"},{"key":"e_1_3_4_41_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS\u201922)","author":"Kojima Takeshi","year":"2022","unstructured":"Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS\u201922). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2022\/hash\/8bb0d291acd4acf06ef112099c16f326-Abstract-Conference.html"},{"key":"e_1_3_4_42_2","unstructured":"Ted Kremenek. 2008. Finding software bugs with the clang static analyzer. Apple Inc (2008) 2008\u201308."},{"key":"e_1_3_4_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-54862-8_26"},{"key":"e_1_3_4_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3527325"},{"key":"e_1_3_4_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2019.00048"},{"key":"e_1_3_4_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00085"},{"key":"e_1_3_4_47_2","volume-title":"Proceedings of the 44th IEEE Symposium on Security and Privacy (SP\u201923)","author":"Li Haonan","year":"2023","unstructured":"Haonan Li, Yu Hao, Yizhuo Zhai, and Zhiyun Qian. 2023. Poster: Assisting static analysis with large language models: A ChatGPT experiment. In Proceedings of the 44th IEEE Symposium on Security and Privacy (SP\u201923). IEEE. Retrieved from https:\/\/www.ieee-security.org\/TC\/SP2023\/downloads\/SP23-posters\/sp23-posters-paper39-final_version_2_page_abstract.pdf"},{"key":"e_1_3_4_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3503222.3507770"},{"key":"e_1_3_4_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP46215.2023.10179296"},{"key":"e_1_3_4_50_2","article-title":"The scope of ChatGPT in software engineering: A thorough investigation","volume":"2305","author":"Ma Wei","year":"2023","unstructured":"Wei Ma, Shangqing Liu, Wenhan Wang, Qiang Hu, Ye Liu, Cen Zhang, Liming Nie, and Yang Liu. 2023. The scope of ChatGPT in software engineering: A thorough investigation. CoRR abs\/2305.12138 (2023)","journal-title":"CoRR"},{"key":"e_1_3_4_51_2","first-page":"1007","volume-title":"Proceedings of the 26th USENIX Security Symposium (USENIX Security\u201917)","author":"Machiry Aravind","year":"2017","unstructured":"Aravind Machiry, Chad Spensky, Jake Corina, Nick Stephens, Christopher Kruegel, and Giovanni Vigna. 2017. DR. CHECKER: A soundy analysis for linux kernel drivers. In Proceedings of the 26th USENIX Security Symposium (USENIX Security\u201917), Engin Kirda and Thomas Ristenpart (Eds.). USENIX Association, 1007\u20131024. Retrieved from https:\/\/www.usenix.org\/conference\/usenixsecurity17\/technical-sessions\/presentation\/machiry"},{"key":"e_1_3_4_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/1029894.1029907"},{"key":"e_1_3_4_53_2","unstructured":"Daniel Marjam\u00e4ki. Cppcheck: A tool for static C\/C++ code analysis. Retrieved ACCESSED: 2024 from https:\/\/cppcheck.sourceforge.io"},{"key":"e_1_3_4_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/2491411.2501854"},{"key":"e_1_3_4_55_2","first-page":"767","volume-title":"Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering (ICSE\u201924)","author":"Murali Aniruddhan","year":"2023","unstructured":"Aniruddhan Murali, Noble Saji Mathews, Mahmoud Alfadel, Meiyappan Nagappan, and Meng Xu. 2023. FuzzSlice: Pruning false positives in static analysis warnings through function-level fuzzing. In Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering (ICSE\u201924). IEEE Computer Society, 767\u2013779."},{"key":"e_1_3_4_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3494521"},{"key":"e_1_3_4_57_2","first-page":"543","volume-title":"Proceedings of the 22nd USENIX Security Symposium","author":"Octeau Damien","year":"2013","unstructured":"Damien Octeau, Patrick D. McDaniel, Somesh Jha, Alexandre Bartel, Eric Bodden, Jacques Klein, and Yves Le Traon. 2013. Effective inter-component communication mapping in Android: An essential step towards holistic security analysis. In Proceedings of the 22nd USENIX Security Symposium, Samuel T. King (Ed.). USENIX Association, 543\u2013558. Retrieved from https:\/\/www.usenix.org\/conference\/usenixsecurity13\/technical-sessions\/presentation\/octeau"},{"key":"e_1_3_4_58_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2012.10.011"},{"key":"e_1_3_4_59_2","unstructured":"OpenAI. ChatGPT: Optimizing language models for dialogue. Retrieved ACCESSED: 2024 from https:\/\/chat.openai.com"},{"key":"e_1_3_4_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP46215.2023.10179420"},{"key":"e_1_3_4_61_2","unstructured":"Kexin Pei David Bieber Kensen Shi Charles Sutton and Pengcheng Yin. 2023. Can large language models reason about program invariants? In Proceedings of the 40th International Conference on Machine Learning (ICML\u201923). JMLR.org Honolulu Hawaii USA."},{"key":"e_1_3_4_62_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.iotcps.2023.04.003"},{"key":"e_1_3_4_63_2","first-page":"5","volume-title":"Proceedings of the International Symposium on Logic Programming","author":"Reps Thomas W.","year":"1997","unstructured":"Thomas W. Reps. 1997. Program analysis via graph reachability. In Proceedings of the International Symposium on Logic Programming, Jan Maluszynski (Ed.). MIT Press, 5\u201319."},{"key":"e_1_3_4_64_2","unstructured":"Ruben. A code-understanding code-browsing or code-search tool. This is a tool to index then query or search C C++ Java Python Ruby Go and JavaScript source code. Retrieved ACCESSED: 2024. from https:\/\/github.com\/ruben2020\/codequery"},{"key":"e_1_3_4_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3188720"},{"key":"e_1_3_4_66_2","doi-asserted-by":"publisher","DOI":"10.1016\/0304-3975(96)00072-2"},{"key":"e_1_3_4_67_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00485"},{"key":"e_1_3_4_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380346"},{"key":"e_1_3_4_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3192366.3192418"},{"key":"e_1_3_4_70_2","volume-title":"Proceedings of the International Conference on Object-Oriented Programming, Languages and Applications (OOPSLA\/SPLASH\u201917)","author":"Sp\u00e4th Johannes","year":"2017","unstructured":"Johannes Sp\u00e4th, Karim Ali, and Eric Bodden. 2017. IDEal: Efficient and precise alias-aware dataflow analysis. In Proceedings of the International Conference on Object-Oriented Programming, Languages and Applications (OOPSLA\/SPLASH\u201917). ACM Press. Retrieved from https:\/\/bodden.de\/pubs\/sab17ideal.pdf"},{"key":"e_1_3_4_71_2","volume-title":"Proceedings of the European Conference on Object-Oriented Programming (ECOOP\u201916)","author":"Sp\u00e4th Johannes","year":"2016","unstructured":"Johannes Sp\u00e4th, Lisa Nguyen Quang Do, Karim Ali, and Eric Bodden. 2016. Boomerang: Demand-driven flow- and context-sensitive pointer analysis for Java. In Proceedings of the European Conference on Object-Oriented Programming (ECOOP\u201916). Retrieved from https:\/\/www.bodden.de\/pubs\/sna+16boomerang.pdf"},{"key":"e_1_3_4_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/2892208.2892235"},{"key":"e_1_3_4_73_2","doi-asserted-by":"publisher","DOI":"10.1145\/2338965.2336784"},{"key":"e_1_3_4_74_2","first-page":"1288","volume-title":"Proceedings of the 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE\u201923)","author":"Sun Maolin","year":"2023","unstructured":"Maolin Sun, Yibiao Yang, Yang Wang, Ming Wen, Haoxiang Jia, and Yuming Zhou. 2023. SMT solver validation empowered by large pre-trained language models. In Proceedings of the 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE\u201923). IEEE, 1288\u20131300."},{"key":"e_1_3_4_75_2","article-title":"Automatic code summarization via ChatGPT: How far are we?","volume":"2305","author":"Sun Weisong","year":"2023","unstructured":"Weisong Sun, Chunrong Fang, Yudu You, Yun Miao, Yi Liu, Yuekang Li, Gelei Deng, Shenghan Huang, Yuchen Chen, Quanjun Zhang, Hanwei Qian, Yang Liu, and Zhenyu Chen. 2023. Automatic code summarization via ChatGPT: How far are we? CoRR abs\/2305.12865 (2023).","journal-title":"CoRR"},{"issue":"01","key":"e_1_3_4_76_2","first-page":"17","article-title":"Use Chat GPT to solve programming bugs","volume":"3","author":"Surameery Nigar M. Shafiq","year":"2023","unstructured":"Nigar M. Shafiq Surameery and Mohammed Y Shakor. 2023. Use Chat GPT to solve programming bugs. Int. J. Inf. Technol. Comput. Eng. 3, 01 (2023), 17\u201322.","journal-title":"Int. J. Inf. Technol. Comput. Eng."},{"key":"e_1_3_4_77_2","first-page":"678","volume-title":"Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER\u201923)","author":"Tsigkanos Christos","year":"2023","unstructured":"Christos Tsigkanos, Pooja Rani, Sebastian M\u00fcller, and Timo Kehrer. 2023. Large language models: The next frontier for variable discovery within metamorphic testing? In Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER\u201923). IEEE, 678\u2013682."},{"key":"e_1_3_4_78_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-019-09750-5"},{"key":"e_1_3_4_79_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510062"},{"key":"e_1_3_4_80_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380386"},{"key":"e_1_3_4_81_2","doi-asserted-by":"publisher","DOI":"10.1145\/3239235.3239523"},{"key":"e_1_3_4_82_2","article-title":"Self-consistency improves chain of thought reasoning in language models","author":"Wang Xuezhi","year":"2022","unstructured":"Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 (2022).","journal-title":"arXiv preprint arXiv:2203.11171"},{"key":"e_1_3_4_83_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS\u201922)","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS\u201922). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2022\/hash\/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html"},{"key":"e_1_3_4_84_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510178"},{"key":"e_1_3_4_85_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380396"},{"key":"e_1_3_4_86_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00129"},{"key":"e_1_3_4_87_2","first-page":"163","volume-title":"Proceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201910)","author":"Xiong Weiwei","year":"2010","unstructured":"Weiwei Xiong, Soyeon Park, Jiaqi Zhang, Yuanyuan Zhou, and Zhiqiang Ma. 2010. Ad hoc synchronization considered harmful. In Proceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201910), Remzi H. Arpaci-Dusseau and Brad Chen (Eds.). USENIX Association, 163\u2013176. Retrieved from http:\/\/www.usenix.org\/events\/osdi10\/tech\/full_papers\/Xiong.pdf"},{"key":"e_1_3_4_88_2","first-page":"24631","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Xu Mengdi","year":"2022","unstructured":"Mengdi Xu, Yikang Shen, Shun Zhang, Yuchen Lu, Ding Zhao, Joshua Tenenbaum, and Chuang Gan. 2022. Prompting decision transformer for few-shot policy generalization. In Proceedings of the International Conference on Machine Learning. PMLR, 24631\u201324645."},{"key":"e_1_3_4_89_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2017.06.010"},{"key":"e_1_3_4_90_2","volume-title":"Proceedings of the 46th International Conference on Software Engineering","author":"Xu Zhiwu","year":"2024","unstructured":"Zhiwu Xu, Bohao Wu, Cheng Wen, Bin Zhang, Shengchao Qin, and Mengda He. 2024. RPG: Rust library fuzzing with pool-based fuzz target generation and generic support. In Proceedings of the 46th International Conference on Software Engineering."},{"key":"e_1_3_4_91_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180178"},{"key":"e_1_3_4_92_2","first-page":"36366","article-title":"Chain of thought imitation with procedure cloning","volume":"35","author":"Yang Mengjiao Sherry","year":"2022","unstructured":"Mengjiao Sherry Yang, Dale Schuurmans, Pieter Abbeel, and Ofir Nachum. 2022. Chain of thought imitation with procedure cloning. Adv. Neural Inf. Process. Syst. 35 (2022), 36366\u201336381.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_4_93_2","volume-title":"Proceedings of the 33rd USENIX Security Symposium (USENIX Security\u201924)","author":"Ye Chengfeng","year":"2024","unstructured":"Chengfeng Ye, Yuandao Cai, and Charles Zhang. 2024. When threads meet interrupts: Effective static detection of interrupt-based deadlocks. In Proceedings of the 33rd USENIX Security Symposium (USENIX Security\u201924), Davide Balzarotti and Wenyuan Xu (Eds.). USENIX Association."},{"key":"e_1_3_4_94_2","first-page":"30378","article-title":"The unreliability of explanations in few-shot prompting for textual reasoning","volume":"35","author":"Ye Xi","year":"2022","unstructured":"Xi Ye and Greg Durrett. 2022. The unreliability of explanations in few-shot prompting for textual reasoning. Adv. Neural Inf. Process. Syst. 35 (2022), 30378\u201330392.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_4_95_2","doi-asserted-by":"publisher","DOI":"10.1145\/3538644"},{"key":"e_1_3_4_96_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409686"},{"key":"e_1_3_4_97_2","doi-asserted-by":"publisher","DOI":"10.1145\/3133881"},{"key":"e_1_3_4_98_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER56733.2023.00022"},{"key":"e_1_3_4_99_2","article-title":"A survey of large language models","volume":"2303","author":"Zhao Wayne Xin","year":"2023","unstructured":"Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A survey of large language models. CoRR abs\/2303.18223 (2023).","journal-title":"CoRR"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3653718","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3653718","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:58Z","timestamp":1750291438000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3653718"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,19]]},"references-count":98,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2024,8,31]]}},"alternative-id":["10.1145\/3653718"],"URL":"https:\/\/doi.org\/10.1145\/3653718","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,19]]},"assertion":[{"value":"2023-09-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-03","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}