{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T22:57:29Z","timestamp":1784761049900,"version":"3.55.0"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"6","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Des. Autom. Electron. Syst."],"published-print":{"date-parts":[[2025,11,30]]},"abstract":"<jats:p>Bug detection in Hardware Design Languages (HDLs) is an important problem in the System-on-Chip (SoC) development cycle. It is crucial to find defects at the earliest stage possible. While most fault localization requires the use of \u201ctests\u201d (e.g., test benches, fuzzing, and assertions) and a simulation or emulation framework, the advent of Large Language Models (LLMs) provides an opportunity for a test-free fault localization approach. This article proposes such a tool, called FLAG, which can identify functional and security defects in Register Transfer Level (RTL) code without synthesis or simulation. FLAG combines syntactic and generative AI techniques to implement fault localization in RTL code. It takes an RTL design as an input and outputs a set of line(s) that likely contain defects. It targets elements of RTL code most likely to contain bugs through static analysis means and then implements token-level and line-level analysis to obtain differences in original code and code generated by LLM to identify a line as buggy or not. The token-level approach evaluates each generated token (one at a time) and the line level approach evaluates the entire line generated by the LLM. We evaluate our approach on a corpus of synthetic and real-world bugs, of both functional and security related issues, in Verilog and SystemVerilog. Using line-level analysis, FLAG can identify 38 out of 120 real-world bugs and using token-level analysis, FLAG can identify 32 out of 81 synthetic bugs through the top-5 most likely bug locations identified without tests.<\/jats:p>","DOI":"10.1145\/3736411","type":"journal-article","created":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T07:16:19Z","timestamp":1747725379000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["FLAG:\n                    <u>F<\/u>\n                    inding\n                    <u>L<\/u>\n                    ine\n                    <u>A<\/u>\n                    nomalies (in RTL code) with\n                    <u>G<\/u>\n                    enerative AI"],"prefix":"10.1145","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6854-3966","authenticated-orcid":false,"given":"Baleegh","family":"Ahmad","sequence":"first","affiliation":[{"name":"Electrical and Computer Engineering, New York University Tandon School of Engineering","place":["Brooklyn, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1009-5499","authenticated-orcid":false,"given":"Joey","family":"Ah-Kiow","sequence":"additional","affiliation":[{"name":"Electrical and Computer Engineering, New York University Tandon School of Engineering","place":["Brooklyn, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7642-3638","authenticated-orcid":false,"given":"Benjamin","family":"Tan","sequence":"additional","affiliation":[{"name":"Electrical and Software Engineering, University of Calgary","place":["Calgary, Canada"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7989-5617","authenticated-orcid":false,"given":"Ramesh","family":"Karri","sequence":"additional","affiliation":[{"name":"ECE, New York University","place":["Brooklyn, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3488-7004","authenticated-orcid":false,"given":"Hammond","family":"Pearce","sequence":"additional","affiliation":[{"name":"University of New South Wales Faculty of Engineering","place":["Sydney, Australia"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,10,21]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","unstructured":"Joey Ah-kiow and Benjamin Tan. 2024. An Investigation of Hardware Security Bug Characteristics in Open-Source Projects. DOI:DOI:10.48550\/arXiv.2402.00684arXiv:2402.00684 [cs].","DOI":"10.48550\/arXiv.2402.00684"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3508352.3549369"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2024.3374558"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.23919\/DATE.2017.7927266"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/2954679.2872414"},{"key":"e_1_3_2_7_2","first-page":"1877","volume-title":"Advances in Neural Information Processing Systems","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems. H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 1877\u20131901. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2020\/file\/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3641289"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde de Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman et al.2021. Evaluating Large Language Models Trained on Code. DOI:DOI:10.48550\/arXiv.2107.03374arXiv:2107.03374 [cs].","DOI":"10.48550\/arXiv.2107.03374"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC48688.2020.0-152"},{"key":"e_1_3_2_11_2","volume-title":"Advances in Neural Information Processing Systems","author":"Christiano Paul F.","year":"2017","unstructured":"Paul F. Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems. I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/d5e2c0adad503c91f91df240d0cd4e49-Paper.pdf"},{"key":"e_1_3_2_12_2","unstructured":"The MITRE Corporation. 2023. CWE\u20142023 CWE Top 25 Most Dangerous Software Weaknesses. Retrieved from https:\/\/cwe.mitre.org\/top25\/archive\/2023\/2023_top25_list.html"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIEM48762.2020.9160152"},{"key":"e_1_3_2_14_2","unstructured":"HACK@EVENT. 2022. HACK@DAC21\u2014HacK@EVENT. Retrieved from https:\/\/hackatevent.org\/hackdac21\/"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/MTV.2014.25"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-21606-5_10"},{"key":"e_1_3_2_17_2","unstructured":"James Hills and Shyamal Anadkat. 2023. Using Logprobs | OpenAI Cookbook. Retrieved from https:\/\/cookbook.openai.com\/examples\/using_logprobs"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/2902362"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447867"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","unstructured":"Hanxian Huang Zhenghan Lin Zixuan Wang Xin Chen Ke Ding and Jishen Zhao. 2024. Towards LLM-Powered Verilog RTL Assistant: Self-Verification and Self-Correction. DOI:DOI:10.48550\/arXiv.2406.00115arXiv:2406.00115 [cs].","DOI":"10.48550\/arXiv.2406.00115"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP40001.2021.00103"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549149"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITAIC49862.2020.9338932"},{"key":"e_1_3_2_24_2","unstructured":"VC SpyGlass Lint. 2022. Synopsys VC SpyGlass Lint. Retrieved from https:\/\/www.synopsys.com\/verification\/static-and-formal-verification\/vc-spyglass\/vc-spyglass-lint.html"},{"key":"e_1_3_2_25_2","unstructured":"lowRISC contributors. 2023. Open Source Silicon Root of Trust (RoT) | OpenTitan. Retrieved from https:\/\/opentitan.org\/"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","unstructured":"Sudipta Paria Aritra Dasgupta and Swarup Bhunia. 2023. DIVAS: An LLM-based End-to-End Framework for SoC Security Analysis and Policy-based Protection. DOI:DOI:10.48550\/arXiv.2308.06932arXiv:2308.06932 [cs].","DOI":"10.48550\/arXiv.2308.06932"},{"key":"e_1_3_2_27_2","unstructured":"Alec Radford Jeffrey Wu Rewon Child David Luan Dario Amodei and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. (2019) 24. Retrieved from https:\/\/cdn.openai.com\/better-language-models\/language_models_are_unsupervised_multitask_learners.pdf"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884848"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2023.3269899"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/VDAT50263.2020.9190603"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.23919\/DATE56975.2023.10137086"},{"key":"e_1_3_2_32_2","unstructured":"Timothy Trippel Kang G. Shin Alex Chernyakhovsky Garret Kelly Dominic Rizzo and Matthew Hicks. 2022. Fuzzing hardware like software. 3237\u20133254. Retrieved fromhttps:\/\/www.usenix.org\/conference\/usenixsecurity22\/presentation\/trippel"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635875"},{"key":"e_1_3_2_34_2","unstructured":"Verific. 2022. Verific Design Automation. Retrieved from https:\/\/www.verific.com\/"},{"key":"e_1_3_2_35_2","unstructured":"Verilator. 2021. Verilator User\u2019s Guide\u2014Verilator 4.202 Documentation. Retrieved from https:\/\/verilator.org\/guide\/latest\/#"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/2970276.2970341"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3623342"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","unstructured":"Bingkun Yao Ning Wang Jie Zhou Xi Wang Hong Gao Zhe Jiang and Nan Guan. 2024. Location is Key: Leveraging Large Language Model for Functional Bug Localization in Verilog. DOI:DOI:10.48550\/arXiv.2409.15186arXiv:2409.15186 [cs].","DOI":"10.48550\/arXiv.2409.15186"},{"key":"e_1_3_2_39_2","unstructured":"Florian Zaruba. 2020. OpenHW Group CVA6 User Manual\u2014CVA6 Documentation. https:\/\/cva6.readthedocs.io\/en\/latest\/index.html"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","unstructured":"Wayne Xin Zhao Kun Zhou Junyi Li Tianyi Tang Xiaolei Wang Yupeng Hou Yingqian Min Beichen Zhang Junjie Zhang Zican Dong et al.2024. A Survey of Large Language Models. DOI:DOI:10.48550\/arXiv.2303.18223arXiv:2303.18223 [cs].","DOI":"10.48550\/arXiv.2303.18223"}],"container-title":["ACM Transactions on Design Automation of Electronic Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3736411","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T12:21:54Z","timestamp":1761135714000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3736411"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,21]]},"references-count":39,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,11,30]]}},"alternative-id":["10.1145\/3736411"],"URL":"https:\/\/doi.org\/10.1145\/3736411","relation":{},"ISSN":["1084-4309","1557-7309"],"issn-type":[{"value":"1084-4309","type":"print"},{"value":"1557-7309","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,21]]},"assertion":[{"value":"2024-09-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-05","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-21","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}