{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T15:40:15Z","timestamp":1780674015135,"version":"3.54.1"},"reference-count":38,"publisher":"Emerald","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,1,5]]},"abstract":"<jats:p>The ever-growing popularity of large language models (LLMs) has resulted in their increasing adoption for hardware design and verification. Prior research has attempted to 9jSS\u0398SS the capability of LLMs to automate digital hardware design by producing superior-quality Register Transfer Logic (RTL) descriptions, particularly in Verilog. However, these tests have revealed that Verilog code production using LLMs at current state-of-the-art lack sufficient functional correctness to be practically viable, compared to automatic generation of programs in general-purpose programming languages such as C, C++, Python, etc. With this as the key insight, in this work we assess the performance of a two-stage software pipeline for automated Verilog RTL generation for combinational circuits: LLM based automatic generation of annotated C++ code suitable for high-level synthesis (HLS), followed by HLS to generate Verilog RTL. We have benchmarked the performance of our proposed scheme using the open-source VerilogEval dataset, for four different industry-scale LLMs, and the Vitis HLS tool. Our experimental results demonstrate that our two-step technique substantially outperforms previous proposed techniques of direct Verilog RTL generation by LLMs in terms of average functional correctness rates, reaching a score of 0.86 in pass@l metric.<\/jats:p>","DOI":"10.1561\/1000000063-3","type":"journal-article","created":{"date-parts":[[2025,5,1]],"date-time":"2025-05-01T04:29:47Z","timestamp":1746073787000},"page":"295-314","source":"Crossref","is-referenced-by-count":4,"title":["Evaluating Large Language Models for Automatic Register Transfer Logic Generation for Combinational Circuits via High-Level Synthesis"],"prefix":"10.1108","volume":"14","author":[{"given":"Sneha","family":"Swaroopa","sequence":"first","affiliation":[{"name":"Indian Institute of Technology Kharagpur ,","place":["India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rijoy","family":"Mukherjee","sequence":"additional","affiliation":[{"name":"Indian Institute of Technology Kharagpur ,","place":["India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anushka","family":"Debnath","sequence":"additional","affiliation":[{"name":"National Institute of Technology Durgapur ,","place":["India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rajat Subhra","family":"Chakraborty","sequence":"additional","affiliation":[{"name":"Indian Institute of Technology Kharagpur ,","place":["India"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"140","published-online":{"date-parts":[[2025,1,5]]},"reference":[{"key":"2026032901025398200_ref001","doi-asserted-by":"publisher","first-page":"4043","DOI":"10.1109\/TIFS.2024.3374558","article-title":"On Hardware Security Bug Code Fixes by Prompting Large Language Models","volume":"19","author":"Ahmad","year":"2024","journal-title":"IEEE Transactions on Information Forensics and Security"},{"key":"2026032901025398200_ref002","unstructured":"AMD\n          . \u201cVitis HLS\u201d. URL: https:\/\/docs.amd.com\/r\/en-US\/ug1399-vitis-hls."},{"key":"2026032901025398200_ref003","unstructured":"Anthropic\n          . (2024a). \u201cClaude 3 Haiku\u201d. URL: https:\/\/www.anthropic.com\/news\/claude-3-haiku."},{"key":"2026032901025398200_ref004","unstructured":"Anthropic\n          . (2024b). \u201cClaude 3.5 Sonnet\u201d. URL: https:\/\/www.anthropic.com\/news\/claude-3-5-sonnet."},{"key":"2026032901025398200_ref005","doi-asserted-by":"crossref","DOI":"10.1109\/MLCAD58807.2023.10299874","article-title":"Chip-Chat: Challenges and Opportunities in Conversational Hardware Design","author":"Blocklove","year":"2023"},{"key":"2026032901025398200_ref006","doi-asserted-by":"crossref","unstructured":"Blocklove, J.\n            et al.\n           (2024). \u201cEvaluating LLMs for Hardware Design and Test\u201d. URL: https:\/\/arxiv.org\/abs\/2405.02326.","DOI":"10.1109\/LAD62341.2024.10691811"},{"key":"2026032901025398200_ref007","unstructured":"Chang, K.\n            et al.\n           (2023). \u201cChipGPT: How far are we from natural language hardware design\u201d. URL: https:\/\/arxiv.org\/abs\/2305.14019."},{"key":"2026032901025398200_ref008","doi-asserted-by":"crossref","unstructured":"Chang, K.\n            et al.\n           (2024). \u201cData is all you need: Finetuning LLMs for Chip Design via an Automated design-data augmentation framework\u201d. URL: https:\/\/arxiv.org\/abs\/2403.11202.","DOI":"10.1145\/3649329.3657356"},{"key":"2026032901025398200_ref009","unstructured":"Chen, M.\n            et al.\n           (2021). \u201cEvaluating Large Language Models Trained on Code\u201d. URL: https:\/\/arxiv.org\/abs\/2107.03374."},{"key":"2026032901025398200_ref010","doi-asserted-by":"crossref","unstructured":"Collini, L., S.Garg, and R.Karri. (2024). \u201cC2HLSC: Can LLMs Bridge the Software-to-Hardware Design Gap?\u201d URL: https:\/\/arxiv.org\/abs\/2406.09233.","DOI":"10.1109\/LAD62341.2024.10691856"},{"issue":"4","key":"2026032901025398200_ref011","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1109\/TCAD.2011.2110592","article-title":"High-Level Synthesis for FPGAs: From Prototyping to Deployment","volume":"30","author":"Cong","year":"2011","journal-title":"IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems"},{"issue":"4","key":"2026032901025398200_ref012","doi-asserted-by":"crossref","DOI":"10.1145\/3530775","article-title":"FPGA HLS Today: Successes, Challenges, and Opportunities","volume":"15","author":"Cong","year":"2022","journal-title":"ACM Transactions on Reconfigurable Technology and Systems"},{"key":"2026032901025398200_ref013","unstructured":"Dehaerne, E.\n            et al.\n           (2023). \u201cA Deep Learning Framework for Verilog Autocompletion Towards Design and Verification Automation\u201d. URL: https:\/\/arxiv.org\/abs\/2304.13840."},{"key":"2026032901025398200_ref014","unstructured":"DeLorenzo, M.\n            et al.\n           (2024a). \u201cMake Every Move Count: LLM-based High-Quality RTL Code Generation Using MCTS\u201d. URL: https:\/\/arxiv.org\/abs\/2402.03289."},{"key":"2026032901025398200_ref015","doi-asserted-by":"crossref","unstructured":"DeLorenzo, M., V.Gohil, and J.Rajendran. (2024b). \u201cCreativEval: Evaluating Creativity of LLM-Based Hardware Code Generation\u201d. URL: https:\/\/arxiv.org\/abs\/2404.08806.","DOI":"10.1109\/LAD62341.2024.10691798"},{"key":"2026032901025398200_ref016","first-page":"13271330","article-title":"Bambu: an Open-Source Research Framework for the High-Level Synthesis of Complex Applications","author":"Ferrandi","year":"2021"},{"key":"2026032901025398200_ref017","first-page":"1","article-title":"GPT4AIGChip: Towards Next-Generation AI Accelerator Design Automation via Large Language Models","author":"Fu","year":"2023"},{"key":"2026032901025398200_ref018","doi-asserted-by":"publisher","first-page":"4374","DOI":"10.1109\/TIFS.2024.3372809","article-title":"(Security) Assertions by Large Language Models","volume":"19","author":"Kande","year":"2024","journal-title":"IEEE Transactions on Information Forensics and Security"},{"key":"2026032901025398200_ref019","article-title":"VerilogEval: Evaluating Large Language Models for Verilog Code Generation","author":"Liu","year":"2023"},{"key":"2026032901025398200_ref020","unstructured":"Liu, M.\n            et al.\n           (2024a). \u201cChipNeMo: Domain-Adapted LLMs for Chip Design\u201d. URL: https:\/\/arxiv.org\/abs\/2311.00176."},{"key":"2026032901025398200_ref021","unstructured":"Liu, M.\n            et al.\n           (2024b). \u201cVerilogEval\u201d. URL: https:\/\/github.com\/NVlabs\/verilog-eval."},{"key":"2026032901025398200_ref022","unstructured":"Meng, X.\n            et al.\n           (2023). \u201cUnlocking Hardware Security Assurance: The Potential of LLMs\u201d. URL: https:\/\/arxiv.org\/abs\/2308.11042."},{"key":"2026032901025398200_ref023","unstructured":"Mukherjee, R.\n           (2024). \u201cHLSEval\u201d. URL: https:\/\/github.com\/rijoym\/HLSEval."},{"key":"2026032901025398200_ref024","unstructured":"Nijkamp, E.\n            et al.\n           (2023). \u201cCodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis\u201d. URL: https:\/\/arxiv.org\/abs\/2203.13474."},{"key":"2026032901025398200_ref025","unstructured":"OpenAI\n          . (2024a). \u201cgpt-3.5\u201d. URL: https:\/\/openai.com\/index\/chatgpt\/."},{"key":"2026032901025398200_ref026","unstructured":"OpenAI\n          . (2024b). \u201cgpt-4\u201d. URL: https:\/\/openai.com\/index\/gpt-4-research\/."},{"key":"2026032901025398200_ref027","unstructured":"OpenAI\n          . (2024c). \u201cgpt-4o\u201d. URL: https:\/\/openai.com\/index\/hello-gpt-4o\/."},{"key":"2026032901025398200_ref028","unstructured":"Orenes-Vera, M., M.Martonosi, and D.Wentzlaff. (2023). \u201cUsing LLMs to Facilitate Formal Verification of RTL\u201d. URL: https:\/\/arxiv.org\/abs\/2309.09437."},{"key":"2026032901025398200_ref029","unstructured":"Paria, S., A.Dasgupta, and S.Bhunia. (2023). \u201cDIVAS: An LLM-based End-to-End Framework for SoC Security Analysis and Policy-based Protection\u201d. URL: https:\/\/arxiv.org\/abs\/2308.06932."},{"key":"2026032901025398200_ref030","first-page":"27","article-title":"DAVE: Deriving Automatically Verilog from English","author":"Pearce","year":"2020"},{"key":"2026032901025398200_ref031","doi-asserted-by":"crossref","unstructured":"Qiu, R.\n            et al.\n           (2024). \u201cAutoBench: Automatic Testbench Generation and Evaluation Using LLMs for HDL Design\u201d. URL: https:\/\/arxiv.org\/abs\/2407.03891.","DOI":"10.1145\/3670474.3685956"},{"key":"2026032901025398200_ref032","unstructured":"Rozi\u00e8re, B.\n            et al.\n           (2024). \u201cCode Llama: Open Foundation Models for Code\u201d. URL: https:\/\/arxiv.org\/abs\/2308.12950."},{"issue":"3","key":"2026032901025398200_ref033","doi-asserted-by":"crossref","DOI":"10.1145\/3643681","article-title":"VeriGen: A Large Language Model for Verilog Code Generation","volume":"29","author":"Thakur","year":"2024","journal-title":"ACM Trans. Des. Autom. Electron. Syst"},{"key":"2026032901025398200_ref034","unstructured":"Thakur, S., J.Blocklove, H.Pearce, B.Tan, S.Garg, and R.Karri. (2024b). \u201cAutoChip: Automating HDL Generation Using LLM Feedback\u201d. URL: https:\/\/arxiv.org\/abs\/2311.04887."},{"key":"2026032901025398200_ref035","unstructured":"Touvron, H.\n            et al.\n           (2023). \u201cLlama 2: Open Foundation and Fine-Tuned Chat Models\u201d. URL: https:\/\/arxiv.org\/abs\/2307.09288."},{"key":"2026032901025398200_ref036","unstructured":"Williams, S.\n           (2024). \u201cThe ICARUS Verilog Compilation System\u201d. URL: https:\/\/github.com\/steveicarus\/iverilog."},{"key":"2026032901025398200_ref037","unstructured":"Wong, H.\n           (2024). \u201cHDLBits\u201d. URL: https:\/\/hdlbits.01xz.net\/wiki\/Problem_sets."},{"key":"2026032901025398200_ref038","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2024.3383347","article-title":"ChatEDA: A Large Language Model Powered Autonomous Agent for EDA","volume-title":"IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems: 1\u20131","author":"Wu","year":"2024"}],"container-title":["Foundations and Trends\u00ae in Electronic Design Automation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/fteda\/article-pdf\/14\/4\/295\/10995729\/1000000063-3en.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/www.emerald.com\/fteda\/article-pdf\/14\/4\/295\/10995729\/1000000063-3en.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T14:13:38Z","timestamp":1777472018000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.emerald.com\/fteda\/article\/14\/4\/295\/1324715\/Evaluating-Large-Language-Models-for-Automatic"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,5]]},"references-count":38,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,1,5]]}},"URL":"https:\/\/doi.org\/10.1561\/1000000063-3","relation":{},"ISSN":["1551-3939","1551-3947"],"issn-type":[{"value":"1551-3939","type":"print"},{"value":"1551-3947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,5]]}}}