{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T12:32:41Z","timestamp":1793363561586,"version":"4.5.0"},"reference-count":50,"publisher":"SAGE Publications","issue":"5-6","license":[{"start":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T00:00:00Z","timestamp":1768348800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/100009980","name":"Old Dominion University","doi-asserted-by":"crossref","award":["AI Initiative"],"award-info":[{"award-number":["AI Initiative"]}],"id":[{"id":"10.13039\/100009980","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Computational Biology"],"published-print":{"date-parts":[[2026,6,1]]},"abstract":"<jats:p>In the past decade, a surge in the amount of electronic health record (EHR) data in the United States has been attributed to a favorable policy environment created by the Health Information Technology for Economic and Clinical Health (HITECH) Act of 2009 and the 21st Century Cures Act of 2016. Clinical notes for patients\u2019 assessments, diagnoses, and treatments are captured in these EHRs in free-form text by physicians, who spend a considerable amount of time entering and editing them. Manually writing clinical notes takes a considerable amount of a doctor\u2019s valuable time, increasing the patient\u2019s waiting time and possibly delaying diagnoses. Large language models (LLMs) possess the ability to generate news articles that closely resemble human-written ones. We investigate the usage of Chain-of-Thought (CoT) prompt engineering to improve the LLM\u2019s response in clinical note generation. In our prompts, we use as input International Classification of Diseases codes and basic patient information. We investigate a strategy that combines the traditional CoT with semantic search results to improve the quality of generated clinical notes. Additionally, we infuse a knowledge graph built from clinical ontology to further enrich the domain-specific knowledge of generated clinical notes. We test our prompting technique on six clinical cases from the CodiEsp test dataset using GPT-4, and our results show that it outperformed the clinical notes generated by standard one-shot prompts.<\/jats:p>","DOI":"10.1177\/15578666251411390","type":"journal-article","created":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T09:25:42Z","timestamp":1768382742000},"page":"624-642","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":0,"title":["Enhancing Clinical Note Generation with ICD-10, Clinical Ontology Knowledge Graphs, and Chain-of-Thought Prompting Using GPT-4"],"prefix":"10.1177","volume":"33","author":[{"given":"Ivan","family":"Makohon","sequence":"first","affiliation":[{"name":"Department of Computer Science, Old Dominion University, Norfolk, Virginia, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mohamad","family":"Najafi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Old Dominion University, Norfolk, Virginia, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jian","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Old Dominion University, Norfolk, Virginia, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mathias","family":"Brochhausen","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yaohang","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Old Dominion University, Norfolk, Virginia, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2026,1,14]]},"reference":[{"key":"e_1_3_2_2_1","doi-asserted-by":"publisher","DOI":"10.1370\/afm.2121"},{"key":"e_1_3_2_3_1","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocaa199"},{"key":"e_1_3_2_4_1","doi-asserted-by":"crossref","unstructured":"Beltagy I Lo K Cohan A. SciBERT: A pretrained language model for scientific text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) 2019; pp. 3615\u20133620.","DOI":"10.18653\/v1\/D19-1371"},{"key":"e_1_3_2_5_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i16.29720"},{"key":"e_1_3_2_6_1","article-title":"Intelligent clinical documentation: Harnessing generative AI for patient-centric clinical note generation","author":"Biswas A","year":"2024","unstructured":"Biswas A, , Talukdar W. Intelligent clinical documentation: Harnessing generative AI for patient-centric clinical note generation. ArXiv, 2024. arXiv:2405.18346.","journal-title":"ArXiv"},{"key":"e_1_3_2_7_1","volume-title":"NeurIPS","author":"Brown TB","year":"2020","unstructured":"Brown TB, , Mann B, , Ryder N, et al. Language models are few-shot learners. In: NeurIPS. Curran Associates Inc.: Vancouver, Canada; 2020."},{"key":"e_1_3_2_8_1","doi-asserted-by":"publisher","DOI":"10.1001\/virtualmentor.2011.13.3.hlaw1-1103"},{"key":"e_1_3_2_9_1","doi-asserted-by":"publisher","DOI":"10.1038\/s43856-024-00717-2"},{"key":"e_1_3_2_10_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0300487"},{"key":"e_1_3_2_11_1","doi-asserted-by":"publisher","DOI":"10.1377\/hlthaff.2018.0728"},{"key":"e_1_3_2_12_1","first-page":"4171","volume-title":"NAACL: HLT","author":"Devlin J","year":"2019","unstructured":"Devlin J, , Chang M, , Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding. In: NAACL: HLT; Minneapolis, Minnesota; 2019; pp. 4171\u20134186."},{"key":"e_1_3_2_13_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2014.17128"},{"key":"e_1_3_2_14_1","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2016.35"},{"key":"e_1_3_2_15_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2019.9609"},{"key":"e_1_3_2_16_1","article-title":"Do large language models understand medical codes","author":"Lee S","year":"2024","unstructured":"Lee S, , Lindsey T. Do large language models understand medical codes? ArXiv, 2024 arXiv:2403.10822v2","journal-title":"ArXiv"},{"key":"e_1_3_2_17_1","article-title":"A gen AI framework for medical note generation","author":"Leong HY","year":"2024","unstructured":"Leong HY, , Gao YF, , Ji S, et al. A gen AI framework for medical note generation. ArXiv, 2024 arXiv:2410.01841.","journal-title":"ArXiv"},{"key":"e_1_3_2_18_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3649449","article-title":"Pre-trained language models for text generation: A survey","volume":"56","author":"Li J","year":"2022","unstructured":"Li J, , Tang T, , Zhao W, et al. Pre-trained language models for text generation: A survey. ACM Comput Surv, 2022; 56:1\u201339.","journal-title":"ACM Comput Surv"},{"key":"e_1_3_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCBB.2018.2817488"},{"key":"e_1_3_2_20_1","article-title":"Can large language models reason about medical questions","volume":"5","author":"Li\u2019evin V","year":"2022","unstructured":"Li\u2019evin V, , Hother C, , Winther O. Can large language models reason about medical questions? Patterns, 2022; 5.","journal-title":"Patterns"},{"key":"e_1_3_2_21_1","unstructured":"Liu PJ. Learning to write notes in electronic health records arXiv 2018. Available from: arXiv:1808.02622"},{"key":"e_1_3_2_22_1","article-title":"Large language model guided tree-of-thought","author":"Long J","year":"2023","unstructured":"Long J. Large language model guided tree-of-thought. ArXiv, 2023 arXiv:2305.08291.","journal-title":"ArXiv"},{"key":"e_1_3_2_23_1","doi-asserted-by":"publisher","DOI":"10.3390\/bioengineering12060631"},{"key":"e_1_3_2_24_1","doi-asserted-by":"publisher","unstructured":"Makohon I Li Y. Multi-label classification of icd-10 coding clinical notes using mimic codiesp. In: 2021 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI). IEEE; 2021; pp. 1\u20134; doi: 10.1109\/BHI50953.2021.9508541","DOI":"10.1109\/BHI50953.2021.9508541"},{"key":"e_1_3_2_25_1","doi-asserted-by":"publisher","DOI":"10.21105\/joss.00861"},{"key":"e_1_3_2_26_1","unstructured":"Miranda-Escalada1 A Gonzalez-Agirre1 A Armengol-Estap J et al. \u2032 Overview of automatic clinical coding: Annotations guidelines and solutions for non-English clinical cases at codiesp track of clef ehealth 2020. In: Conference and Labs of the Evaluation Forum (CLEF) 2020."},{"key":"e_1_3_2_27_1","doi-asserted-by":"publisher","DOI":"10.2967\/jnmt.119.227819"},{"key":"e_1_3_2_28_1","unstructured":"National Library of Medicine. SNOMED CT. U.S. Department of Health and Human Services 2019. Available from: https:\/\/www.nlm.nih.gov\/healthit\/snomedct\/index.html [Last accessed: May31 2025]."},{"key":"e_1_3_2_29_1","unstructured":"National Library of Medicine. SNOMED CT to ICD-10-CM Map. U.S. Department of Health and Human Services; 2021. Available from: https:\/\/www.nlm.nih.gov\/research\/umls\/mapping_projects\/snomedct_to_icd10cm.html [Last accessed: May31 2025]."},{"issue":"1","key":"e_1_3_2_30_1","first-page":"2153","article-title":"De-identification of free text data containing personal health information: A scoping review of reviews","volume":"8","author":"Negash B","year":"2023","unstructured":"Negash B, , Katz A, , Neilson CJ, et al. De-identification of free text data containing personal health information: A scoping review of reviews. Int J Popul Data Sci, 2023; 8(1):2153; doi: 10.23889\/ijpds.v8i1.2153","journal-title":"Int J Popul Data Sci"},{"key":"e_1_3_2_31_1","doi-asserted-by":"publisher","DOI":"10.1001\/virtualmentor.2011.13.3.ccas1-1103"},{"key":"e_1_3_2_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2023.104396"},{"key":"e_1_3_2_33_1","unstructured":"Office of the National Coordinator for Health Information Technology. Office-based physician electronic health record adoption health it quick-stat #50. 2025. Available from: https:\/\/www.healthit.gov\/data\/quickstats\/office-based-physician-electronic-health-record-adoption"},{"key":"e_1_3_2_34_1","article-title":"GPT-4 technical report","author":"OpenAI","year":"2023","unstructured":"OpenAI. GPT-4 technical report. ArXiv, 2023; doi: 10.48550\/arXiv.2303.08774arXiv:2303.08774","journal-title":"ArXiv"},{"key":"e_1_3_2_35_1","article-title":"A systematic survey of prompt engineering in large language models: Techniques and applications","author":"Sahoo P","year":"2024","unstructured":"Sahoo P, , Singh AK, , Saha S, et al. A systematic survey of prompt engineering in large language models: Techniques and applications. ArXiv, 2024 arXiv:2402.07927.","journal-title":"ArXiv"},{"key":"e_1_3_2_36_1","unstructured":"Stenetorp P Pyysalo S Topi\u0107 G et al. brat: A web-based tool for NLP-assisted text annotation. In: Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics 2012."},{"key":"e_1_3_2_37_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01258-7"},{"key":"e_1_3_2_38_1","unstructured":"U.S. Food & Drug Administration. 21st century cures act. 2020. Available from: https:\/\/www.fda.gov\/regulatory-information\/selected-amendments-fdc-act\/21st-century-cures-act"},{"key":"e_1_3_2_39_1","doi-asserted-by":"publisher","DOI":"10.1101\/2022.04.13.22273750"},{"key":"e_1_3_2_40_1","doi-asserted-by":"publisher","DOI":"10.3390\/healthcare13060603"},{"key":"e_1_3_2_41_1","article-title":"Towards adapting open-source large language models for cost-effective, expert-level clinical note generation with on-policy reinforcement learning","author":"Wang H","year":"2024","unstructured":"Wang H, , Gao C, , Liu B, et al. Towards adapting open-source large language models for cost-effective, expert-level clinical note generation with on-policy reinforcement learning. ArXiv, 2024 arXiv:2405.00715.","journal-title":"ArXiv"},{"key":"e_1_3_2_42_1","article-title":"Self-consistency improves chain of thought reasoning in language models","author":"Wang X","year":"2022","unstructured":"Wang X, , Wei J, , Schuurmans D, et al. Self-consistency improves chain of thought reasoning in language models. ArXiv, 2022.","journal-title":"ArXiv"},{"key":"e_1_3_2_43_1","doi-asserted-by":"crossref","unstructured":"Wei J Wang X Schuurmans D et al. Chain-of-thought prompting elicits reasoning in large language models. In: Proceedings of the 36th International Conference on Neural Information Processing Systems NIPS \u201822 2022.","DOI":"10.52202\/068431-1800"},{"key":"e_1_3_2_44_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.167"},{"key":"e_1_3_2_45_1","article-title":"A survey on LLM-generated text detection: Necessity, methods, and future directions","author":"Wu J","year":"2023","unstructured":"Wu J, , Yang S, , Zhan R, et al. A survey on LLM-generated text detection: Necessity, methods, and future directions. ArXiv, 2023 arXiv:2310.14724.","journal-title":"ArXiv"},{"key":"e_1_3_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2022.3193291"},{"key":"e_1_3_2_47_1","doi-asserted-by":"publisher","DOI":"10.52202\/075280-0517"},{"key":"e_1_3_2_48_1","article-title":"Beyond chain-of-thought, effective graph-of-thought reasoning in large language models","author":"Yao Y","year":"2023","unstructured":"Yao Y, , Li Z, , Zhao H. Beyond chain-of-thought, effective graph-of-thought reasoning in large language models. ArXiv, 2023b arXiv:2305.16582.","journal-title":"ArXiv"},{"key":"e_1_3_2_49_1","article-title":"Investigating the effectiveness of task-agnostic prefix prompt for instruction following","author":"Ye S","year":"2023","unstructured":"Ye S, , Hwang H, , Yang S, et al. Investigating the effectiveness of task-agnostic prefix prompt for instruction following. ArXiv, 2023 arXiv:2302.14691.","journal-title":"ArXiv"},{"key":"e_1_3_2_50_1","doi-asserted-by":"crossref","unstructured":"Yuan D Rastogi E Naik G et al. A continued pretrained LLM approach for automatic medical note generation. In: Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics 2024; pp. 565\u2013571.","DOI":"10.18653\/v1\/2024.naacl-short.47"},{"key":"e_1_3_2_51_1","article-title":"Multimodal chain-of-thought reasoning in language models","author":"Zhang Z","year":"2023","unstructured":"Zhang Z, , Zhang A, , Li M, et al. Multimodal chain-of-thought reasoning in language models. ArXiv, 2023 arXiv:2302.00923.","journal-title":"ArXiv"}],"container-title":["Journal of Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/15578666251411390","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/15578666251411390","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/15578666251411390","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T11:48:33Z","timestamp":1793360913000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/15578666251411390"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,14]]},"references-count":50,"journal-issue":{"issue":"5-6","published-print":{"date-parts":[[2026,6,1]]}},"alternative-id":["10.1177\/15578666251411390"],"URL":"https:\/\/doi.org\/10.1177\/15578666251411390","relation":{},"ISSN":["1066-5277","1557-8666"],"issn-type":[{"value":"1066-5277","type":"print"},{"value":"1557-8666","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,14]]}}}