{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T23:01:51Z","timestamp":1790809311955,"version":"4.1.0"},"reference-count":44,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2025,9,4]],"date-time":"2025-09-04T00:00:00Z","timestamp":1756944000000},"content-version":"vor","delay-in-days":2,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"National Institute of Health T32 program","award":["GR0125656"],"award-info":[{"award-number":["GR0125656"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,9,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objectives<\/jats:title>\n                  <jats:p>Unstructured data, such as procedure notes, contain valuable medical information that is frequently underutilized due to the labor-intensive nature of data extraction. This study aims to develop a generative artificial intelligence (GenAI) pipeline using an open-source Large Language Model (LLM) with built-in guardrails and a retry mechanism to extract data from unstructured right heart catheterization (RHC) notes while minimizing errors, including hallucinations.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>A total of 220 RHC notes were randomly selected for pipeline development and 200 for validation from the Pulmonary Vascular Disease Registry. The pipeline comprised three main components: the Engineered Preload Framework (EPF), which integrated schemas and instructions; the LLM module, enhanced by reasoning capabilities; and the validation and retry mechanism, which ensured data accuracy through iterative self-correction. A clinical expert manually extracted data from the validation cohort to establish the ground truth. Pipeline performance was evaluated using precision, recall, and F1 score. Additionally, the dataset was stratified into quartiles to assess the pipeline\u2019s ability to handle varying levels of data availability.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The pipeline achieved 99.0% precision, 85.0% recall, and a 91.5% F1 score, with an overall accuracy of 90% when evaluated at the note level. The most common error was missed values (5.2%), while hallucinations were the least frequent (&amp;lt;0.01%).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion and Conclusion<\/jats:title>\n                  <jats:p>This study demonstrates the feasibility of a robust GenAI pipeline for automating structured data extraction from unstructured RHC procedure notes. The approach highlights the potential of LLMs in medical data mining, improving research efficiency and clinical applications.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamiaopen\/ooaf097","type":"journal-article","created":{"date-parts":[[2025,8,19]],"date-time":"2025-08-19T11:34:00Z","timestamp":1755603240000},"source":"Crossref","is-referenced-by-count":15,"title":["Generative artificial intelligence for automated data extraction from unstructured medical text"],"prefix":"10.1093","volume":"8","author":[{"given":"Nam","family":"Dao","sequence":"first","affiliation":[{"name":"Division of Pulmonary and Critical Care, Brigham and Women\u2019s Hospital , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Luisa","family":"Quesada","sequence":"additional","affiliation":[{"name":"Division of Pulmonary and Critical Care, Brigham and Women\u2019s Hospital , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1544-3280","authenticated-orcid":false,"given":"Syed Moin","family":"Hassan","sequence":"additional","affiliation":[{"name":"Division of Sleep Medicine, Brigham and Women\u2019s Hospital , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Monica Iturrioz","family":"Campo","sequence":"additional","affiliation":[{"name":"Department of Radiology, Brigham and Women\u2019s Hospital , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shelsey","family":"Johnson","sequence":"additional","affiliation":[{"name":"Division of Pulmonary and Critical Care, Brigham and Women\u2019s Hospital , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Suchandra","family":"Ghose","sequence":"additional","affiliation":[{"name":"Division of Pulmonary and Critical Care, Brigham and Women\u2019s Hospital , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ra\u00fal","family":"San Jos\u00e9 Est\u00e9par","sequence":"additional","affiliation":[{"name":"Department of Radiology, Brigham and Women\u2019s Hospital , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aaron","family":"Waxman","sequence":"additional","affiliation":[{"name":"Division of Pulmonary and Critical Care, Brigham and Women\u2019s Hospital , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"George","family":"Washko","sequence":"additional","affiliation":[{"name":"Division of Pulmonary and Critical Care, Brigham and Women\u2019s Hospital , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Farbod N","family":"Rahaghi","sequence":"additional","affiliation":[{"name":"Division of Pulmonary and Critical Care, Brigham and Women\u2019s Hospital , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,9,4]]},"reference":[{"key":"2025090419494279200_ooaf097-B1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4258\/hir.2019.25.1.1","article-title":"Managing unstructured big data in healthcare system","volume":"25","author":"Kong","year":"2019","journal-title":"Healthc Inform Res"},{"key":"2025090419494279200_ooaf097-B2","doi-asserted-by":"crossref","first-page":"e2425981","DOI":"10.1001\/jamanetworkopen.2024.25981","article-title":"Use of generative AI to identify helmet status among patients with micromobility-related injuries from unstructured clinical notes","volume":"7","author":"Burford","year":"2024","journal-title":"JAMA Netw Open"},{"key":"2025090419494279200_ooaf097-B3","author":"Malmasi","year":"2018"},{"key":"2025090419494279200_ooaf097-B4","doi-asserted-by":"crossref","first-page":"106649","DOI":"10.1016\/j.compbiomed.2023.106649","article-title":"Natural language processing in electronic health records in relation to healthcare decision-making: a systematic review","volume":"155","author":"Hossain","year":"2023","journal-title":"Comput Biol Med"},{"key":"2025090419494279200_ooaf097-B5","first-page":"4161","author":"Agarwal"},{"key":"2025090419494279200_ooaf097-B6","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1038\/s41746-022-00730-6","article-title":"A survey on clinical natural language processing in the United Kingdom from 2007 to 2022","volume":"5","author":"Wu","year":"2022","journal-title":"NPJ Digit Med"},{"key":"2025090419494279200_ooaf097-B7","author":"Vaswani"},{"key":"2025090419494279200_ooaf097-B8","doi-asserted-by":"crossref","first-page":"192","DOI":"10.3390\/fi15060192","article-title":"ChatGPT and Open-AI models: a preliminary review","volume":"15","author":"Roumeliotis","year":"2023","journal-title":"Future Internet"},{"key":"2025090419494279200_ooaf097-B9","doi-asserted-by":"crossref","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of chatgpt on USMLE: potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLOS Digit Health"},{"key":"2025090419494279200_ooaf097-B10","doi-asserted-by":"crossref","first-page":"e2440969","DOI":"10.1001\/jamanetworkopen.2024.40969","article-title":"Large language model influence on diagnostic reasoning","volume":"7","author":"Goh","year":"2024","journal-title":"JAMA Netw Open"},{"key":"2025090419494279200_ooaf097-B11","author":"Pal"},{"key":"2025090419494279200_ooaf097-B12","doi-asserted-by":"crossref","first-page":"AIdbp2300110","DOI":"10.1056\/AIdbp2300110","article-title":"CORAL: expert-curated oncology reports to advance language model inference","volume":"1","author":"Sushil","year":"2024","journal-title":"Nejm Ai"},{"key":"2025090419494279200_ooaf097-B13","author":"Li"},{"key":"2025090419494279200_ooaf097-B14","author":"Use of Natural Language Processing and Machine Learning to Predict Severe COPD Exacerbations | C23","year":"2020"},{"key":"2025090419494279200_ooaf097-B15","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1016\/j.annemergmed.2024.01.036","article-title":"Development and validation of a natural language processing model to identify low-risk pulmonary embolism in real time to facilitate safe outpatient management","volume":"84","author":"Amin","year":"2024","journal-title":"Ann Emerg Med"},{"key":"2025090419494279200_ooaf097-B16","doi-asserted-by":"crossref","first-page":"2991","DOI":"10.1182\/bloodadvances.2023012200","article-title":"Machine learning natural language processing for identifying venous thromboembolism: systematic review and meta-analysis","volume":"8","author":"Lam","year":"2024","journal-title":"Blood Adv"},{"key":"2025090419494279200_ooaf097-B17","doi-asserted-by":"crossref","first-page":"e872","DOI":"10.1016\/S2589-7500(23)00177-2","article-title":"Machine learning and deep learning predictive models for long-term prognosis in patients with chronic obstructive pulmonary disease: a systematic review and meta-analysis","volume":"5","author":"Smith","year":"2023","journal-title":"Lancet Digit Health"},{"key":"2025090419494279200_ooaf097-B18","doi-asserted-by":"crossref","first-page":"909","DOI":"10.1136\/heartjnl-2021-319769","article-title":"Systematic review of current natural language processing methods and applications in cardiology","volume":"108","author":"Turchioe","year":"2022","journal-title":"Heart"},{"key":"2025090419494279200_ooaf097-B19","author":"Grattafiori","year":"2024"},{"key":"2025090419494279200_ooaf097-B20","doi-asserted-by":"crossref","first-page":"e012347","DOI":"10.1161\/CIRCIMAGING.120.012347","article-title":"Loss of pulmonary vascular volume as a predictor of right ventricular dysfunction and mortality in acute pulmonary embolism","volume":"14","author":"Minhas","year":"2021","journal-title":"Circ Cardiovasc Imaging"},{"key":"2025090419494279200_ooaf097-B21","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1164\/rccm.202306-1072OC","article-title":"Implications of mean pulmonary arterial wedge pressure trajectories in pulmonary arterial hypertension","volume":"209","author":"Harder","year":"2024","journal-title":"Am J Respir Crit Care Med"},{"key":"2025090419494279200_ooaf097-B22","doi-asserted-by":"crossref","first-page":"00201","DOI":"10.1183\/23120541.00201-2024","article-title":"Vasoreactivity and inhaled treprostinil response in interstitial lung disease pulmonary hypertension","volume":"10","author":"Harder","year":"2024","journal-title":"ERJ Open Res"},{"key":"2025090419494279200_ooaf097-B23","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1038\/s41746-024-01258-7","article-title":"A framework for human evaluation of large language models in healthcare derived from literature review","volume":"7","author":"Tam","year":"2024","journal-title":"NPJ Digit Med"},{"key":"2025090419494279200_ooaf097-B24","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1007\/s00134-024-07745-5","article-title":"Leveraging the power of routinely collected ICU data","volume":"51","author":"Lijovi\u0107","year":"2025","journal-title":"Intensive Care Med"},{"key":"2025090419494279200_ooaf097-B25","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1038\/s41586-023-06647-8","article-title":"Role play with large language models","volume":"623","author":"Shanahan","year":"2023","journal-title":"Nature"},{"key":"2025090419494279200_ooaf097-B26","author":"Zhang","year":"2023"},{"key":"2025090419494279200_ooaf097-B27","first-page":"24824","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","author":"Wei","year":"2022","journal-title":"Adv Neur Inf Process Syst"},{"key":"2025090419494279200_ooaf097-B28","author":"Jaech","year":"2024"},{"key":"2025090419494279200_ooaf097-B29","author":"Pydantic"},{"key":"2025090419494279200_ooaf097-B30","first-page":"46534","article-title":"Self-refine: Iterative refinement with self-feedback","volume":"36","author":"Madaan","year":"2024","journal-title":"Adv Neur Inf Process Syst"},{"key":"2025090419494279200_ooaf097-B31","author":"Gou","year":"2023"},{"key":"2025090419494279200_ooaf097-B32","first-page":"1417","article-title":"When can llms actually correct their own mistakes? A critical survey of self-correction of llms","volume":"12","author":"Kamoi","year":"2024","journal-title":"Trans Assoc Comput Ling"},{"key":"2025090419494279200_ooaf097-B33","author":"Wang"},{"key":"2025090419494279200_ooaf097-B34","author":"Kwon"},{"key":"2025090419494279200_ooaf097-B35","author":"Python Software Foundation","year":"2021"},{"key":"2025090419494279200_ooaf097-B36","author":"Team R","year":"2024"},{"key":"2025090419494279200_ooaf097-B37","author":"Wan","year":"2025"},{"key":"2025090419494279200_ooaf097-B38","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1016\/0895-4356(90)90158-L","article-title":"High agreement but low kappa: I. the problems of two paradoxes","volume":"43","author":"Feinstein","year":"1990","journal-title":"J Clin Epidemiol"},{"key":"2025090419494279200_ooaf097-B39","doi-asserted-by":"publisher","author":"Krippendorff","DOI":"10.4135\/9781071878781"},{"key":"2025090419494279200_ooaf097-B40","author":"Xia"},{"key":"2025090419494279200_ooaf097-B41","author":"Meta","year":"2024"},{"key":"2025090419494279200_ooaf097-B42","author":"Chen"},{"key":"2025090419494279200_ooaf097-B43","doi-asserted-by":"publisher","author":"Aronson","DOI":"10.1056\/AIcs2400245"},{"key":"2025090419494279200_ooaf097-B44","doi-asserted-by":"crossref","first-page":"2171","DOI":"10.1056\/NEJMp2102616","article-title":"HIPAA and the leak of \u201cdeidentified\u201d EHR data","volume":"384","author":"Mandl","year":"2021","journal-title":"N Engl J Med"}],"container-title":["JAMIA Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/8\/5\/ooaf097\/64212221\/ooaf097.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/8\/5\/ooaf097\/64212221\/ooaf097.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,4]],"date-time":"2025-09-04T23:49:50Z","timestamp":1757029790000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article\/doi\/10.1093\/jamiaopen\/ooaf097\/8248163"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,2]]},"references-count":44,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,9,2]]}},"URL":"https:\/\/doi.org\/10.1093\/jamiaopen\/ooaf097","relation":{},"ISSN":["2574-2531"],"issn-type":[{"value":"2574-2531","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,10]]},"published":{"date-parts":[[2025,9,2]]},"article-number":"ooaf097"}}