{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T09:47:37Z","timestamp":1791884857475,"version":"4.3.4"},"reference-count":32,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T00:00:00Z","timestamp":1789430400000},"content-version":"vor","delay-in-days":11,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,9,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Aims<\/jats:title>\n                    <jats:p>Immunotherapy with immune checkpoint inhibitors (ICIs) is an effective treatment for many cancers, but it can induce immune-related adverse events (irAEs). Among those, myocarditis is a severe complication associated with high morbidity and mortality. To support the rapidly emerging research, labour-intensive manual data extraction is often needed. Recent studies adopted large language models (LLMs) to systematically process clinical notes and identify patients with irAEs. However, research gaps persist: (i) For ICI-associated myocarditis, the applicability and scalability of LLMs have not been systematically evaluated; (ii) most existing studies focused on patient-level irAE detection while omitting the clinical nuances; and (iii) cutting-edge LLM techniques have not been adopted. Our aim was to evaluate an LLM approach to extract ICI-associated myocarditis evidence, including inflammatory infiltrate, life-threatening arrhythmias, heart failure, and stroke, from clinical notes.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods and results<\/jats:title>\n                    <jats:p>We collected clinical notes of patients with ICI-associated myocarditis at a single centre to develop and evaluate LLM-based methods that convert free-text clinical notes into structured data with entities (e.g. diagnosis, treatment, and imaging) and attributes (e.g. date, assertion, and status). We systematically evaluated three techniques: LLM reasoning, context engineering, and hierarchical prompting against ground truth created by a cardio-oncologist. Our proposed sentence-based context engineering and hierarchical prompting method, with a reasoning LLM, is significantly more accurate than the research trainees (F1 score of 0.6332 vs. 0.4933, P &amp;lt; 0.001), faster (37\u2005min vs. 72\u2005h per 1000 notes), and more cost-effective ($2.04 USD per 1000 notes).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>We proposed an optimized method that integrates hierarchical prompting, context engineering, and a reasoning LLM for ICI-associated myocarditis evidence extraction to support physicians and researchers in rapid clinical data collection for research to make diagnostic and treatment inferences.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/ehjdh\/ztag145","type":"journal-article","created":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T08:22:38Z","timestamp":1789460558000},"source":"Crossref","is-referenced-by-count":0,"title":["Hierarchical prompting with reasoning large language models for immune checkpoint inhibitor-associated myocarditis evidence extraction pilot study"],"prefix":"10.1093","volume":"7","author":[{"given":"Enshuo","family":"Hsu","sequence":"first","affiliation":[{"name":"Enterprise Development and Integration, University of Texas MD Anderson Cancer Center , Houston, TX ,","place":["USA"]},{"name":"McWilliams School of Biomedical Informatics, University of Texas Health Science Center at Houston , Houston, TX ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sheng-Chieh","family":"Lu","sequence":"additional","affiliation":[{"name":"Department of Cardiology, University of Texas MD Anderson Cancer Center , 1515 Holcombe Blvd, Unit 1451, Houston, TX ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sara","family":"Ebrahimi","sequence":"additional","affiliation":[{"name":"Department of Cardiology, University of Texas MD Anderson Cancer Center , 1515 Holcombe Blvd, Unit 1451, Houston, TX ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Noah","family":"Beinart","sequence":"additional","affiliation":[{"name":"Department of Cardiology, University of Texas MD Anderson Cancer Center , 1515 Holcombe Blvd, Unit 1451, Houston, TX ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Keila C","family":"Ostos-Mendoza","sequence":"additional","affiliation":[{"name":"Department of Cardiology, University of Texas MD Anderson Cancer Center , 1515 Holcombe Blvd, Unit 1451, Houston, TX ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6147-9591","authenticated-orcid":false,"given":"Anita","family":"Deswal","sequence":"additional","affiliation":[{"name":"Department of Cardiology, University of Texas MD Anderson Cancer Center , 1515 Holcombe Blvd, Unit 1451, Houston, TX ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6525-5213","authenticated-orcid":false,"given":"Kirk","family":"Roberts","sequence":"additional","affiliation":[{"name":"McWilliams School of Biomedical Informatics, University of Texas Health Science Center at Houston , Houston, TX ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Trey","family":"Kell","sequence":"additional","affiliation":[{"name":"Enterprise Development and Integration, University of Texas MD Anderson Cancer Center , Houston, TX ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7252-9702","authenticated-orcid":false,"given":"Jeff","family":"Jin","sequence":"additional","affiliation":[{"name":"Enterprise Development and Integration, University of Texas MD Anderson Cancer Center , Houston, TX ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7565-1797","authenticated-orcid":false,"given":"Nicolas L","family":"Palaskas","sequence":"additional","affiliation":[{"name":"Department of Cardiology, University of Texas MD Anderson Cancer Center , 1515 Holcombe Blvd, Unit 1451, Houston, TX ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,9,15]]},"reference":[{"key":"2026101305113047700_ztag145-B1","doi-asserted-by":"crossref","first-page":"2352","DOI":"10.1002\/ijc.35347","article-title":"How many people in the US are eligible for and respond to checkpoint inhibitors: an empirical analysis","volume":"156","author":"Haslam","year":"2025","journal-title":"Int J Cancer"},{"key":"2026101305113047700_ztag145-B2","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1038\/s41416-024-02887-1","article-title":"Cancer and treatment specific incidence rates of immune-related adverse events induced by immune checkpoint inhibitors: a systematic review","volume":"132","author":"Jayathilaka","year":"2024","journal-title":"Br J Cancer"},{"key":"2026101305113047700_ztag145-B3","doi-asserted-by":"crossref","first-page":"1721","DOI":"10.1001\/jamaoncol.2018.3923","article-title":"Fatal toxic effects associated with immune checkpoint inhibitors: a systematic review and meta-analysis","volume":"4","author":"Wang","year":"2018","journal-title":"JAMA Oncol"},{"key":"2026101305113047700_ztag145-B4","doi-asserted-by":"crossref","DOI":"10.2196\/65984","article-title":"Large language model applications for health information extraction in oncology: scoping review","volume":"11","author":"Chen","year":"2025","journal-title":"JMIR Cancer"},{"key":"2026101305113047700_ztag145-B5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10462-025-11316-5","article-title":"A survey of NLP methods for oncology in the past decade with a focus on cancer registry applications","volume":"58","author":"Hands","year":"2025","journal-title":"Artif Intell Rev"},{"key":"2026101305113047700_ztag145-B6","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1186\/s12859-023-05480-0","article-title":"Extracting cancer concepts from clinical notes using natural language processing: a systematic review","volume":"24","author":"Gholipour","year":"2023","journal-title":"BMC Bioinformatics"},{"key":"2026101305113047700_ztag145-B7","doi-asserted-by":"crossref","first-page":"5487","DOI":"10.1038\/s41598-025-89754-y","article-title":"Machine learning tools match physician accuracy in multilingual text annotation","volume":"15","author":"Zielonka","year":"2025","journal-title":"Sci Rep"},{"key":"2026101305113047700_ztag145-B8","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1053\/j.gastro.2023.12.019","article-title":"A comparison of a large language model vs manual chart review for the extraction of data elements from the electronic health record","volume":"166","author":"Ge","year":"2024","journal-title":"Gastroenterology"},{"key":"2026101305113047700_ztag145-B9","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1002\/path.6232","article-title":"Extracting structured information from unstructured histopathology reports using generative pre-trained transformer 4 (GPT-4)","volume":"262","author":"Truhn","year":"2024","journal-title":"J Pathol"},{"key":"2026101305113047700_ztag145-B10","doi-asserted-by":"crossref","first-page":"11176","DOI":"10.1200\/JCO.2025.43.16_suppl.11176","article-title":"Use of large language models to extract cancer diagnosis, histology, grade, and staging from unstructured electronic health records","volume":"43","author":"Namasivayam","year":"2025","journal-title":"J Clin Oncol"},{"key":"2026101305113047700_ztag145-B11","doi-asserted-by":"crossref","first-page":"105321","DOI":"10.1016\/j.ijmedinf.2023.105321","article-title":"Zero-shot information extraction from radiological reports using ChatGPT","volume":"183","author":"Hu","year":"2024","journal-title":"Int J Med Inf"},{"key":"2026101305113047700_ztag145-B12","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1038\/s41746-025-01780-2","article-title":"Large language model trained on clinical oncology data predicts cancer progression","volume":"8","author":"Zhu","year":"2025","journal-title":"Npj Digit Med"},{"key":"2026101305113047700_ztag145-B13","doi-asserted-by":"crossref","first-page":"4134","DOI":"10.1200\/JCO.24.00326","article-title":"Enhancing precision in detecting severe immune-related adverse events: comparative analysis of large language models and international classification of disease codes in patient records","volume":"42","author":"Sun","year":"2024","journal-title":"J Clin Oncol"},{"key":"2026101305113047700_ztag145-B14","doi-asserted-by":"crossref","first-page":"1558","DOI":"10.1200\/JCO.2025.43.16_suppl.1558","article-title":"Use of a large language model (LLM) for pan-cancer automated detection of anti-cancer therapy toxicities and translational toxicity research","volume":"43","author":"Bakouny","year":"2025","journal-title":"J Clin Oncol"},{"key":"2026101305113047700_ztag145-B15","doi-asserted-by":"crossref","first-page":"106227","DOI":"10.1016\/j.ebiom.2026.106227","article-title":"irAE-GPT: leveraging large language models to identify immune-related adverse events in electronic health records and clinical trial datasets","volume":"127","author":"Bejan","year":"2025","journal-title":"EBioMedicine"},{"key":"2026101305113047700_ztag145-B16","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1093\/eurheartj\/ehab674","article-title":"Defining cardiovascular toxicities of cancer therapies: an International Cardio-Oncology Society (IC-OS) consensus statement","volume":"43","author":"Herrmann","year":"2022","journal-title":"Eur Heart J"},{"key":"2026101305113047700_ztag145-B17","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1161\/CIRCULATIONAHA.118.034497","article-title":"Myocarditis in the setting of cancer therapeutics: proposed case definitions for emerging clinical syndromes in cardio-oncology","volume":"140","author":"Bonaca","year":"2019","journal-title":"Circulation"},{"key":"2026101305113047700_ztag145-B18","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1197\/jamia.M1733","article-title":"Agreement, the F-measure, and reliability in information retrieval","volume":"12","author":"Hripcsak","year":"2005","journal-title":"J Am Med Inform Assoc"},{"key":"2026101305113047700_ztag145-B19","first-page":"144","article-title":"Building gold standard corpora for medical natural language processing tasks","volume":"2012","author":"Deleger","year":"2012","journal-title":"AMIA Annu Symp Proc"},{"key":"2026101305113047700_ztag145-B20","author":"Tkachenko"},{"key":"2026101305113047700_ztag145-B21","author":"Agrawal"},{"key":"2026101305113047700_ztag145-B22","doi-asserted-by":"publisher","author":"Agarwal","DOI":"10.48550\/arXiv.2508.10925"},{"key":"2026101305113047700_ztag145-B23"},{"key":"2026101305113047700_ztag145-B24","doi-asserted-by":"publisher","author":"Sellergren","DOI":"10.48550\/arXiv.2507.05201"},{"key":"2026101305113047700_ztag145-B25","doi-asserted-by":"crossref","first-page":"ooaf012","DOI":"10.1093\/jamiaopen\/ooaf012","article-title":"LLM-IE: a python package for biomedical generative information extraction with large language models","volume":"8","author":"Hsu","year":"2025","journal-title":"JAMIA Open"},{"key":"2026101305113047700_ztag145-B26","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1093\/jamia\/ocz166","article-title":"2018 n2c2 shared task on adverse drug events and medication extraction in electronic health records","volume":"27","author":"Henry","year":"2020","journal-title":"J Am Med Inform Assoc"},{"key":"2026101305113047700_ztag145-B27","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1214\/ss\/1032280214","article-title":"Bootstrap confidence intervals","volume":"11","author":"DiCiccio","year":"1996","journal-title":"Stat Sci"},{"key":"2026101305113047700_ztag145-B28","first-page":"611","article-title":"Efficient memory management for large language model serving with pagedattention","author":"Kwon","year":"2023","journal-title":"In: Proceedings of the 29th symposium on operating systems principles"},{"key":"2026101305113047700_ztag145-B29"},{"key":"2026101305113047700_ztag145-B30","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1200\/CCI.20.00109","article-title":"Automated identification of patients with immune-related adverse events from clinical notes using word embedding and machine learning","volume":"5","author":"Gupta","year":"2021","journal-title":"JCO Clin Cancer Inform"},{"key":"2026101305113047700_ztag145-B31","article-title":"Identification and characterization of immune checkpoint inhibitor\u2013induced toxicities from electronic health records using natural language processing","volume":"8","author":"Barman","year":"2024","journal-title":"JCO Clin Cancer Inform"},{"key":"2026101305113047700_ztag145-B32","author":"Hsu"}],"container-title":["European Heart Journal - Digital Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/ehjdh\/advance-article-pdf\/doi\/10.1093\/ehjdh\/ztag145\/71083263\/ztag145.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ehjdh\/article-pdf\/7\/8\/ztag145\/71083263\/ztag145.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ehjdh\/article-pdf\/7\/8\/ztag145\/71083263\/ztag145.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T09:11:42Z","timestamp":1791882702000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/ehjdh\/article\/doi\/10.1093\/ehjdh\/ztag145\/8795257"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,4]]},"references-count":32,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2026,9,4]]}},"URL":"https:\/\/doi.org\/10.1093\/ehjdh\/ztag145","relation":{},"ISSN":["2634-3916"],"issn-type":[{"value":"2634-3916","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,10]]},"published":{"date-parts":[[2026,9,4]]},"article-number":"ztag145"}}