{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T06:07:34Z","timestamp":1788847654324,"version":"build-2803163510"},"reference-count":26,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T00:00:00Z","timestamp":1788825600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Digit. Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Post-market surveillance for adverse events of special interest is often constrained by key diagnostic elements being captured primarily in unstructured clinical notes and reports. This feasibility study evaluated large language model (LLM) and clinical quality language (CQL) methodologies to perform phenotype feature extraction and outcome determination on Fast Healthcare Interoperability Resources (FHIR) data.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      Two health systems identified patients with an mRNA COVID-19 vaccination and an inpatient myocarditis diagnosis within 42 days. Patient FHIR data was retrieved via a Health Information Exchange and converted into narrative text (\u201ctextified\u201d). Llama3.3 70B Instruct answered yes\/no Brighton Collaborative\u2013based myocarditis feature questions; answers were written back as FHIR Observation resources to enhance each patient bundle. Three methods were compared: simple CQL (structured data only), LLM (textified FHIR data), and enhanced CQL (CQL on enhanced bundles). Three clinicians provided gold-standard determinations. An optimization set (\n                      <jats:italic>n<\/jats:italic>\n                      \u2009=\u200910) supported pipeline refinement; performance was evaluated in a test set (\n                      <jats:italic>n<\/jats:italic>\n                      \u2009=\u200925).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>In the test set, simple CQL performed poorly for identifying cardiac and non-specific symptoms [Matthews Correlation Coefficient (MCC): 0.08; F1: 0.02] but excellently for cardiac biomarkers (MCC:0.83; F1: 0.88). The LLM achieved excellent performance identifying symptoms and imaging features (e.g., echocardiogram abnormalities MCC: 0.86) but weaker performance for biomarkers (MCC: 0.54). Enhanced CQL had excellent feature identification performance across most domains. Both LLM and enhanced CQL correctly determined the myocarditis outcome for 23 of 25 patients.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>In this small feasibility study, structured and unstructured FHIR data provided the computable phenotypes using an LLM with sufficient clinical information for myocarditis feature identification and outcome determination.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fdgth.2026.1903755","type":"journal-article","created":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T05:33:56Z","timestamp":1788845636000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["LLM and CQL computable phenotypes for myocarditis adverse event feature and outcome determinations using patient FHIR data: a feasibility study"],"prefix":"10.3389","volume":"8","author":[{"given":"Fraser W.","family":"Gaspar","sequence":"first","affiliation":[{"name":"Center for Transforming Health, MITRE Corp.","place":["McLean, VA, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Seth B.","family":"Roberts","sequence":"additional","affiliation":[{"name":"Center for Transforming Health, MITRE Corp.","place":["McLean, VA, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Francis X.","family":"Campion","sequence":"additional","affiliation":[{"name":"Center for Transforming Health, MITRE Corp.","place":["McLean, VA, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lynda","family":"Hoeksema","sequence":"additional","affiliation":[{"name":"Center for Transforming Health, MITRE Corp.","place":["McLean, VA, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jessica","family":"Kasirsky","sequence":"additional","affiliation":[{"name":"Center for Transforming Health, MITRE Corp.","place":["McLean, VA, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Raymond","family":"Duncan","sequence":"additional","affiliation":[{"name":"Departments of Enterprise Information Services and Pediatrics, Cedars-Sinai Health System","place":["Los Angeles, CA, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew","family":"Sonesen","sequence":"additional","affiliation":[{"name":"Departments of Enterprise Information Services and Pediatrics, Cedars-Sinai Health System","place":["Los Angeles, CA, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Renier","family":"Estiandan","sequence":"additional","affiliation":[{"name":"Departments of Enterprise Information Services and Pediatrics, Cedars-Sinai Health System","place":["Los Angeles, CA, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Khyzer B.","family":"Aziz","sequence":"additional","affiliation":[{"name":"Johns Hopkins Medicine","place":["Baltimore, MD, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julie A.","family":"Riddler","sequence":"additional","affiliation":[{"name":"Johns Hopkins Medicine","place":["Baltimore, MD, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jessica","family":"Zhou","sequence":"additional","affiliation":[{"name":"Center for Biologics Evaluation and Research, Food and Drug Administration","place":["Silver Spring, MD, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hussein","family":"Ezzeldin","sequence":"additional","affiliation":[{"name":"Center for Biologics Evaluation and Research, Food and Drug Administration","place":["Silver Spring, MD, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,8]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"777905","DOI":"10.3389\/fdgth.2021.777905","article-title":"The food and drug administration biologics effectiveness and safety initiative facilitates detection of vaccine administrations from unstructured data in medical records through natural language processing","volume":"3","author":"Deady","year":"2021","journal-title":"Front Digit Health"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1379973","DOI":"10.3389\/fpubh.2024.1379973","article-title":"Data quality and timeliness analysis for post-vaccination adverse event cases reported through healthcare data exchange to FDA BEST pilot platform","volume":"12","author":"Deady","year":"2024","journal-title":"Front Public Health"},{"key":"B3","doi-asserted-by":"publisher","first-page":"e54597","DOI":"10.2196\/54597","article-title":"A computable phenotype algorithm for postvaccination myocarditis\/pericarditis detection using real-world data: validation study","volume":"26","author":"Deady","year":"2024","journal-title":"J Med Internet Res"},{"key":"B4","doi-asserted-by":"publisher","first-page":"e49811","DOI":"10.2196\/49811","article-title":"Development of interoperable computable phenotype algorithms for adverse events of special interest to be used for biologics safety surveillance: validation study","volume":"10","author":"Holdefer","year":"2024","journal-title":"JMIR Public Health Surveill"},{"key":"B5","doi-asserted-by":"publisher","first-page":"5490","DOI":"10.3390\/jcm14155490","article-title":"Large language models for adverse drug events: a clinical perspective","volume":"14","author":"Zitu","year":"2025","journal-title":"J Clin Med"},{"key":"B6","doi-asserted-by":"publisher","first-page":"10.1056\/aics2400420","DOI":"10.1056\/aics2400420","article-title":"Large language models for more efficient reporting of hospital quality measures","volume":"1","author":"Boussina","year":"2024","journal-title":"NEJM AI"},{"key":"B7","doi-asserted-by":"publisher","first-page":"e0323535","DOI":"10.1371\/journal.pone.0323535","article-title":"A comparative analysis of large language models versus traditional information extraction methods for real-world evidence of patient symptomatology in acute and post-acute sequelae of SARS-CoV-2","volume":"20","author":"Thakkar","year":"2025","journal-title":"PLoS One"},{"key":"B8","doi-asserted-by":"publisher","first-page":"71","DOI":"10.3233\/SHTI250051","article-title":"Evaluation of the performance of a large language model to extract signs and symptoms from clinical notes","volume":"323","author":"Reategui-Rivera","year":"2025","journal-title":"Stud Health Technol Inform"},{"key":"B9","doi-asserted-by":"publisher","first-page":"ooaf089","DOI":"10.1093\/jamiaopen\/ooaf089","article-title":"A natural language processing pipeline for identifying pediatric long COVID symptoms and functional impacts in freeform clinical notes: a RECOVER study","volume":"8","author":"Bunnell","year":"2025","journal-title":"JAMIA Open"},{"key":"B10","year":"2026"},{"key":"B11","doi-asserted-by":"publisher","first-page":"1499","DOI":"10.1016\/j.vaccine.2021.11.074","article-title":"Myocarditis and pericarditis: case definition and guidelines for data collection, analysis, and presentation of immunization safety data","volume":"40","author":"Sexson Tejtel","year":"2022","journal-title":"Vaccine"},{"key":"B12","author":"Law","year":"2022"},{"key":"B13","year":"2024"},{"key":"B14","article-title":"Creatine kinase MB: diagnostic utility and limitations","volume-title":"StatPearls","author":"Kurapati","year":"2025"},{"key":"B15","doi-asserted-by":"publisher","first-page":"222","DOI":"10.1038\/s41746-024-01219-0","article-title":"A scoping review of large language model based approaches for information extraction from radiology reports","volume":"7","author":"Reichenpfader","year":"2024","journal-title":"Npj Digit Med"},{"key":"B16","doi-asserted-by":"publisher","first-page":"176","DOI":"10.1148\/rg.2016150080","article-title":"Natural language processing technologies in radiology research and clinical applications","volume":"36","author":"Cai","year":"2016","journal-title":"Radiographics"},{"key":"B17","year":""},{"key":"B18","year":"2026"},{"key":"B19","doi-asserted-by":"crossref","first-page":"1141","DOI":"10.1002\/(SICI)1097-0258(20000515)19:9<1141::AID-SIM479>3.0.CO;2-F","article-title":"Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians","volume":"19","author":"Carpenter","year":"2000","journal-title":"Statist Med"},{"key":"B20","doi-asserted-by":"publisher","first-page":"369","DOI":"10.1111\/j.1467-9868.2007.00593.x","article-title":"Bootstrapping clustered data","volume":"69","author":"Field","year":"2007","journal-title":"J R Stat Soc Ser B Stat Methodol"},{"key":"B21","year":""},{"key":"B22","year":""},{"key":"B23","doi-asserted-by":"publisher","first-page":"3015","DOI":"10.1001\/jama.285.23.3015","article-title":"National health information PrivacyRegulations under the health insurance portability and accountability act","volume":"285","author":"Gostin","year":"2001","journal-title":"JAMA"},{"key":"B24","doi-asserted-by":"publisher","first-page":"e68291","DOI":"10.2196\/68291","article-title":"Applications of federated large language model for adverse drug reactions prediction: scoping review","volume":"27","author":"Guo","year":"2025","journal-title":"J Med Internet Res"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1007\/s40119-026-00453-9","article-title":"Concordance of large language model recommendations with multidisciplinary heart team decisions in coronary revascularization and aortic valve intervention: a systematic review and pooled analysis","author":"Rouhi","year":"2026","journal-title":"Cardiol Ther"},{"key":"B26","doi-asserted-by":"publisher","first-page":"e101139","DOI":"10.1136\/bmjhci-2024-101139","article-title":"Large language models for data extraction from unstructured and semi-structured electronic health records: a multiple model performance evaluation","volume":"32","author":"Ntinopoulos","year":"2025","journal-title":"BMJ Health Care Inform"}],"container-title":["Frontiers in Digital Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2026.1903755\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T05:33:57Z","timestamp":1788845637000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2026.1903755\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,8]]},"references-count":26,"alternative-id":["10.3389\/fdgth.2026.1903755"],"URL":"https:\/\/doi.org\/10.3389\/fdgth.2026.1903755","relation":{},"ISSN":["2673-253X"],"issn-type":[{"value":"2673-253X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,8]]},"article-number":"1903755"}}