{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,21]],"date-time":"2026-09-21T19:02:32Z","timestamp":1790017352161,"version":"4.0.1"},"reference-count":30,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Form Res"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Inadequate postcare patient education contributes to preventable readmissions and adverse outcomes that disproportionately affect medically complex, high-need communities. Large language models (LLMs) show promise for generating personalized, plain-language patient education at scale. However, existing LLM evaluation frameworks prioritize technical accuracy over patient accessibility and alignment with health literacy, and few explicitly account for the attitudinal influences that clinician evaluators may introduce into the rating process. In this viewpoint, we introduce an evaluation framework that pairs the Technology Acceptance Model (TAM) with the Medical Condition Regard Scale (MCRS). We call it the TAM-MCRS LLM evaluation framework, a novel clinician-centered approach for comparing which LLMs produce the highest-quality postcare patient education across accuracy, appropriateness, clarity, and completeness. We intend for this framework to be used to evaluate LLM-generated patient education outputs through a 2-arm design that pairs an expert clinician panel with automated assessment methods, allowing for interarm comparison using clinical vignettes while accounting for measured evaluator attitudinal variance. The framework was developed through the National Institutes of Health Artificial Intelligence\/Machine Learning Consortium to Advance Health Equity and Researcher Diversity (AIM-AHEAD) Clinicians Leading Ingenuity IN AI Quality (CLINAQ) fellowship program, in partnership with Ochsner Health and Xavier University of Louisiana. The TAM-MCRS framework integrates two theoretical lenses. TAM maps perceived usefulness onto accuracy and completeness, and perceived ease of use onto clarity and appropriateness. Clinicians rate each output with a TAM-based questionnaire, and we then administer the MCRS as a postscoring attitudinal covariate to see whether their regard for the conditions represented in the vignettes influences those ratings. Together, the 2 lenses are intended to produce evidence that is objective, theoretically grounded, clinically realistic, and disparity-responsive. Implications for clinician informaticists, health system governance, and responsible AI deployment are discussed. This viewpoint reflects the authors\u2019 position and is written for clinician informaticists, health system AI governance leaders, implementation scientists, and investigators evaluating LLM-generated patient education.<\/jats:p>","DOI":"10.2196\/100423","type":"journal-article","created":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T18:55:07Z","timestamp":1787770507000},"page":"e100423-e100423","source":"Crossref","is-referenced-by-count":0,"title":["A Clinician-Centered Evaluation Framework for Large Language Models in Patient Education: Integrating the Technology Acceptance Model and Medical Condition Regard Scale"],"prefix":"10.2196","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7568-8442","authenticated-orcid":false,"given":"Davis","family":"Austria","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7697-0113","authenticated-orcid":false,"given":"Grace Lord","family":"Williams","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0469-5925","authenticated-orcid":false,"given":"Christopher","family":"Girardo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9418-5346","authenticated-orcid":false,"given":"Micheal Olaolu","family":"Arowolo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1489-0887","authenticated-orcid":false,"given":"Meenakshi","family":"Mishra","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-9443-7655","authenticated-orcid":false,"given":"Charlene","family":"Pope","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8821-7295","authenticated-orcid":false,"given":"R Neal","family":"Axon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-0557-5212","authenticated-orcid":false,"given":"Jason Bradley","family":"Hill","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,9,21]]},"reference":[{"key":"R1","unstructured":"Weiss AJ Jiang HJ . Overview of clinical conditions with frequent and costly hospital readmissions by payer, 2018. In: Healthcare Cost and Utilization Project (HCUP) Statistical Briefs. Agency for Healthcare Research and Quality (US); 2021."},{"issue":"3","key":"R2","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1016\/j.ajem.2015.06.001","article-title":"Costs of ED episodes of care in the United States","volume":"34","author":"Galarraga","journal-title":"Am J Emerg Med"},{"issue":"3","key":"R3","doi-asserted-by":"publisher","DOI":"10.1136\/bmjoq-2021-001419","article-title":"Empowering patients: simplifying discharge instructions","volume":"10","author":"DeSai","journal-title":"BMJ Open Qual"},{"issue":"11","key":"R4","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.74398","article-title":"Analyzing the effectiveness of AI-generated patient education materials: a comparative study of ChatGPT and Google Gemini","volume":"16","author":"Karnan","journal-title":"Cureus"},{"issue":"1","key":"R5","doi-asserted-by":"publisher","first-page":"601","DOI":"10.1007\/s12070-024-05161-1","article-title":"Comparative analysis of artificial intelligence platforms in generating post-operative instructions for rhinologic surgery","volume":"77","author":"Shaari","journal-title":"Indian J Otolaryngol Head Neck Surg"},{"issue":"6","key":"R6","doi-asserted-by":"publisher","first-page":"1751","DOI":"10.1002\/ohn.927","article-title":"Enhancing health literacy: evaluating the readability of patient handouts revised by ChatGPT\u2019s large language model","volume":"171","author":"Swisher","journal-title":"Otolaryngol Head Neck Surg"},{"issue":"6","key":"R7","doi-asserted-by":"publisher","first-page":"435","DOI":"10.1016\/j.prro.2019.06.005","article-title":"Readability of patient education materials in radiation oncology-are we improving?","volume":"9","author":"Rooney","journal-title":"Pract Radiat Oncol"},{"key":"R8","doi-asserted-by":"publisher","DOI":"10.2196\/50898","article-title":"Roles of health literacy in relation to social determinants of health and recommendations for informatics-based interventions: systematic review","volume":"16","author":"Bindhu","journal-title":"Online J Public Health Inform"},{"key":"R9","doi-asserted-by":"crossref","unstructured":"Bender EM Gebru T McMillan-Major A Shmitchell S . On the dangers of stochastic parrots: can language models be too big? In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery; 2021. [doi: 10.1145\/3442188.3445922]","DOI":"10.1145\/3442188.3445922"},{"issue":"1","key":"R10","doi-asserted-by":"publisher","DOI":"10.1038\/s43856-024-00717-2","article-title":"Current applications and challenges in large language models for patient care: a systematic review","volume":"5","author":"Busch","journal-title":"Commun Med (Lond)"},{"issue":"10","key":"R11","doi-asserted-by":"publisher","DOI":"10.1016\/j.patter.2021.100347","article-title":"Addressing bias in big data and AI for health care: a call for open science","volume":"2","author":"Norori","journal-title":"Patterns (N Y)"},{"issue":"2","key":"R12","doi-asserted-by":"publisher","first-page":"010318","DOI":"10.7189\/jogh.09.020318","article-title":"Artificial intelligence and algorithmic bias: implications for health systems","volume":"9","author":"Panch","journal-title":"J Glob Health"},{"issue":"4","key":"R13","doi-asserted-by":"publisher","first-page":"434","DOI":"10.1016\/j.amepre.2010.12.019","article-title":"Geographic distribution of diagnosed diabetes in the U.S.: a diabetes belt","volume":"40","author":"Barker","journal-title":"Am J Prev Med"},{"key":"R14","unstructured":"National diabetes statistics report. U.S. Centers for Disease Control and Prevention; 2026. URL: https:\/\/www.cdc.gov\/diabetes\/php\/data-research\/index.html [Accessed 02-09-2026]"},{"key":"R15","unstructured":"United States cancer statistics: data visualizations. U.S. Centers for Disease Control and Prevention. URL: https:\/\/gis.cdc.gov\/Cancer\/USCS\/#\/ [Accessed 06-05-2026]"},{"key":"R16","unstructured":"Cancer incidence in Louisiana by census tract - 2023. LSU Health New Orleans. 2023. URL: https:\/\/sph.lsuhsc.edu\/louisiana-tumor-registry\/data-usestatistics\/monographs-publications\/cancer-incidence-in-louisiana-by-census-tract-2023\/ [Accessed 06-05-2026]"},{"issue":"5","key":"R17","doi-asserted-by":"publisher","first-page":"1083","DOI":"10.1158\/2767-9764.CRC-26-0163","article-title":"Association between diagnostic history and cancer incidence within 5 years: a real-world observational analysis","volume":"6","author":"Alam","journal-title":"Cancer Res Commun"},{"issue":"3","key":"R18","doi-asserted-by":"publisher","first-page":"319","DOI":"10.2307\/249008","article-title":"Perceived usefulness, perceived ease of use, and user acceptance of information technology","volume":"13","author":"Davis","journal-title":"MIS Q"},{"issue":"3","key":"R19","doi-asserted-by":"publisher","first-page":"425","DOI":"10.2307\/30036540","article-title":"User acceptance of information technology: toward a unified view","volume":"27","author":"Venkatesh","journal-title":"MIS Q"},{"key":"R20","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2024.105556","article-title":"Health information systems with technology acceptance model approach: a systematic review","volume":"190","author":"Tetik","journal-title":"Int J Med Inform"},{"issue":"3","key":"R21","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1097\/00001888-200203000-00017","article-title":"The medical condition regard scale: measuring reactions to diagnoses","volume":"77","author":"Christison","journal-title":"Acad Med"},{"issue":"6","key":"R22","doi-asserted-by":"publisher","DOI":"10.3390\/bioengineering12060631","article-title":"Large language models in healthcare and medical applications: a review","volume":"12","author":"Maity","journal-title":"Bioengineering (Basel)"},{"key":"R23","doi-asserted-by":"crossref","unstructured":"Min S Krishna K Lyu X et al. FActScore: fine-grained atomic evaluation of factual precision in long form text generation. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics; 2023:12076-12100. [doi: 10.18653\/v1\/2023.emnlp-main.741]","DOI":"10.18653\/v1\/2023.emnlp-main.741"},{"issue":"1","key":"R24","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01258-7","article-title":"A framework for human evaluation of large language models in healthcare derived from literature review","volume":"7","author":"Tam","journal-title":"NPJ Digit Med"},{"key":"R25","unstructured":"Kanithi PK Christophe C Pimentel MA et al. MEDIC: towards a comprehensive framework for evaluating LLMs in clinical applications. arXiv. Preprint posted online on Sep 11, 2024. [doi: 10.48550\/arXiv.2409.07314]"},{"issue":"2","key":"R26","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1016\/j.imed.2025.03.002","article-title":"Evaluating large language models and agents in healthcare: key challenges in clinical applications","volume":"5","author":"Chen","journal-title":"Intell Med"},{"key":"R27","doi-asserted-by":"crossref","unstructured":"Papineni K Roukos S Ward T Zhu WJ . BLEU: a method for automatic evaluation of machine translation. In: ACL \u201902: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics; 2002:311-318. [doi: 10.3115\/1073083.1073135]","DOI":"10.3115\/1073083.1073135"},{"key":"R28","unstructured":"Lin CY . ROUGE: a package for automatic evaluation of summaries. In: Text Summarization Branches Out. Association for Computational Linguistics; 2004:74-81. URL: https:\/\/aclanthology.org\/W04-1013\/ [Accessed 02-09-2026]"},{"key":"R29","doi-asserted-by":"crossref","unstructured":"Liu Y Iter D Xu Y Wang S Xu R Zhu C . G-Eval: NLG evaluation using GPT-4 with better human alignment. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics; 2023:2511-2522. [doi: 10.18653\/v1\/2023.emnlp-main.153]","DOI":"10.18653\/v1\/2023.emnlp-main.153"},{"issue":"3","key":"R30","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1016\/j.pec.2014.05.027","article-title":"Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information","volume":"96","author":"Shoemaker","journal-title":"Patient Educ Couns"}],"container-title":["JMIR Formative Research"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,9,21]],"date-time":"2026-09-21T18:26:17Z","timestamp":1790015177000},"score":1,"resource":{"primary":{"URL":"https:\/\/formative.jmir.org\/2026\/1\/e100423"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,21]]},"references-count":30,"URL":"https:\/\/doi.org\/10.2196\/100423","relation":{},"ISSN":["2561-326X"],"issn-type":[{"value":"2561-326X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,21]]},"article-number":"v10i3e100423"}}