{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T06:52:24Z","timestamp":1791442344808,"version":"4.3.3"},"reference-count":96,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Large language model (LLM)\u2013based systems have tremendous potential to improve patient-centered health care, especially for medically underserved populations. However, realizing this potential requires careful consideration of the socio-technical contexts in which LLMs are used. The design of such systems should consider the vulnerabilities of any medically underserved group that the system intends to support, and provide trustworthy evidence for its use. This viewpoint reports the lessons learned from our experience and research with the CREATE (Culture, Respect, Education, Advancement, Trust, and Expertise) framework for engaging multiple stakeholders to guide the integration of LLM-based systems for improved health care delivery. We propose an extension of the traditional hierarchy of evidence model and identify key junction points in clinical workflows at which LLM-based systems can potentially be used to improve patient care. The key messages can be summarized as follows: (1) users of AI technologies, such as LLM-based tools, must be aware of the strengths, limitations, and impact of these technologies on health care delivery workflows and on the quality of generated evidence on which health care recommendations are based; (2) users must also be aware of the impact of data quality on AI outputs and on the evidence generated from the use of these systems; (3) the evidence pyramid provides a framework that facilitates the evaluation of the output generated by AI systems; (4) the CREATE framework facilitates the engagement of a variety of stakeholders. We propose concrete approaches for its validation and implementation; (5) any system designed to support people with opioid use disorder (OUD) needs to consider the overall lack of trust and stigmatizing experiences of this population with the health care system, and we discuss various aspects of the evaluation process necessary to build trust; (6) to discuss research directions that need to be addressed before LLM-based systems can be integrated usefully in patient health care.<\/jats:p>","DOI":"10.2196\/97131","type":"journal-article","created":{"date-parts":[[2026,9,2]],"date-time":"2026-09-02T20:15:06Z","timestamp":1788380106000},"page":"e97131-e97131","source":"Crossref","is-referenced-by-count":0,"title":["From Innovation to Impact: The CREATE Framework as a Blueprint for Large Language Model Adoption in Opioid Treatment Programs"],"prefix":"10.2196","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1453-8229","authenticated-orcid":false,"given":"Marianthi","family":"Markatou","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-8972-6682","authenticated-orcid":false,"given":"Raktim","family":"Mukhopadhyay","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8679-4654","authenticated-orcid":false,"given":"Jeff","family":"Good","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-9196-8166","authenticated-orcid":false,"given":"Yihao","family":"Tan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9015-5337","authenticated-orcid":false,"given":"Arpan","family":"Dharia","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5694-9774","authenticated-orcid":false,"given":"Lawrence S","family":"Brown","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5565-7515","authenticated-orcid":false,"given":"Andrew H","family":"Talal","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,10,7]]},"reference":[{"issue":"18","key":"R1","doi-asserted-by":"publisher","first-page":"1650","DOI":"10.1001\/jama.2025.18490","article-title":"AI, health, and health care today and tomorrow: the JAMA summit report on artificial intelligence","volume":"334","author":"Angus","journal-title":"JAMA"},{"issue":"1","key":"R2","doi-asserted-by":"publisher","DOI":"10.1186\/s12913-025-12619-7","article-title":"Considerations for equitable distribution of digital healthcare for people who use drugs","volume":"25","author":"Papalamprakopoulou","journal-title":"BMC Health Serv Res"},{"issue":"1","key":"R3","doi-asserted-by":"publisher","DOI":"10.1186\/s40900-023-00474-x","article-title":"Engaging people who use drugs in clinical research: integrating facilitated telemedicine for HCV into substance use treatment","volume":"9","author":"Talal","journal-title":"Res Involv Engagem"},{"key":"R4","doi-asserted-by":"publisher","DOI":"10.1016\/j.jsat.2021.108421","article-title":"Facilitating engagement of persons with opioid use disorder in treatment for hepatitis C virus infection via telemedicine: stories of onsite case managers","volume":"127","author":"Talal","journal-title":"J Subst Abuse Treat"},{"issue":"4","key":"R5","doi-asserted-by":"crossref","first-page":"304","DOI":"10.1001\/archinte.1966.00290160004002","volume":"118","author":"Dole","journal-title":"Arch Intern Med"},{"issue":"24","key":"R6","doi-asserted-by":"publisher","first-page":"2395","DOI":"10.1056\/NEJMsb1610128","article-title":"Transforming evidence generation to support health and health care decisions","volume":"375","author":"Califf","journal-title":"N Engl J Med"},{"issue":"7","key":"R7","doi-asserted-by":"publisher","first-page":"421","DOI":"10.1136\/bmjqs-2025-018559","article-title":"Understanding the evidence for artificial intelligence in healthcare","volume":"34","author":"Jackson","journal-title":"BMJ Qual Saf"},{"key":"R8","unstructured":"Rogers EM . Diffusion of Innovations. 5th ed. Free Press; 2003. ISBN: 9780743258234"},{"key":"R9","unstructured":"Generative artificial intelligence: overview, issues, and considerations for congress. United States Congress. 2025. URL: https:\/\/www.congress.gov\/crs_external_products\/IF\/HTML\/IF12426.html [Accessed 14-09-2026]"},{"key":"R10","unstructured":"Vaswani A Shazeer N Parmar N Uszkoreit J Jones L Gomez AN et al. Attention is all you need. In: Guyon I Luxburg UV Bengio S Wallach H Fergus R Vishwanathan S , editors. Presented at: NIPS\u201917: Proceedings of the 31st International Conference on Neural Information Processing Systems; Dec 4-9, 2017; Long Beach California USA. 2017.[doi: 10.5555\/3295222.3295349]"},{"key":"R11","doi-asserted-by":"publisher","DOI":"10.3389\/fmed.2024.1477898","article-title":"Large language models in patient education: a scoping review of applications in medicine","volume":"11","author":"Aydin","journal-title":"Front Med (Lausanne)"},{"issue":"1","key":"R12","doi-asserted-by":"publisher","DOI":"10.1186\/s12909-025-08085-8","article-title":"Generative AI in medical education: feasibility and educational value of LLM-generated clinical cases with MCQs","volume":"25","author":"Zhang","journal-title":"BMC Med Educ"},{"key":"R13","doi-asserted-by":"publisher","DOI":"10.2196\/49324","article-title":"Large language models for therapy recommendations across 3 clinical specialties: comparative study","volume":"25","author":"Wilhelm","journal-title":"J Med Internet Res"},{"issue":"1","key":"R14","doi-asserted-by":"publisher","first-page":"471","DOI":"10.1146\/annurev-biodatasci-103123-095406","article-title":"Biomedical natural language processing in the era of large language models","volume":"8","author":"Usuyama","journal-title":"Annu Rev Biomed Data Sci"},{"issue":"7","key":"R15","doi-asserted-by":"publisher","DOI":"10.1145\/3757544","article-title":"A risk taxonomy and reflection tool for large language model adoption in public health","volume":"9","author":"Zhou","journal-title":"Proc ACM Hum Comput Interact"},{"key":"R16","unstructured":"Sociotechnical systems. Taylor & Francis Knowledge Centers. URL: https:\/\/taylorandfrancis.com\/knowledge\/Engineering_and_technology\/Computer_science\/Sociotechnical_systems [Accessed 14-09-2026]"},{"issue":"Suppl 3","key":"R17","doi-asserted-by":"publisher","first-page":"i68","DOI":"10.1136\/qshc.2010.042085","article-title":"A new sociotechnical model for studying health information technology in complex adaptive healthcare systems","volume":"19 Suppl 3","author":"Sittig","journal-title":"Qual Saf Health Care"},{"key":"R18","doi-asserted-by":"crossref","unstructured":"Selbst AD Boyd D Friedler SA Venkatasubramanian S Vertesi J . Fairness and abstraction in sociotechnical systems. Presented at: FAT* \u201919: Proceedings of the Conference on Fairness, Accountability, and Transparency; Jan 29-31, 2019:59-68; Atlanta GA USA. Jan 29, 2019.[doi: 10.1145\/3287560.3287598]","DOI":"10.1145\/3287560.3287598"},{"key":"R19","unstructured":"Artificial intelligence. Epic systems c. URL: https:\/\/www.epic.com\/software\/ai [Accessed 14-09-2026]"},{"issue":"1","key":"R20","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-025-31113-y","article-title":"Enhancing patient participation in emergency department through patient-friendly clinical notes generated by large language models","volume":"16","author":"Kim","journal-title":"Sci Rep"},{"issue":"3","key":"R21","doi-asserted-by":"publisher","first-page":"932","DOI":"10.1038\/s41591-024-03416-6","article-title":"A generalist medical language model for disease diagnosis assistance","volume":"31","author":"Liu","journal-title":"Nat Med"},{"key":"R22","unstructured":"Panagoulias DP Virvou M Tsihrintzis GA . Evaluating LLM--generated multimodal diagnosis from medical images and symptom analysis. arXiv. Preprint posted online on Jan 28, 2024. [doi: 10.48550\/arXiv.2402.01730]"},{"key":"R23","doi-asserted-by":"crossref","unstructured":"Panagoulias DP Palamidas FA Virvou M Tsihrintzis GA . Evaluating the potential of llms and chatgpt on medical diagnosis and treatment. Presented at: 2023 14th International Conference on Information, Intelligence, Systems & Applications (IISA); Jul 10-12, 2023; Volos, Greece. [doi: 10.1109\/IISA59645.2023.10345968]","DOI":"10.1109\/IISA59645.2023.10345968"},{"key":"R24","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejim.2024.09.017","article-title":"A large language model-based clinical decision support system for syncope recognition in the emergency department: a framework for clinical workflow integration","volume":"131","author":"Levra","journal-title":"Eur J Intern Med"},{"key":"R25","unstructured":"Tools and resources to support engagement in research. Patient-Centered Outcomes Research Institute (PCORI). URL: http:\/\/pcori.org\/engagement-research\/tools-and-resources-support-engagement-research [Accessed 24-09-2026]"},{"issue":"10","key":"R26","doi-asserted-by":"publisher","first-page":"842","DOI":"10.1001\/jama.2023.1044","article-title":"Appropriateness of cardiovascular disease prevention recommendations obtained from a popular online chat-based artificial intelligence model","volume":"329","author":"Sarraju","journal-title":"JAMA"},{"issue":"19","key":"R27","doi-asserted-by":"publisher","first-page":"1519","DOI":"10.2106\/JBJS.23.00209","article-title":"Assessing ChatGPT responses to common patient questions regarding total hip arthroplasty","volume":"105","author":"Mika","journal-title":"J Bone Joint Surg Am"},{"issue":"6","key":"R28","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.40135","article-title":"Radiology gets chatty: the ChatGPT saga unfolds","volume":"15","author":"Grewal","journal-title":"Cureus"},{"issue":"2","key":"R29","doi-asserted-by":"publisher","first-page":"210","DOI":"10.7326\/M23-2772","article-title":"Large language models in medicine: the potentials and pitfalls: a narrative review","volume":"177","author":"Omiye","journal-title":"Ann Intern Med"},{"key":"R30","doi-asserted-by":"crossref","unstructured":"Liu S McCoy AB Wright AP et al. Leveraging large language models for generating responses to patient messages. medRxiv. Jul 16, 2023:2023.07.14.23292669. [doi: 10.1101\/2023.07.14.23292669] [Medline: 37503263]","DOI":"10.1101\/2023.07.14.23292669"},{"issue":"5","key":"R31","doi-asserted-by":"publisher","first-page":"503","DOI":"10.1080\/08820538.2023.2209166","article-title":"ChatGPT and ophthalmology: exploring its potential with discharge summaries and operative notes","volume":"38","author":"Singh","journal-title":"Semin Ophthalmol"},{"issue":"1","key":"R32","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1038\/s41523-023-00557-8","article-title":"Large language model (ChatGPT) as a support tool for breast tumor board","volume":"9","author":"Sorin","journal-title":"NPJ Breast Cancer"},{"issue":"7","key":"R33","doi-asserted-by":"publisher","first-page":"964","DOI":"10.7326\/M24-0146","article-title":"Artificial Intelligence in the provision of health care: an American College of Physicians policy position paper","volume":"177","author":"Daneshvar","journal-title":"Ann Intern Med"},{"key":"R34","doi-asserted-by":"publisher","DOI":"10.2196\/77279","article-title":"Monitoring opioid-related social media chatter using natural language processing and large language models: temporal analysis","volume":"5","author":"Sidorov","journal-title":"JMIR Infodemiology"},{"issue":"3","key":"R35","doi-asserted-by":"publisher","first-page":"408","DOI":"10.7326\/ANNALS-24-02338","article-title":"Guidelines international network: principles for use of artificial intelligence in the health guideline enterprise","volume":"178","author":"Sousa-Pinto","journal-title":"Ann Intern Med"},{"issue":"3","key":"R36","doi-asserted-by":"publisher","first-page":"389","DOI":"10.7326\/ANNALS-24-02189","article-title":"Development of prompt templates for large language model-driven screening in systematic reviews","volume":"178","author":"Cao","journal-title":"Ann Intern Med"},{"key":"R37","unstructured":"McNemar E . How does artificial intelligence compare to augmented intelligence. TechTarget Health IT Analytics. 2021. URL: https:\/\/www.techtarget.com\/healthtechanalytics\/news\/366591056\/How-Does-Artificial-Intelligence-Compare-to-Augmented-Intelligence [Accessed 14-09-2026]"},{"key":"R38","unstructured":"Augmented intelligence in medicine. American Medical Association. URL: https:\/\/www.ama-assn.org\/practice-management\/digital-health\/augmented-intelligence-medicine [Accessed 14-09-2026]"},{"key":"R39","unstructured":"Zhang X Yu J Yan P Jiang L Shen X Cheng M et al. Human-in-the-loop interactive report generation for chronic disease adherence. arXiv. Preprint posted online on Jan 10, 2026. [doi: 10.48550\/arXiv.2601.06364]"},{"key":"R40","unstructured":"Kieseberg P Weippl E Holzinger A . Trust for the \u201cdoctor in the loop\u201d. ERCIM News. Jan 13, 2016. URL: https:\/\/ercim-news.ercim.eu\/en104\/special\/trust-for-the-doctor-in-the-loop [Accessed 24-09-2026]"},{"key":"R41","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2025.105630","article-title":"Doctor-in-the-Loop: an explainable, multi-view deep learning framework for predicting pathological response in non-small cell lung cancer","volume":"161","author":"Caragliano","journal-title":"Image Vis Comput"},{"key":"R42","doi-asserted-by":"publisher","DOI":"10.1162\/99608f92.8036d03b","article-title":"AI transparency in the age of LLMs: a human-centered research roadmap","author":"Liao","journal-title":"Harvard Data Science Review"},{"key":"R43","doi-asserted-by":"crossref","unstructured":"Bouzoubaa L Aghakhani E Rezapour R , editors. Words Matter: Reducing Stigma in Online Conversations about Substance Use with Large Language Models. Association for Computational Linguistics; 2024. [doi: 10.18653\/v1\/2024.emnlp-main.516]","DOI":"10.18653\/v1\/2024.emnlp-main.516"},{"issue":"3","key":"R44","doi-asserted-by":"publisher","DOI":"10.3390\/biomedicines13030636","article-title":"Effectiveness of generative artificial intelligence-driven responses to patient concerns in long-term opioid therapy: cross-model assessment","volume":"13","author":"Lo Bianco","journal-title":"Biomedicines"},{"issue":"12","key":"R45","doi-asserted-by":"publisher","first-page":"1865","DOI":"10.1093\/jamia\/ocaf157","article-title":"Predicting treatment retention in medication for opioid use disorder: a machine learning approach using NLP and LLM-derived clinical features","volume":"32","author":"Nateghi Haredasht","journal-title":"J Am Med Inform Assoc"},{"key":"R46","unstructured":"Voigt H Sugamiya Y Lawonn K Zarrie\u00df S Takanishi A . LLM-powered virtual patient agents for interactive clinical skills training with automated feedback. arXiv. Preprint posted online on Aug 19, 2025. [doi: 10.48550\/arXiv.2508.13943]"},{"issue":"8","key":"R47","doi-asserted-by":"publisher","first-page":"3032","DOI":"10.1038\/s41591-026-04503-6","article-title":"Generative AI-enabled clinical decision support system in primary care: a pragmatic, cluster-randomized trial","volume":"32","author":"Agweyu","journal-title":"Nat Med"},{"issue":"10","key":"R48","doi-asserted-by":"publisher","DOI":"10.1016\/j.xcrm.2025.102323","article-title":"Large language model as clinical decision support system augments medication safety in 16 clinical specialties","volume":"6","author":"Ong","journal-title":"Cell Rep Med"},{"key":"R49","doi-asserted-by":"publisher","DOI":"10.1101\/2025.08.08.25333315","article-title":"Detecting stigmatizing language in clinical notes with large language models for addiction care","author":"Sethi","journal-title":"medRxiv"},{"issue":"5","key":"R50","doi-asserted-by":"publisher","DOI":"10.1093\/jamiaopen\/ooaf103","article-title":"Automated survey collection with LLM-based conversational agents","volume":"8","author":"Kaiyrbekov","journal-title":"JAMIA Open"},{"issue":"4","key":"R51","doi-asserted-by":"publisher","first-page":"1134","DOI":"10.1038\/s41591-024-02855-5","article-title":"Adapted large language models can outperform medical experts in clinical text summarization","volume":"30","author":"Van Veen","journal-title":"Nat Med"},{"issue":"1","key":"R52","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-025-32150-3","article-title":"Transforming free-text coronary angiography reports into structured, analyzable data using large language models","volume":"16","author":"Song","journal-title":"Sci Rep"},{"issue":"3.5","key":"R53","doi-asserted-by":"publisher","first-page":"250215698","DOI":"10.6004\/jnccn.2024.7135","article-title":"BPI25-012: developing an artificial intelligence tool for personalized breast cancer treatment plans based on the NCCN guidelines","volume":"23","author":"Mansoor","journal-title":"J Natl Compr Canc Netw"},{"issue":"7","key":"R54","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/adbff1","article-title":"A feasibility study of automating radiotherapy planning with large language model agents","volume":"70","author":"Wang","journal-title":"Phys Med Biol"},{"key":"R55","doi-asserted-by":"publisher","DOI":"10.1177\/20552076261472569","article-title":"Feasibility of LLM-assisted post-discharge tuberculosis care: a comparative study of medication counseling, patient education, and follow-up planning","volume":"12","author":"He","journal-title":"Digit HEALTH"},{"key":"R56","unstructured":"Wang H Fu W Tang Y Chen Z Huang Y Piao J et al. A survey on responsible llms: inherent risk, malicious use, and mitigation strategy. arXiv. Preprint posted online on Jan 16, 2025. [doi: 10.48550\/arXiv.2501.09431]"},{"issue":"6","key":"R57","doi-asserted-by":"publisher","first-page":"5795","DOI":"10.1007\/s43681-025-00814-5","article-title":"Navigating LLM ethics: advancements, challenges, and future directions","volume":"5","author":"Jiao","journal-title":"AI Ethics"},{"key":"R58","unstructured":"Bender EM Hanna A . The AI Con: How to Fight Big Tech\u2019s Hype and Create the Future We Want. HarperCollins; 2025. ISBN: 9780063418554"},{"issue":"6","key":"R59","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000877","article-title":"Retrieval augmented generation for large language models in healthcare: a systematic review","volume":"4","author":"Amugongo","journal-title":"PLOS Digit Health"},{"issue":"1","key":"R60","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-02008-z","article-title":"When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior","volume":"8","author":"Chen","journal-title":"NPJ Digit Med"},{"key":"R61","doi-asserted-by":"crossref","unstructured":"Rebedea T Dinu R Sreedhar MN Parisien C . Cohen J , editor. NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails. Association for Computational Linguistics; 2023. [doi: 10.18653\/v1\/2023.emnlp-demo.40]","DOI":"10.18653\/v1\/2023.emnlp-demo.40"},{"issue":"2","key":"R62","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3639372","article-title":"Explainability for large language models: a survey","volume":"15","author":"Zhao","journal-title":"ACM Trans Intell Syst Technol"},{"key":"R63","doi-asserted-by":"crossref","unstructured":"Yuan J Li H Ding X et al. Understanding and mitigating numerical sources of nondeterminism in LLM inference. Presented at: Advances in Neural Information Processing Systems 38; Dec 2-7, 2025; San Diego, California, USA and Mexico City, Mexico. 2025.[doi: 10.52202\/085713-5653]","DOI":"10.52202\/085713-5653"},{"issue":"3","key":"R64","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3744238","article-title":"A survey on uncertainty quantification of large language models: taxonomy, open research challenges, and future directions","volume":"58","author":"Shorinwa","journal-title":"ACM Comput Surv"},{"key":"R65","unstructured":"Atf Z Safavi-Naini SAA Lewis PR Mahjoubfar A Naderi N Savage TR et al. The challenge of uncertainty quantification of large language models in medicine. arXiv. Preprint posted online on Apr 7, 2025. [doi: 10.48550\/arXiv.2504.05278]"},{"issue":"1","key":"R66","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1093\/jamia\/ocae254","article-title":"Large language model uncertainty proxies: discrimination and calibration for medical diagnosis and treatment","volume":"32","author":"Savage","journal-title":"J Am Med Inform Assoc"},{"key":"R67","unstructured":"AMIA artificial intelligence evaluation showcase. American Medical Informatics Association. URL: https:\/\/web.archive.org\/web\/20250908082941\/https:\/\/amia.org\/education-events\/amia-artificial-intelligence-evaluation-showcase [Accessed 14-09-2026]"},{"key":"R68","unstructured":"Minaee S Mikolov T Nikzad N Chenaghlu M Socher R Amatriain X et al. Large language models: a survey. arXiv. Preprint posted online on Feb 9, 2024. [doi: 10.48550\/arXiv.2402.06196]"},{"key":"R69","doi-asserted-by":"publisher","DOI":"10.1101\/2025.08.06.25333170","article-title":"A statistical framework for evaluating the repeatability and reproducibility of large language models","author":"Shyr","journal-title":"medRxiv"},{"key":"R70","unstructured":"Usability. Computer Security Resource Center, National Institute of Standards and Technology. URL: https:\/\/csrc.nist.gov\/glossary\/term\/usability [Accessed 14-09-2026]"},{"issue":"1","key":"R71","doi-asserted-by":"publisher","DOI":"10.1186\/s12913-025-13745-y","article-title":"Usability evaluation methods for hospital information systems: a systematic review","volume":"25","author":"Dehghani Mahmoodabadi","journal-title":"BMC Health Serv Res"},{"key":"R72","doi-asserted-by":"publisher","DOI":"10.3389\/fmed.2023.1076794","article-title":"Social determinants of health derived from people with opioid use disorder: Improving data collection, integration and use with cross-domain collaboration and reproducible, data-centric, notebook-style workflows","volume":"10","author":"Markatou","journal-title":"Front Med (Lausanne)"},{"issue":"6","key":"R73","doi-asserted-by":"publisher","DOI":"10.1056\/CAT.25.0372","article-title":"Why and how to monitor deployed AI systems in health care","volume":"7","author":"Keyes","journal-title":"NEJM Catalyst"},{"issue":"1","key":"R74","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-026-02418-7","article-title":"Advancing healthcare AI governance through a comprehensive maturity model based on systematic review","volume":"9","author":"Hussein","journal-title":"NPJ Digit Med"},{"issue":"11","key":"R75","doi-asserted-by":"publisher","DOI":"10.3390\/healthcare14111459","article-title":"Beyond model development in healthcare AI: post-development robustness, post-deployment monitoring, and lifecycle governance-a scoping review of reviews","volume":"14","author":"El Arab","journal-title":"Healthcare (Basel)"},{"key":"R76","unstructured":"O\u2019Donnell B Gupta V . Continuous Quality Improvement. StatPearls; 2026. URL: https:\/\/www.ncbi.nlm.nih.gov\/books\/NBK559239\/ [Accessed 24-09-2026]"},{"key":"R77","unstructured":"Ethics and governance of artificial intelligence for health: guidance on large multi-modal models. World Health Organization. 2024. URL: https:\/\/www.who.int\/publications\/i\/item\/9789240084759 [Accessed 14-09-2026]"},{"key":"R78","unstructured":"Regulation (EU) 2024\/1689 of the european parliament and of the council of 13 june 2024 laying down harmonised rules on artificial intelligence and amending regulations (EC) no 300\/2008, (EU) no 167\/2013, (EU) no 168\/2013, (EU) 2018\/858, (EU) 2018\/1139 and (EU) 2019\/2144 and directives 2014\/90\/EU, (EU) 2016\/797 and (EU) 2020\/1828 (artificial intelligence act). EUR-Lex. URL: https:\/\/eur-lex.europa.eu\/eli\/reg\/2024\/1689\/oj [Accessed 14-09-2026]"},{"key":"R79","unstructured":"EU AI act: first regulation on artificial intelligence. European Parliament. URL: https:\/\/www.europarl.europa.eu\/topics\/en\/article\/20230601STO93804\/eu-ai-act-first-regulation-on-artificial-intelligence [Accessed 14-09-2026]"},{"key":"R80","unstructured":"Madiega T . Artificial intelligence act. European Parliamentary Research Service; 2024. URL: https:\/\/www.europarl.europa.eu\/RegData\/etudes\/BRIE\/2021\/698792\/EPRS_BRI(2021)698792_EN.pdf [Accessed 14-09-2026]"},{"key":"R81","unstructured":"California senate bill 53 (2025-2026 regular session): artificial intelligence models: large developers. Legi Scan. 2025. URL: https:\/\/legiscan.com\/CA\/bill\/SB53\/2025 [Accessed 14-09-2026]"},{"key":"R82","unstructured":"Center for drug evaluation and research artificial intelligence for drug development. US Food and Drug Administration. 2025. URL: https:\/\/www.fda.gov\/about-fda\/center-drug-evaluation-and-research-cder\/artificial-intelligence-drug-development [Accessed 14-09-2026]"},{"key":"R83","unstructured":"Ensuring a national policy framework for artificial intelligence. Fed Regist. 2025. URL: https:\/\/www.federalregister.gov\/documents\/2025\/12\/16\/2025-23092\/ensuring-a-national-policy-framework-for-artificial-intelligence [Accessed 14-09-2026]"},{"key":"R84","unstructured":"Governor Hochul signs nation-leading legislation to require AI frameworks for AI frontier models. New York State Governor\u2019s Office. 2025. URL: https:\/\/www.governor.ny.gov\/news\/governor-hochul-signs-nation-leading-legislation-require-ai-frameworks-ai-frontier-models [Accessed 14-09-2026]"},{"issue":"1","key":"R85","doi-asserted-by":"publisher","DOI":"10.1038\/s43856-024-00492-0","article-title":"Reporting guidelines in medical artificial intelligence: a systematic review and meta-analysis","volume":"4","author":"Kolbinger","journal-title":"Commun Med (Lond)"},{"key":"R86","doi-asserted-by":"publisher","DOI":"10.2196\/46089","article-title":"Guidelines, consensus statements, and standards for the use of artificial intelligence in medicine: systematic review","volume":"25","author":"Wang","journal-title":"J Med Internet Res"},{"issue":"3","key":"R87","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1056\/NEJMp2314771","article-title":"The ethics of relational AI \u2014 expanding and implementing the Belmont principles","volume":"391","author":"Sim","journal-title":"N Engl J Med"},{"key":"R88","unstructured":"Augmented intelligence development, deployment, and use in health care. American Medical Association. URL: https:\/\/www.ama-assn.org\/system\/files\/ama-ai-principles.pdf [Accessed 14-09-2026]"},{"key":"R89","unstructured":"Blueprint for trustworthy AI implementation guidance and assurance for healthcare. Coalition for Health AI. 2023. URL: https:\/\/assets.ctfassets.net\/7s4afyr9pmov\/4AXIWGIlcrjWDaW2ueTaRS\/f98e5cb2528187635895cce6ba5ec309\/Blueprint_for_Trustworthy_AI.pdf [Accessed 14-09-2026]"},{"key":"R90","doi-asserted-by":"crossref","unstructured":"Xu F Uszkoreit H Du Y Fan W Zhao D Zhu J . Explainable AI: a brief survey on history, research areas, approaches and challenges. Presented at: Natural Language Processing and Chinese Computing: 8th CCF International Conference, NLPCC 2019; Oct 9-14, 2019:563-574; Dunhuang, China. [doi: 10.1007\/978-3-030-32236-6_51]","DOI":"10.1007\/978-3-030-32236-6_51"},{"issue":"3","key":"R91","doi-asserted-by":"publisher","first-page":"511","DOI":"10.1097\/j.pain.0000000000003388","article-title":"Racial, ethnic, and sex bias in large language model opioid recommendations for pain management","volume":"166","author":"Young","journal-title":"Pain"},{"key":"R92","doi-asserted-by":"publisher","DOI":"10.3389\/fpubh.2019.00064","article-title":"RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review","volume":"7","author":"Glasgow","journal-title":"Front Public Health"},{"issue":"1","key":"R93","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1186\/s13012-022-01245-0","article-title":"The updated consolidated framework for implementation research based on user feedback","volume":"17","author":"Damschroder","journal-title":"Implementation Sci"},{"key":"R94","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2026.106362","article-title":"Human in the loop artificial intelligence in healthcare: applications, outcomes, and implementation challenges","volume":"213","author":"Olawade","journal-title":"Int J Med Inform"},{"issue":"4","key":"R95","doi-asserted-by":"publisher","DOI":"10.3390\/e28040377","article-title":"Human-in-the-loop artificial intelligence: a systematic review of concepts, methods, and applications","volume":"28","author":"Lazaros","journal-title":"Entropy (Basel)"},{"issue":"3","key":"R96","doi-asserted-by":"publisher","first-page":"661","DOI":"10.1007\/s11548-026-03568-5","article-title":"Environmental and economic costs behind LLMs","volume":"21","author":"L\u00f3pez-\u00dabeda","journal-title":"Int J CARS"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T06:22:21Z","timestamp":1791440541000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2026\/1\/e97131"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,7]]},"references-count":96,"URL":"https:\/\/doi.org\/10.2196\/97131","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,7]]},"article-number":"v28i18e97131"}}