{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T08:09:51Z","timestamp":1789027791237,"version":"build-2803163510"},"reference-count":45,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T00:00:00Z","timestamp":1788998400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001711","name":"Schweizerischer Nationalfonds zur F\u00f6rderung der Wissenschaftlichen Forschung","doi-asserted-by":"publisher","award":["211751"],"award-info":[{"award-number":["211751"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Digit. Health"],"abstract":"<jats:p>\n                    Medical large language models are evaluated through factuality, physician preference, source support, retrieval quality, safety, and clinical utility. These measures describe content and performance while leaving unresolved whether a generated claim meets the requirements for a specified clinical use. This article proposes\n                    <jats:italic>clinical warrant<\/jats:italic>\n                    as a use-specific judgment for a particular output\u2013use pair. Clinical use includes reliance within a patient-related encounter, task, or care pathway, including education that can shape understanding or later action. The proposed use and the standards used to judge it are documented separately. For action-guiding outputs, clinical appropriateness is a gateway. Five further conditions examine role, case information, evidence, preserved limits, and authority\/accountability. The framework asks whether these requirements are met at the point of proposed reliance, records failed or unresolved conditions and the resulting disposition, and tests whether a relevant change stops or redirects use. Fifteen constructed examples across five domains distinguish established, not established, and indeterminate warrant.\n                  <\/jats:p>","DOI":"10.3389\/fdgth.2026.1918724","type":"journal-article","created":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T08:05:36Z","timestamp":1789027536000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Clinical warrant in medical LLM evaluation"],"prefix":"10.3389","volume":"8","author":[{"given":"Murat","family":"Sariyar","sequence":"first","affiliation":[{"name":"Institute for Optimization and Data Analysis (IODA), Bern University of Applied Sciences","place":["Bern, Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,10]]},"reference":[{"key":"B1","first-page":"1877","article-title":"Language models are few-shot learners","volume":"33","author":"Brown","year":"2020","journal-title":"Adv Neural Inf Process Syst"},{"key":"B2","doi-asserted-by":"crossref","first-page":"27730","DOI":"10.52202\/068431-2011","article-title":"Training language models to follow instructions with human feedback","volume":"35","author":"Ouyang","year":"2022","journal-title":"Adv Neural Inf Process Syst"},{"key":"B3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3571730","article-title":"Survey of hallucination in natural language generation","volume":"55","author":"Ji","year":"2023","journal-title":"ACM Comput Surv"},{"key":"B4","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"B5","doi-asserted-by":"publisher","first-page":"943","DOI":"10.1038\/s41591-024-03423-7","article-title":"Toward expert-level medical question answering with large language models","volume":"31","author":"Singhal","year":"2025","journal-title":"Nat Med"},{"key":"B6","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1038\/s41746-024-01258-7","article-title":"A framework for human evaluation of large language models in healthcare derived from literature review","volume":"7","author":"Tam","year":"2024","journal-title":"NPJ Digit Med"},{"key":"B7","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1038\/s41591-024-03328-5","article-title":"An evaluation framework for clinical use of large language models in patient interaction tasks","volume":"31","author":"Johri","year":"2025","journal-title":"Nat Med"},{"key":"B8","first-page":"9459","article-title":"Retrieval-augmented generation for knowledge-intensive NLP tasks","volume":"33","author":"Lewis","year":"2020","journal-title":"Adv Neural Inf Process Syst"},{"key":"B9","doi-asserted-by":"publisher","first-page":"e0000877","DOI":"10.1371\/journal.pdig.0000877","article-title":"Retrieval augmented generation for large language models in healthcare: a systematic review","volume":"4","author":"Abdullahi","year":"2025","journal-title":"PLoS Digit Health"},{"key":"B10","doi-asserted-by":"publisher","first-page":"3615","DOI":"10.1038\/s41467-025-58551-6","article-title":"An automated framework for assessing how well LLMs cite relevant medical references","volume":"16","author":"Wu","year":"2025","journal-title":"Nat Commun"},{"key":"B11","author":"Fitch","year":""},{"key":"B12","doi-asserted-by":"publisher","first-page":"17885","DOI":"10.1038\/s41598-023-45223-y","article-title":"A vignette-based evaluation of ChatGPT\u2019s ability to provide appropriate and equitable medical advice across care contexts","volume":"13","author":"Nastasi","year":"2023","journal-title":"Sci Rep"},{"key":"B13","volume-title":"Ecological Communication","author":"Luhmann","year":"1989"},{"key":"B14","volume-title":"Social Systems","author":"Luhmann","year":"1995"},{"key":"B15","author":"Alaa","year":""},{"key":"B16","doi-asserted-by":"publisher","first-page":"e58329","DOI":"10.2196\/58329","article-title":"Evaluation framework of large language models in medical documentation: development and usability study","volume":"26","author":"Seo","year":"2024","journal-title":"J Med Internet Res"},{"key":"B17","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1038\/s41591-024-03425-5","article-title":"The TRIPOD-LLM reporting guideline for studies using large language models","volume":"31","author":"Gallifant","year":"2025","journal-title":"Nat Med"},{"key":"B18","doi-asserted-by":"publisher","first-page":"e081554","DOI":"10.1136\/bmj-2024-081554","article-title":"FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare","volume":"388","author":"Lekadir","year":"2025","journal-title":"BMJ"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1347","DOI":"10.1056\/NEJMra1814259","article-title":"Machine learning in medicine","volume":"380","author":"Rajkomar","year":"2019","journal-title":"New Engl J Med"},{"key":"B20","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","article-title":"High-performance medicine: the convergence of human and artificial intelligence","volume":"25","author":"Topol","year":"2019","journal-title":"Nat Med"},{"key":"B21","doi-asserted-by":"publisher","first-page":"310","DOI":"10.1186\/s12911-020-01332-6","article-title":"Explainability for artificial intelligence in healthcare: a multidisciplinary perspective","volume":"20","author":"Amann","year":"2020","journal-title":"BMC Med Inform Decis Mak"},{"key":"B22","doi-asserted-by":"publisher","first-page":"72","DOI":"10.1186\/s12911-024-02459-6","article-title":"Assessing the research landscape and clinical utility of large language models: a scoping review","volume":"24","author":"Park","year":"2024","journal-title":"BMC Med Inform Decis Mak"},{"key":"B23","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1038\/s41746-024-01064-1","article-title":"To warrant clinical adoption AI models require a multi-faceted implementation evaluation","volume":"7","author":"van de Sande","year":"2024","journal-title":"NPJ Digit Med"},{"key":"B24","author":"Yang","year":""},{"key":"B25","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1136\/bmj.312.7023.71","article-title":"Evidence based medicine: what it is and what it isn\u2019t","volume":"312","author":"Sackett","year":"1996","journal-title":"BMJ"},{"key":"B26","doi-asserted-by":"publisher","first-page":"924","DOI":"10.1136\/bmj.39489.470347.AD","article-title":"GRADE: an emerging consensus on rating quality of evidence and strength of recommendations","volume":"336","author":"Guyatt","year":"2008","journal-title":"BMJ"},{"key":"B27","author":"Higgins","year":""},{"key":"B28","author":"Schumacher","year":""},{"key":"B29","doi-asserted-by":"publisher","first-page":"501","DOI":"10.1038\/s42256-019-0114-4","article-title":"Principles alone cannot guarantee ethical AI","volume":"1","author":"Mittelstadt","year":"2019","journal-title":"Nat Mach Intell"},{"key":"B30","doi-asserted-by":"publisher","first-page":"196","DOI":"10.1038\/s41746-024-01197-3","article-title":"Autonomous artificial intelligence for diabetic eye disease increases access and health equity in underserved populations","volume":"7","author":"Huang","year":"2024","journal-title":"NPJ Digit Med"},{"key":"B31","doi-asserted-by":"publisher","first-page":"e230424","DOI":"10.1148\/radiol.230424","article-title":"Appropriateness of breast cancer prevention and screening recommendations provided by ChatGPT","volume":"307","author":"Haver","year":"2023","journal-title":"Radiology"},{"key":"B32","doi-asserted-by":"publisher","first-page":"1065","DOI":"10.1001\/jamainternmed.2021.2626","article-title":"External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients","volume":"181","author":"Wong","year":"2021","journal-title":"JAMA Intern Med"},{"key":"B33","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1038\/s41746-018-0048-y","article-title":"With an eye to AI and autonomous diagnosis","volume":"1","author":"Keane","year":"2018","journal-title":"NPJ Digit Med"},{"key":"B34","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1038\/s41746-024-01225-2","article-title":"Development, deployment and scaling of operating room-ready artificial intelligence for real-time surgical decision support","volume":"7","author":"Protserov","year":"2024","journal-title":"NPJ Digit Med"},{"key":"B35","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1038\/s41746-024-01102-y","article-title":"Levels of autonomy in FDA-cleared surgical robots: a systematic review","volume":"7","author":"Lee","year":"2024","journal-title":"NPJ Digit Med"},{"key":"B36","doi-asserted-by":"publisher","first-page":"464","DOI":"10.1038\/clpt.2010.279","article-title":"CPIC: clinical pharmacogenetics implementation consortium of the pharmacogenomics research network","volume":"89","author":"Relling","year":"2011","journal-title":"Clin Pharmacol Ther"},{"key":"B37","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1038\/nature25753","article-title":"Meta-analysis and the science of research synthesis","volume":"555","author":"Gurevitch","year":"2018","journal-title":"Nature"},{"key":"B38","doi-asserted-by":"publisher","first-page":"241776","DOI":"10.1098\/rsos.241776","article-title":"Generalization bias in large language model summarization of scientific research","volume":"12","author":"Peters","year":"2025","journal-title":"R Soc Open Sci"},{"key":"B39","doi-asserted-by":"publisher","first-page":"158","DOI":"10.1038\/s41746-023-00896-7","article-title":"Evaluating large language models on medical evidence summarization","volume":"6","author":"Tang","year":"2023","journal-title":"NPJ Digit Med"},{"key":"B40","first-page":"881","article-title":"Towards more patient friendly clinical notes through language models and ontologies","volume":"2021","author":"Moramarco","year":"2021","journal-title":"AMIA Annu Symp Proc"},{"key":"B41","author":"Kim","year":""},{"key":"B42","author":"Lewis","year":""},{"key":"B43","doi-asserted-by":"publisher","first-page":"330","DOI":"10.1038\/s43856-025-01021-3","article-title":"Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support","volume":"5","author":"Omar","year":"2025","journal-title":"Commun Med"},{"key":"B44","doi-asserted-by":"publisher","first-page":"9011","DOI":"10.1038\/s41467-025-64062-1","article-title":"Adversarial prompt and fine-tuning attacks threaten medical large language models","volume":"16","author":"Yang","year":"2025","journal-title":"Nat Commun"},{"key":"B45","doi-asserted-by":"publisher","first-page":"e2549963","DOI":"10.1001\/jamanetworkopen.2025.49963","article-title":"Vulnerability of large language models to prompt injection when providing medical advice","volume":"8","author":"Lee","year":"2025","journal-title":"JAMA Netw Open"}],"container-title":["Frontiers in Digital Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2026.1918724\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T08:05:37Z","timestamp":1789027537000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2026.1918724\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,10]]},"references-count":45,"alternative-id":["10.3389\/fdgth.2026.1918724"],"URL":"https:\/\/doi.org\/10.3389\/fdgth.2026.1918724","relation":{},"ISSN":["2673-253X"],"issn-type":[{"value":"2673-253X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,10]]},"article-number":"1918724"}}