{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T14:53:31Z","timestamp":1791903211148,"version":"4.3.4"},"reference-count":37,"publisher":"BMJ","license":[{"start":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T00:00:00Z","timestamp":1791849600000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ"],"accepted":{"date-parts":[[2026,8,26]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>To characterise the reporting completeness of clinical trials involving large language model (LLM) interventions using established reporting guidelines.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Design<\/jats:title>\n                    <jats:p>Systematic review and meta-research analysis.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Data sources<\/jats:title>\n                    <jats:p>Bibliographic databases (Embase, Medline, and CENTRAL) were searched for trial reports and protocols, and trial registries (ClinicalTrials.gov and World Health Organization International Clinical Trials Registry Platform) were searched for trial registrations published from 1 January 2017 to 6 December 2025.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Eligibility criteria<\/jats:title>\n                    <jats:p>Trial reports, trial protocols, and trial registrations describing clinical trials of LLM interventions in medicine. LLM interventions used for a healthcare function in a clinical trial setting included patient education, clinical documentation, communication support, triage, care navigation, management recommendations, or clinical decision support. Eligible studies prospectively assigned human participants to the intervention and evaluated a health related biomedical or behavioural outcome.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Main outcome measures<\/jats:title>\n                    <jats:p>Overall study level and item level reporting completeness assessed using reporting guidelines matched to study design and purpose, including TRIPOD-LLM (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis\u2014LLM), CONSORT (Consolidated Standards of Reporting Trials) 2025 and CONSORT-AI, SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) 2025 and SPIRIT-AI, CHART (Chatbot Assessment Reporting Tool), and the WHO Trial Registration Dataset (version 1.3.1). Two reviewers independently screened and extracted trial and intervention characteristics, and three calibrated raters assessed item level reporting completeness.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The systematic search yielded 461 records, including 68 trial reports, 15 protocols, and 378 registrations. Trial reports were predominantly small sample, single site randomised studies published between 2024 and 2025, spanning diverse clinical specialties and primarily targeting patient facing education and management recommendations. Mean reporting completeness was modest across eligible trial reports (TRIPOD-LLM 40%, CONSORT 2025 44%, CONSORT-AI 59%, CHART 52%), trial protocols (SPIRIT 2025 51%, SPIRIT-AI 57%), and trial registrations (WHO Trial Registration Dataset 61%). Clinical trial reports of LLM interventions prioritise narrative framing and commentary over the methodological detail needed for replication, risk-of-bias assessment, and governance of model performance over time. Critical methodological elements for reproducibility, including data and intervention provenance, input preprocessing, model versioning and access modality, inference parameters, output postprocessing, as well as protocol, data, and code access, were inconsistently reported. Harms planning, monitoring, and reporting were often incompletely reported, limiting the critical appraisal of the safety of LLM interventions in healthcare.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Clinical trials of LLM interventions show modest adherence to established reporting guidelines, with persistent underreporting of LLM specific methodological details, reproducibility artefacts (protocols, data, code), and harms reporting required to support reproducibility and confident appraisal of safety risks before their clinical translation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1136\/bmj-2026-100855","type":"journal-article","created":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T10:35:14Z","timestamp":1791887714000},"page":"e100855","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":0,"title":["Reporting completeness of clinical trials evaluating large language model interventions in medicine: systematic review"],"prefix":"10.1136","volume":"395","author":[{"given":"David","family":"Chen","sequence":"first","affiliation":[{"name":"Division of Radiation Oncology, Department of Surgery, University of British Columbia, Vancouver, BC, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ayden","family":"Blayne","sequence":"additional","affiliation":[{"name":"Division of Radiation Oncology, Department of Surgery, University of British Columbia, Vancouver, BC, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tony","family":"Ning","sequence":"additional","affiliation":[{"name":"Division of Radiation Oncology, Department of Surgery, University of British Columbia, Vancouver, BC, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Inaam","family":"Chattha","sequence":"additional","affiliation":[{"name":"Division of Radiation Oncology, Department of Surgery, University of British Columbia, Vancouver, BC, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Majid","family":"Afshar","sequence":"additional","affiliation":[{"name":"Department of Medicine and Department of Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jasmine Chiat Ling","family":"Ong","sequence":"additional","affiliation":[{"name":"Duke-NUS AI + Medical Sciences Initiative, Duke-NUS Medical School, Singapore"},{"name":"Division of Pharmacy, Singapore General Hospital, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julian C","family":"Hong","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, University of California, San Francisco, San Francisco, CA, USA"},{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA, USA"},{"name":"UCSF UC Berkeley Joint Program in Computational Health, San Francisco, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nan","family":"Liu","sequence":"additional","affiliation":[{"name":"Duke-NUS AI + Medical Sciences Initiative, Duke-NUS Medical School, Singapore"},{"name":"Centre for Biomedical Data Science, Duke-NUS Medical School, Singapore"},{"name":"Pre-hospital and Emergency Research Centre, Health Services Research and Population Health, Duke-NUS Medical School, Singapore"},{"name":"NUS Artificial Intelligence Institute, National University of Singapore, Singapore"},{"name":"Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leo Anthony","family":"Celi","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Critical Care and Sleep Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA"},{"name":"Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA, USA"},{"name":"Department of Biostatistics, Harvard T H Chan School of Public Health, Boston, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Karel G M","family":"Moons","sequence":"additional","affiliation":[{"name":"Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gary S","family":"Collins","sequence":"additional","affiliation":[{"name":"Department of Applied Health Sciences, School of Health Sciences, College of Medicine and Health, University of Birmingham, Birmingham, UK"},{"name":"NIHR Biomedical Research Centre: Birmingham, Birmingham, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0345-2232","authenticated-orcid":true,"given":"Danielle S","family":"Bitterman","sequence":"additional","affiliation":[{"name":"Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA, USA"},{"name":"Department of Radiation Oncology, Brigham and Women\u2019s Hospital\/Dana-Farber Cancer Institute, Boston, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"239","published-online":{"date-parts":[[2026,10,13]]},"reference":[{"key":"2026101307350704000_395.oct13_2.e100855.1","doi-asserted-by":"publisher","DOI":"10.1016\/j.isci.2024.109713"},{"key":"2026101307350704000_395.oct13_2.e100855.2","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2024.21700"},{"key":"2026101307350704000_395.oct13_2.e100855.3","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-025-04348-9"},{"key":"2026101307350704000_395.oct13_2.e100855.4","doi-asserted-by":"publisher","DOI":"10.1093\/nsr\/nwaf050"},{"key":"2026101307350704000_395.oct13_2.e100855.5","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-024-10896-y"},{"key":"2026101307350704000_395.oct13_2.e100855.6","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2025.4347"},{"key":"2026101307350704000_395.oct13_2.e100855.7","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2025.4486"},{"key":"2026101307350704000_395.oct13_2.e100855.8","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m3164"},{"key":"2026101307350704000_395.oct13_2.e100855.9","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m3210"},{"key":"2026101307350704000_395.oct13_2.e100855.10","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-024-03425-5"},{"key":"2026101307350704000_395.oct13_2.e100855.11","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2023-078378"},{"key":"2026101307350704000_395.oct13_2.e100855.12","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2024-083305"},{"key":"2026101307350704000_395.oct13_2.e100855.13","unstructured":"National Institutes of Health. NIH\u2019s Definition of a Clinical Trial. https:\/\/grants.nih.gov\/policy-and-compliance\/policy-topics\/clinical-trials\/definition"},{"key":"2026101307350704000_395.oct13_2.e100855.14","unstructured":"U.S. Food & Drug Administration. FDA Digital Health and Artificial Intelligence Glossary - Educational Resorce. 2024. https:\/\/www.fda.gov\/science-research\/artifical-intelligence-and-medical-products\/fda-digital-health-and-artificial-intelligence-glossary-educational-resource"},{"key":"2026101307350704000_395.oct13_2.e100855.15","unstructured":"Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. 2023. http:\/\/arxiv\/org\/abs\/1706.03762"},{"key":"2026101307350704000_395.oct13_2.e100855.16","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.0050020"},{"key":"2026101307350704000_395.oct13_2.e100855.17","unstructured":"World Health Organization. WHO Trial Registration Data Set (version 1.3.1). https:\/\/www.who.int\/tools\/clinical-trials-registry-platform\/network\/who-data-set"},{"key":"2026101307350704000_395.oct13_2.e100855.18","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-45355-3"},{"key":"2026101307350704000_395.oct13_2.e100855.19","doi-asserted-by":"publisher","DOI":"10.1093\/oncolo\/oyaf112"},{"key":"2026101307350704000_395.oct13_2.e100855.20","doi-asserted-by":"publisher","DOI":"10.1136\/bmjonc-2025-000733"},{"key":"2026101307350704000_395.oct13_2.e100855.21","unstructured":"European Commission. EU AI Act. https:\/\/artificialintelligenceact.eu\/"},{"key":"2026101307350704000_395.oct13_2.e100855.22","unstructured":"International Medical Device Regulators Forum. Good machine learning practice for medical device development: guiding principles. https:\/\/www.imdrf.org\/sites\/default\/files\/2025-02\/IMDRF_AIML%20WG_GMLP_N88%20Final.pdf"},{"key":"2026101307350704000_395.oct13_2.e100855.23","doi-asserted-by":"publisher","DOI":"10.1001\/jamahealthforum.2025.2724"},{"key":"2026101307350704000_395.oct13_2.e100855.24","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocz192"},{"key":"2026101307350704000_395.oct13_2.e100855.25","doi-asserted-by":"publisher","DOI":"10.1001\/jamahealthforum.2023.5514"},{"key":"2026101307350704000_395.oct13_2.e100855.26","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2023.9630"},{"key":"2026101307350704000_395.oct13_2.e100855.27","doi-asserted-by":"publisher","DOI":"10.1056\/AIdbp2500120"},{"key":"2026101307350704000_395.oct13_2.e100855.28","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.40969"},{"key":"2026101307350704000_395.oct13_2.e100855.29","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-018-1168-6"},{"key":"2026101307350704000_395.oct13_2.e100855.30","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2021.09.012"},{"key":"2026101307350704000_395.oct13_2.e100855.31","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-025-04328-z"},{"key":"2026101307350704000_395.oct13_2.e100855.32","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-025-03787-4"},{"key":"2026101307350704000_395.oct13_2.e100855.33","doi-asserted-by":"publisher","DOI":"10.1016\/j.landig.2025.100925"},{"key":"2026101307350704000_395.oct13_2.e100855.34","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-024-03470-0"},{"key":"2026101307350704000_395.oct13_2.e100855.35","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2020-036799"},{"key":"2026101307350704000_395.oct13_2.e100855.36","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocaf223"},{"key":"2026101307350704000_395.oct13_2.e100855.37","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2023.22295"}],"container-title":["BMJ"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/data.bmj.org\/tdm\/10.1136\/bmj-2026-100855","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmj-2026-100855","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T14:35:13Z","timestamp":1791902113000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.bmj.com\/lookup\/doi\/10.1136\/bmj-2026-100855"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,13]]},"references-count":37,"alternative-id":["10.1136\/bmj-2026-100855"],"URL":"https:\/\/doi.org\/10.1136\/bmj-2026-100855","relation":{},"ISSN":["1756-1833"],"issn-type":[{"value":"1756-1833","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,13]]}}}