{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T06:14:27Z","timestamp":1788243267445,"version":"build-2803163510"},"reference-count":34,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T00:00:00Z","timestamp":1788220800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Large language model (LLM)-based chatbots are increasingly used by the public to obtain health information, but their performance in answering questions related to traumatic brain injury (TBI) and concussion remains unclear. This study evaluated five publicly accessible LLM-based chatbots across safety, accuracy, empathy, information quality, transparency, and readability.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Sixty-five English-language, public-facing questions about TBI and concussion were submitted to ChatGPT, Gemini, Copilot, DeepSeek, and Doubao, generating 325 responses. Five blinded independent raters assessed the generated responses using guideline-informed criteria and established tools, including DISCERN, EQIP, JAMA benchmark criteria, the Global Quality Score, and readability indices.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Inter-rater agreement was high. The proportions of responses rated as safe ranged from 89.2 to 95.4%, and all models achieved a median accuracy score of 4.00. However, 27 responses were classified as potentially harmful, mainly because of under-triage, overly reassuring advice regarding imaging findings, premature return-to-activity or driving guidance, and insufficient pediatric caution. Accuracy differences were statistically significant but small, whereas empathy, information quality, transparency, and readability showed clearer model-level variation. DeepSeek produced the easiest-to-read responses.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>LLM-based chatbots generated responses that were generally rated as safe and informative by expert evaluators, but potentially harmful advice and readability problems remained. These findings characterize expert-rated response performance and do not establish patient comprehension, educational effectiveness, or clinical benefit. LLM-based chatbots may have potential as supplementary sources of patient-facing health information, but they should not replace professional medical evaluation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fpubh.2026.1926980","type":"journal-article","created":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T05:26:35Z","timestamp":1788240395000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Safety, accuracy, empathy, information quality, and readability of publicly accessible LLM-based chatbots for traumatic brain injury and concussion questions: a cross-sectional comparative study"],"prefix":"10.3389","volume":"14","author":[{"given":"Xin","family":"Zuo","sequence":"first","affiliation":[{"name":"Department of Acupuncture and Moxibustion, Heilongjiang University of Traditional Chinese Medicine","place":["Harbin, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huan","family":"Zuo","sequence":"additional","affiliation":[{"name":"Department of Acupuncture and Moxibustion, Heilongjiang University of Traditional Chinese Medicine","place":["Harbin, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Min","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Neurology, Xiamen Humanity Hospital, Fujian Medical University","place":["Xiamen, Fujian, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weihong","family":"Zheng","sequence":"additional","affiliation":[{"name":"Department of Neurology, Xiamen Humanity Hospital, Fujian Medical University","place":["Xiamen, Fujian, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,1]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1016\/S1474-4422(18)30415-0","article-title":"Global, regional, and national burden of traumatic brain injury and spinal cord injury, 1990\u20132016: a systematic analysis for the global burden of disease study 2016","volume":"18","author":"James","year":"2019","journal-title":"Lancet Neurol"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"987","DOI":"10.1016\/S1474-4422(17)30371-X","article-title":"Traumatic brain injury: integrated approaches to improve prevention, clinical care, and research","volume":"16","author":"Maas","year":"2017","journal-title":"Lancet Neurol"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"1080","DOI":"10.3171\/2017.10.JNS17352","article-title":"Estimating the global incidence of traumatic brain injury","volume":"130","author":"Dewan","year":"2019","journal-title":"J Neurosurg"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"1343","DOI":"10.1016\/j.apmr.2023.03.036","article-title":"The American congress of rehabilitation medicine diagnostic criteria for mild traumatic brain injury","volume":"104","author":"Silverberg","year":"2023","journal-title":"Arch Phys Med Rehabil"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1227\/NEU.0000000000001432","article-title":"Guidelines for the management of severe traumatic brain injury, fourth edition","volume":"80","author":"Carney","year":"2017","journal-title":"Neurosurgery"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"e182853","DOI":"10.1001\/jamapediatrics.2018.2853","article-title":"Centers for disease control and prevention guideline on the diagnosis and management of mild traumatic brain injury among children","volume":"172","author":"Lumba-Brown","year":"2018","journal-title":"JAMA Pediatr"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"1049","DOI":"10.1001\/jamaneurol.2019.1313","article-title":"Recovery after mild traumatic brain injury in patients presenting to US level I trauma centers: a transforming research and clinical knowledge in traumatic brain injury (TRACK-TBI) study","volume":"76","author":"Nelson","year":"2019","journal-title":"JAMA Neurol"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"1014","DOI":"10.1001\/jama.2016.1203","article-title":"Clinical risk score for persistent postconcussion symptoms among children with acute concussion in the ED","volume":"315","author":"Zemek","year":"2016","journal-title":"JAMA"},{"key":"ref9","year":"2023"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"S13","DOI":"10.1016\/j.jacr.2021.01.006","article-title":"ACR appropriateness criteria\u00ae head trauma: 2021 update","volume":"18","author":"Shih","year":"2021","journal-title":"J Am Coll Radiol"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"e63","DOI":"10.1016\/j.annemergmed.2023.01.014","article-title":"Clinical policy: critical issues in the management of adult patients presenting to the emergency department with mild traumatic brain injury","volume":"81","author":"Valente","year":"2023","journal-title":"Ann Emerg Med"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"695","DOI":"10.1136\/bjsports-2023-106898","article-title":"Consensus statement on concussion in sport: the 6th international conference on concussion in sport\u2013Amsterdam, October 2022","volume":"57","author":"Patricios","year":"2023","journal-title":"Br J Sports Med"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"693","DOI":"10.1080\/02699052.2022.2033848","article-title":"The management and rehabilitation of post-acute mild traumatic brain injury","volume":"36","author":"Eapen","year":"2022","journal-title":"Brain Inj"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"838","DOI":"10.1136\/bjsports-2017-097699","article-title":"Consensus statement on concussion in sport\u2014the 5th international conference on concussion in sport held in Berlin, October 2016","volume":"51","author":"McCrory","year":"2017","journal-title":"Br J Sports Med"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"e20183074","DOI":"10.1542\/peds.2018-3074","article-title":"Sport-related concussion in children and adolescents","volume":"142","author":"Halstead","year":"2018","journal-title":"Pediatrics"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"S1","DOI":"10.1097\/PCC.0000000000001735","article-title":"Guidelines for the management of pediatric severe traumatic brain injury, third edition: update of the brain trauma foundation guidelines","volume":"20","author":"Kochanek","year":"2019","journal-title":"Pediatr Crit Care Med"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"487","DOI":"10.1177\/1062860617751639","article-title":"Readability of online health information: a meta-narrative systematic review","volume":"33","author":"Daraz","year":"2018","journal-title":"Am J Med Qual"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"e264","DOI":"10.2196\/jmir.6349","article-title":"Health literacy and health information technology adoption: the potential for a new digital divide","volume":"18","author":"Mackert","year":"2016","journal-title":"J Med Internet Res"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"1079","DOI":"10.1016\/j.pec.2016.01.020","article-title":"Health literacy and adherence to medical treatment in chronic and acute illness: A meta-analysis","volume":"99","author":"Miller","year":"2016","journal-title":"Patient Educ Couns"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLoS Digit Health"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum","volume":"183","author":"Ayers","year":"2023","journal-title":"JAMA Intern Med"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"887","DOI":"10.3390\/healthcare11060887","article-title":"ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns","volume":"11","author":"Sallam","year":"2023","journal-title":"Health Care (Don Mills)"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"1233","DOI":"10.1056\/NEJMsr2214184","article-title":"Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine","volume":"388","author":"Lee","year":"2023","journal-title":"N Engl J Med"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","year":"2023","journal-title":"Nat Med"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"842","DOI":"10.1001\/jama.2023.1044","article-title":"Appropriateness of cardiovascular disease prevention recommendations obtained from a popular online chat-based artificial intelligence model","volume":"329","author":"Sarraju","year":"2023","journal-title":"JAMA"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"721","DOI":"10.3350\/cmh.2023.0089","article-title":"Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma","volume":"29","author":"Yeo","year":"2023","journal-title":"Clin Mol Hepatol"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"e46939","DOI":"10.2196\/46939","article-title":"Putting ChatGPT\u2019s medical advice to the (turing) test: survey study","volume":"9","author":"Nov","year":"2023","journal-title":"JMIR Med Educ"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1370\/afm.250386","article-title":"Reporting guideline for chatbot health advice studies: chatbot assessment reporting tool (CHART) statement","volume":"23","year":"2025","journal-title":"Ann Fam Med"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","article-title":"A guideline of selecting and reporting Intraclass correlation coefficients for reliability research","volume":"15","author":"Koo","year":"2016","journal-title":"J Chiropr Med"},{"key":"ref31","doi-asserted-by":"publisher","first-page":"pkad015","DOI":"10.1093\/jncics\/pkad015","article-title":"Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information","volume":"7","author":"Johnson","year":"2023","journal-title":"JNCI Cancer Spectr"},{"key":"ref32","doi-asserted-by":"publisher","first-page":"506","DOI":"10.3390\/healthcare10030506","article-title":"YouTube as a source of patient information for meningiomas: a content quality and audience engagement analysis","volume":"10","author":"Krakowiak","year":"2022","journal-title":"Healthcare"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"349","DOI":"10.1097\/SAP.0000000000000438","article-title":"Online patient resources for liposuction: a comparative analysis of readability","volume":"76","author":"Vargas","year":"2016","journal-title":"Ann Plast Surg"},{"key":"ref34","doi-asserted-by":"publisher","first-page":"926","DOI":"10.1007\/s13187-017-1181-z","article-title":"Accuracy and readability of websites on kidney and bladder cancers","volume":"33","author":"Azer","year":"2018","journal-title":"J Canc Educ"}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1926980\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T05:26:36Z","timestamp":1788240396000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1926980\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,1]]},"references-count":34,"alternative-id":["10.3389\/fpubh.2026.1926980"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2026.1926980","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,1]]},"article-number":"1926980"}}