{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T06:57:12Z","timestamp":1790751432954,"version":"4.1.0"},"reference-count":37,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T00:00:00Z","timestamp":1790726400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>To compare the performance of five widely used large language models in answering public questions about postoperative rehabilitation after hip fracture in older adults, focusing on safety, accuracy, reliability, readability, empathy, and overall information quality.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This cross-sectional comparative evaluation used a 42-question bank addressing general postoperative rehabilitation after proximal femoral fracture treated with internal fixation or hip arthroplasty. All questions and the standardized safety-oriented prompt were submitted in English, and all responses were returned and analyzed in their original English form without translation. Between May 1 and May 10, 2026, each question was submitted once to five generative artificial intelligence-driven chatbots through their official public interfaces, yielding 210 responses. Three independent raters evaluated each response using DISCERN, EQIP, JAMA benchmark criteria, Global Quality Score, and anchored 5-point accuracy and empathy scales. Safety was classified using prespecified binary criteria and summarized descriptively; readability was assessed using six established indices applied to the original English responses. Paired inferential analyses were used for non-binary outcomes.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Inter-rater agreement was excellent (Fleiss' kappa 0.855; ICC 0.829\u20130.884). Under the standardized safety-oriented prompt, 199 of 210 responses (94.8%) met the study's prespecified binary safety criterion; the 11 responses that did not meet the criterion were concentrated in three questions and were summarized descriptively. Accuracy ratings were high across models, with no statistically detectable difference (\n                      <jats:italic>P<\/jats:italic>\n                      = 0.572). Empathy differed across models (\n                      <jats:italic>P<\/jats:italic>\n                      = 0.003; Kendall's\n                      <jats:italic>W<\/jats:italic>\n                      = 0.096), with Gemini showing the highest median score. DISCERN, EQIP, and Global Quality Score varied across models (all\n                      <jats:italic>P<\/jats:italic>\n                      &amp;lt;0.001), although the GQS effect size was small (\n                      <jats:italic>W<\/jats:italic>\n                      = 0.169). Readability was suboptimal: only 2 of 210 responses (1.0%) met the commonly recommended sixth-grade threshold by FKGL.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Under a standardized safety-oriented prompt, most responses met the study's prespecified binary safety criterion and received high accuracy ratings; however, models differed in empathy, reliability, information quality, and readability. These prompt-conditioned findings do not establish unguided LLM responses as generally safe. LLMs may serve only as supervised adjunctive educational tools, with outputs framed as complementary to procedure-specific professional care.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fpubh.2026.1961597","type":"journal-article","created":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T06:07:05Z","timestamp":1790748425000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Performance of large language models in postoperative hip fracture rehabilitation counseling for older adults: a comparative evaluation of safety, accuracy, reliability, readability, and empathy"],"prefix":"10.3389","volume":"14","author":[{"given":"Genrong","family":"Hu","sequence":"first","affiliation":[{"name":"Lower Limb Department I, Quanzhou Orthopedic-traumatological Hospital","place":["Quanzhou, Fujian, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qiuju","family":"Lai","sequence":"additional","affiliation":[{"name":"Lower Limb Department I, Quanzhou Orthopedic-traumatological Hospital","place":["Quanzhou, Fujian, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuqiong","family":"Liu","sequence":"additional","affiliation":[{"name":"Lower Limb Department I, Quanzhou Orthopedic-traumatological Hospital","place":["Quanzhou, Fujian, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Baoyuan","family":"Xie","sequence":"additional","affiliation":[{"name":"Nursing Department, The Second Affiliated Hospital of Fujian Medical University (Donghai Campus)","place":["Quanzhou, Fujian, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhishui","family":"Wu","sequence":"additional","affiliation":[{"name":"Spine Surgery Department, The Third Affiliated Hospital of Soochow University","place":["Changzhou, Jiangsu, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qiongxia","family":"Zhuang","sequence":"additional","affiliation":[{"name":"Lower Limb Department I, Quanzhou Orthopedic-traumatological Hospital","place":["Quanzhou, Fujian, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haiyan","family":"Tang","sequence":"additional","affiliation":[{"name":"Lower Limb Department I, Quanzhou Orthopedic-traumatological Hospital","place":["Quanzhou, Fujian, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jing","family":"Xu","sequence":"additional","affiliation":[{"name":"Lower Limb Department I, Quanzhou Orthopedic-traumatological Hospital","place":["Quanzhou, Fujian, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,30]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"285","DOI":"10.1007\/BF01623184","article-title":"Hip fractures in the elderly: a world-wide projection","volume":"2","author":"Cooper","year":"1992","journal-title":"Osteoporos Int."},{"key":"B2","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1007\/s00198-023-06942-0","article-title":"Systematic review and meta-analysis of preoperative predictors for early mortality following hip fracture surgery","volume":"35","author":"Bui","year":"2024","journal-title":"Osteoporos Int."},{"key":"B3","doi-asserted-by":"publisher","first-page":"1674881","DOI":"10.3389\/fpubh.2025.1674881","article-title":"Global, regional, and national burden of hip fractures attributable to falls in older adults: changes from 1990\u20132021 and 2036 projections","volume":"13","author":"Zhang","year":"2025","journal-title":"Front Public Health."},{"key":"B4","doi-asserted-by":"publisher","first-page":"e1297","DOI":"10.5435\/JAAOS-D-22-00273","article-title":"Management of hip fractures in older adults","volume":"30","author":"O'Connor","year":"2022","journal-title":"J Am Acad Orthop Surg."},{"key":"B5","doi-asserted-by":"publisher","first-page":"CPG1","DOI":"10.2519\/jospt.2021.0301","article-title":"Physical therapy management of older adults with hip fracture","volume":"51","author":"McDonough","year":"2021","journal-title":"J Orthop Sports Phys Ther."},{"key":"B6","doi-asserted-by":"publisher","first-page":"e0315707","DOI":"10.1371\/journal.pone.0315707","article-title":"Effectiveness of home-based exercise for functional rehabilitation in older adults after hip fracture surgery: a systematic review and meta-analysis of randomized controlled trials","volume":"19","author":"Zhao","year":"2024","journal-title":"PLoS ONE."},{"key":"B7","doi-asserted-by":"publisher","first-page":"e58831","DOI":"10.2196\/58831","article-title":"\u201cDoctor ChatGPT, can you help me?","volume":"26","author":"Armbruster","year":"2024","journal-title":"J Med Internet Res."},{"key":"B8","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1111\/hir.12503","article-title":"Internet and social media preferences of orthopaedic patients vary according to factors such as age and education levels","volume":"41","author":"Gencer","year":"2024","journal-title":"Health Info Libr J."},{"key":"B9","doi-asserted-by":"publisher","first-page":"109713","DOI":"10.1016\/j.isci.2024.109713","article-title":"The application of large language models in medicine: a scoping review","volume":"27","author":"Meng","year":"2024","journal-title":"iScience."},{"key":"B10","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","year":"2023","journal-title":"Nat Med."},{"key":"B11","doi-asserted-by":"publisher","first-page":"943","DOI":"10.1038\/s41591-024-03423-7","article-title":"Toward expert-level medical question answering with large language models","volume":"31","author":"Singhal","year":"2025","journal-title":"Nat Med."},{"key":"B12","doi-asserted-by":"publisher","first-page":"656","DOI":"10.5435\/JAAOS-D-23-00877","article-title":"Evaluating ChatGPT's ability to answer common patient questions regarding hip fracture","volume":"32","author":"Wrenn","year":"2024","journal-title":"J Am Acad Orthop Surg."},{"key":"B13","doi-asserted-by":"publisher","first-page":"1834","DOI":"10.3390\/diagnostics15141834","article-title":"To self-treat or not to self-treat: evaluating the diagnostic, advisory and referral effectiveness of ChatGPT responses to the most common musculoskeletal disorders","volume":"15","author":"Arzu","year":"2025","journal-title":"Diagnostics."},{"key":"B14","doi-asserted-by":"publisher","first-page":"2942","DOI":"10.3390\/jcm14092942","article-title":"Evaluation of ChatGPT responses about sexual activity after total hip arthroplasty: a comparative study with observers of different experience levels","volume":"14","author":"Gencer","year":"2025","journal-title":"J Clin Med."},{"key":"B15","doi-asserted-by":"publisher","first-page":"e428","DOI":"10.1016\/S2589-7500(24)00061-X","article-title":"Ethical and regulatory challenges of large language models in medicine","volume":"6","author":"Ong","year":"2024","journal-title":"Lancet Digit Health."},{"key":"B16","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1001\/jama.2024.21700","article-title":"Testing and evaluation of health care applications of large language models: a systematic review","volume":"333","author":"Bedi","year":"2025","journal-title":"JAMA."},{"key":"B17","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1038\/s41746-024-01258-7","article-title":"A framework for human evaluation of large language models in healthcare derived from literature review","volume":"7","author":"Tam","year":"2024","journal-title":"NPJ Digit Med."},{"key":"B18","doi-asserted-by":"publisher","first-page":"e2457879","DOI":"10.1001\/jamanetworkopen.2024.57879","article-title":"Large language models for chatbot health advice studies: a systematic review","volume":"8","author":"Huo","year":"2025","journal-title":"JAMA Netw Open."},{"key":"B19","doi-asserted-by":"publisher","first-page":"e2530220","DOI":"10.1001\/jamanetworkopen.2025.30220","article-title":"Collaborative. Reporting guideline for chatbot health advice studies: the CHART statement","volume":"8","year":"2025","journal-title":"JAMA Netw Open."},{"key":"B20","doi-asserted-by":"publisher","first-page":"105563","DOI":"10.1016\/j.ijmedinf.2024.105563","article-title":"Google trends in infodemiology: methodological steps to avoid irreproducible results and invalid conclusions","volume":"190","author":"Rovetta","year":"2024","journal-title":"Int J Med Inform."},{"key":"B21","doi-asserted-by":"publisher","first-page":"225","DOI":"10.5535\/arm.21110","article-title":"Clinical practice guideline for postoperative rehabilitation in older patients with hip fractures","volume":"45","author":"Min","year":"2021","journal-title":"Ann Rehabil Med."},{"key":"B22","doi-asserted-by":"publisher","first-page":"15988","DOI":"10.3390\/ijerph192315988","article-title":"Delphi-based consensus to determine core aspects of post-hip-fracture surgery rehabilitation based on the international classification of functioning, disability, and health","volume":"19","author":"Lin","year":"2022","journal-title":"Int J Environ Res Public Health."},{"key":"B23","doi-asserted-by":"publisher","first-page":"2191","DOI":"10.1001\/jama.2013.281053","article-title":"World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects","volume":"310","year":"2013","journal-title":"JAMA."},{"key":"B24","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum","volume":"183","author":"Ayers","year":"2023","journal-title":"JAMA Intern Med."},{"key":"B25","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1136\/jech.53.2.105","article-title":"DISCERN: an instrument for judging the quality of written consumer health information on treatment choices","volume":"53","author":"Charnock","year":"1999","journal-title":"J Epidemiol Community Health."},{"key":"B26","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1111\/j.1369-7625.2004.00273.x","article-title":"Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information","volume":"7","author":"Moult","year":"2004","journal-title":"Health Expect."},{"key":"B27","doi-asserted-by":"publisher","first-page":"1244","DOI":"10.1001\/jama.1997.03540390074039","article-title":"Assessing, controlling, and assuring the quality of medical information on the internet: Caveant lector et viewor","volume":"277","author":"Silberg","year":"1997","journal-title":"JAMA."},{"key":"B28","doi-asserted-by":"publisher","first-page":"2070","DOI":"10.1111\/j.1572-0241.2007.01325.x","article-title":"A systematic review of patient inflammatory bowel disease information resources on the World Wide Web","volume":"102","author":"Bernard","year":"2007","journal-title":"Am J Gastroenterol."},{"key":"B29","doi-asserted-by":"publisher","first-page":"3591","DOI":"10.2147\/OPTH.S483024","article-title":"Impact of demographic modifiers on readability of myopia education materials generated by large language models","volume":"18","author":"Lee","year":"2024","journal-title":"Clin Ophthalmol"},{"key":"B30","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1037\/h0057532","article-title":"A new readability yardstick","volume":"32","author":"Flesch","year":"1948","journal-title":"J Appl Psychol."},{"key":"B31","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1037\/h0076540","article-title":"A computer readability formula designed for machine scoring","volume":"60","author":"Coleman","year":"1975","journal-title":"J Appl Psychol."},{"key":"B32","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1080\/19325037.2008.10599016","article-title":"Using the SMOG formula to revise a health-related document","volume":"39","author":"Hedman","year":"2008","journal-title":"Am J Health Educ."},{"key":"B33","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","article-title":"A guideline of selecting and reporting intraclass correlation coefficients for reliability research","volume":"15","author":"Koo","year":"2016","journal-title":"J Chiropr Med."},{"key":"B34","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1037\/h0031619","article-title":"Measuring nominal scale agreement among many raters","volume":"76","author":"Fleiss","year":"1971","journal-title":"Psychol Bull."},{"key":"B35","doi-asserted-by":"publisher","first-page":"675","DOI":"10.1080\/01621459.1937.10503522","article-title":"The use of ranks to avoid the assumption of normality implicit in the analysis of variance","volume":"32","author":"Friedman","year":"1937","journal-title":"J Am Stat Assoc."},{"key":"B36","doi-asserted-by":"publisher","first-page":"20552076251412989","DOI":"10.1177\/20552076251412989","article-title":"Comparative evaluation of large language models for hip fracture-related patient questions: DeepSeek-V3-FW, Gemini 20 Flash, and ChatGPT-45","volume":"12","author":"Zhang","year":"2026","journal-title":"Digit Health."},{"key":"B37","doi-asserted-by":"publisher","first-page":"e60291","DOI":"10.2196\/60291","article-title":"Accuracy of prospective assessments of 4 large language model chatbot responses to patient questions about emergency care: experimental comparative study","volume":"26","author":"Yau","year":"2024","journal-title":"J Med Internet Res."}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1961597\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T06:07:07Z","timestamp":1790748427000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1961597\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,30]]},"references-count":37,"alternative-id":["10.3389\/fpubh.2026.1961597"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2026.1961597","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,30]]},"article-number":"1961597"}}