{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T02:13:23Z","timestamp":1791339203035,"version":"4.3.1"},"reference-count":56,"publisher":"Public Library of Science (PLoS)","issue":"7","license":[{"start":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T00:00:00Z","timestamp":1785283200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>This study aims to comparatively examine the readability, accuracy, and quality of responses provided by artificial intelligence (AI)-based chatbots such as Perplexity, ChatGPT-5, and Gemini to questions about knee osteoarthritis (KOA), which accounts for approximately four-fifths of the global osteoarthritis (OA) burden. In this study, 8 keywords were determined by excluding repetitive, irrelevant or synonymous ones from the 25 most frequently used English keywords associated with KOA based on Google Trends data, and these terms were asked as questions to three different artificial intelligence-based chatbots. The study measured readability using formulas like Coleman-Liau Index (CLI), Automated Readability Index (ARI), and Linsear Write (LW). Reliability of the information was assessed using the Journal of the American Medical Association (JAMA) benchmarks along with the modified DISCERN instrument. To determine overall content quality, the Global Quality Score (GQS) and the Ensuring Quality Information for Patients (EQIP) scale were applied. Together, these tools provided a comprehensive assessment of how understandable, reliable, and high-quality each chatbot\u2019s responses were. The most frequently searched keywords related to OA were \u201costeoarthritis of knee,\u201d \u201cknee pain,\u201d and \u201costeoarthritis knee pain.\u201d A readability analysis of responses from three different AI-based chat systems revealed that all platforms had text levels above the Grade 6 threshold, and this difference was statistically significant (p\u2009&lt;\u20090.05). Comparisons demonstrated that ChatGPT-5 produced the most readable content (FRES:45, GFOG:11.9, FKGL:9.24, CLI:14.03, SMOG:8.37, ARI:11.37, LW:7.2). However, Perplexity achieved significantly higher scores than ChatGPT-5 across all quality and reliability assessments, yielding superior median scores (DISCERN: 4, JAMA: 2, GQS: 4, EQIP: 92.8). Perplexity also outperformed Gemini in the mDISCERN reliability assessment (p\u2009=\u20090.001), while no significant difference in quality or reliability was found between Gemini and ChatGPT-5. No statistically significant difference was found between Gemini and ChatGPT in reliability and quality surveys. This analysis of KOA highlights significant challenges regarding the potential of popular AI chatbots for patient information. When examining readability levels, responses from these tools consistently exceed the recommended comprehensibility threshold, making it difficult for patients to absorb critical information. Furthermore, the relatively low scores recorded in reliability and content quality assessments raise significant concerns about the scientific validity and integrity of the medical information presented. Given these findings, the sufficient quality, robustness, and appropriate levels of understandability of future AI-based tools can only be ensured by the establishment and operation of an effective oversight mechanism.<\/jats:p>","DOI":"10.1371\/journal.pone.0353355","type":"journal-article","created":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T17:38:18Z","timestamp":1785346698000},"page":"e0353355","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":4,"title":["A comparative analysis of readability, quality, and reliability in large language model outputs pertaining to knee osteoarthritis queries"],"prefix":"10.1371","volume":"21","author":[{"given":"Erdem","family":"Mara\u015fl\u0131","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3425-313X","authenticated-orcid":true,"given":"Erkan","family":"Ozduran","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Volkan","family":"Hanc\u0131","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,7,29]]},"reference":[{"key":"pone.0353355.ref001","doi-asserted-by":"crossref","first-page":"101346","DOI":"10.1016\/j.cegh.2023.101346","article-title":"Knee pain and its influence on quality of life and physical function in Jazan adult population: A cross sectional study","volume":"22","author":"Z Khired","year":"2023","journal-title":"Clinical Epidemiology and Global Health"},{"key":"pone.0353355.ref002","doi-asserted-by":"crossref","first-page":"100587","DOI":"10.1016\/j.eclinm.2020.100587","article-title":"Global, regional prevalence, incidence and risk factors of knee osteoarthritis in population-based studies","author":"A Cui","year":"2020","journal-title":"EClinicalMedicine"},{"issue":"2","key":"pone.0353355.ref003","first-page":"205","article-title":"Knee osteoarthritis prevalence, risk factors, pathogenesis and features: Part I","volume":"2","author":"B Heidari","year":"2011","journal-title":"Caspian J Intern Med"},{"issue":"4","key":"pone.0353355.ref004","doi-asserted-by":"crossref","first-page":"668","DOI":"10.3390\/jcm10040668","article-title":"Estimating the prevalence of knee pain and the association between illness perception profiles and self-management strategies in the frederiksberg cohort of elderly individuals with knee pain: A cross-sectional study","volume":"10","author":"E Ginnerup-Nielsen","year":"2021","journal-title":"J Clin Med"},{"issue":"1","key":"pone.0353355.ref005","doi-asserted-by":"crossref","first-page":"134","DOI":"10.36469\/jheor.2024.117155","article-title":"Reduction in healthcare resource utilization following treatment with a home-based footworn device in patients with knee osteoarthritis: A retrospective claims analysis","volume":"11","author":"J Mark","year":"2024","journal-title":"J Health Econ Outcomes Res"},{"key":"pone.0353355.ref006","doi-asserted-by":"crossref","first-page":"1583153","DOI":"10.3389\/fphys.2025.1583153","article-title":"Strength training intervention for adult individuals with knee osteoarthritis: Establishing fidelity","volume":"16","author":"H K\u00fc\u00e7\u00fck","year":"2025","journal-title":"Front Physiol"},{"issue":"2","key":"pone.0353355.ref007","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1002\/art.41142","article-title":"2019 American College of Rheumatology\/Arthritis Foundation guideline for the management of osteoarthritis of the hand, hip, and knee","volume":"72","author":"SL Kolasinski","year":"2020","journal-title":"Arthritis & Rheumatology (Hoboken, NJ)"},{"key":"pone.0353355.ref008","doi-asserted-by":"crossref","first-page":"108917","DOI":"10.1016\/j.compbiomed.2024.108917","article-title":"Advancements in AI based healthcare techniques with FOCUS ON diagnostic techniques","volume":"179","author":"N Kalra","year":"2024","journal-title":"Comput Biol Med"},{"issue":"5","key":"pone.0353355.ref009","doi-asserted-by":"crossref","first-page":"1184","DOI":"10.1016\/j.arth.2024.01.029","article-title":"Chat Generative Pretrained Transformer (ChatGPT) and Bard: Artificial intelligence does not yet provide clinically supported answers for hip and knee osteoarthritis","volume":"39","author":"J Yang","year":"2024","journal-title":"J Arthroplasty"},{"key":"pone.0353355.ref010","doi-asserted-by":"crossref","DOI":"10.2196\/48392","article-title":"The ChatGPT (Generative Artificial Intelligence) revolution has made artificial intelligence approachable for medical professionals","volume":"25","author":"B Mesko","year":"2023","journal-title":"J Med Internet Res"},{"key":"pone.0353355.ref011","doi-asserted-by":"crossref","first-page":"1226776","DOI":"10.3389\/fpubh.2023.1226776","article-title":"AI chatbots and (mis)information in public health: impact on vulnerable communities","volume":"11","author":"DW Meyrowitsch","year":"2023","journal-title":"Front Public Health"},{"issue":"6","key":"pone.0353355.ref012","doi-asserted-by":"crossref","DOI":"10.1111\/jep.70269","article-title":"Artificial general intelligence and its threat to public health","volume":"31","author":"RC Armitage","year":"2025","journal-title":"J Eval Clin Pract"},{"key":"pone.0353355.ref013","author":"Z Bai","year":"2024"},{"issue":"5","key":"pone.0353355.ref014","doi-asserted-by":"crossref","first-page":"562","DOI":"10.3390\/healthcare12050562","article-title":"Global regulatory frameworks for the use of artificial intelligence (AI) in the healthcare services sector","volume":"12","author":"K Palaniappan","year":"2024","journal-title":"Healthcare (Basel)"},{"issue":"1","key":"pone.0353355.ref015","doi-asserted-by":"crossref","first-page":"247","DOI":"10.30621\/jbachs.1018824","article-title":"Evaluation of the 100 most cited articles on Behcet\u2019s disease: A bibliometric analysis","volume":"6","author":"E \u00d6zduran","year":"2022","journal-title":"Journal of Basic and Clinical Health Sciences"},{"issue":"4","key":"pone.0353355.ref016","doi-asserted-by":"crossref","first-page":"907","DOI":"10.54029\/2023kif","article-title":"Youtube as a source of information about stroke rehabilitation during the COVID-19 pandemic","volume":"28","author":"E \u00d6zduran","year":"2023","journal-title":"NeuroAsia"},{"issue":"27","key":"pone.0353355.ref017","article-title":"Exploring online health information-seeking behavior among young adults: Scoping review","volume":"9","author":"K Stifjell","year":"2025","journal-title":"J Med Internet Res"},{"key":"pone.0353355.ref018","doi-asserted-by":"crossref","DOI":"10.2196\/69955","article-title":"Enhancing the readability of online patient education materials using large language models: Cross-sectional study","volume":"27","author":"J Will","year":"2025","journal-title":"J Med Internet Res"},{"issue":"1","key":"pone.0353355.ref019","doi-asserted-by":"crossref","first-page":"1148","DOI":"10.1186\/s12913-022-08527-9","article-title":"Impact of low health literacy on patients\u2019 health outcomes: A multicenter cohort study","volume":"22","author":"R Shahid","year":"2022","journal-title":"BMC Health Serv Res"},{"issue":"8","key":"pone.0353355.ref020","doi-asserted-by":"crossref","DOI":"10.1161\/JAHA.124.033916","article-title":"Promoting personal health literacy through readability, understandability, and actionability of online patient education materials","volume":"13","author":"CA Tucker","year":"2024","journal-title":"J Am Heart Assoc"},{"issue":"10","key":"pone.0353355.ref021","doi-asserted-by":"crossref","first-page":"1437","DOI":"10.1001\/jamaoncol.2023.2947","article-title":"Assessment of artificial intelligence chatbot responses to top searched queries about cancer","volume":"9","author":"A Pan","year":"2023","journal-title":"JAMA Oncol"},{"issue":"7","key":"pone.0353355.ref022","doi-asserted-by":"crossref","first-page":"1425684","DOI":"10.3389\/fdgth.2025.1425684","article-title":"The COVID-19 pandemic and the worldwide online interest in telepsychiatry: An infodemiological study from 2004 to 2022","volume":"1","author":"R Alibudbud","year":"2025","journal-title":"Front Digit Health"},{"issue":"18","key":"pone.0353355.ref023","article-title":"How artificial intelligence can provide information about subdural hematoma: Assessment of readability, reliability, and quality of ChatGPT, BARD, and perplexity responses","volume":"103","author":"\u015e G\u00fcl","year":"2024","journal-title":"Medicine (Baltimore)"},{"issue":"1","key":"pone.0353355.ref024","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1186\/s40942-024-00579-9","article-title":"Evaluation of the accuracy and readability of ChatGPT-4 and Google Gemini in providing information on retinal detachment: A multicenter expert comparative study","volume":"10","author":"P Strzalkowski","year":"2024","journal-title":"Int J Retina Vitreous"},{"issue":"22","key":"pone.0353355.ref025","article-title":"Assessing the readability, reliability, and quality of artificial intelligence chatbot responses to the 100 most searched queries about cardiopulmonary resuscitation: An observational study","volume":"103","author":"D \u00d6m\u00fcr","year":"2024","journal-title":"Med"},{"issue":"33","key":"pone.0353355.ref026","doi-asserted-by":"crossref","DOI":"10.1097\/MD.0000000000039305","article-title":"Assessment of readability, reliability, and quality of ChatGPT\u00ae, BARD\u00ae, Gemini\u00ae, Copilot\u00ae, Perplexity\u00ae responses on palliative care","volume":"103","author":"V Hanc\u0131","year":"2024","journal-title":"Medicine (Baltimore)"},{"issue":"2","key":"pone.0353355.ref027","doi-asserted-by":"crossref","DOI":"10.4081\/reumatismo.2022.1495","article-title":"Evaluating the readability, quality and reliability of online information on Beh\u00e7et\u2019s disease","volume":"74","author":"E \u00d6zduran","year":"2022","journal-title":"Reumatismo"},{"key":"pone.0353355.ref028","doi-asserted-by":"crossref","DOI":"10.7717\/peerj.14089","article-title":"A content analysis of the reliability and quality of Youtube videos as a source of information on health-related post-COVID pain","volume":"10","author":"E Ozduran","year":"2022","journal-title":"PeerJ"},{"issue":"16","key":"pone.0353355.ref029","doi-asserted-by":"crossref","DOI":"10.1097\/MD.0000000000033529","article-title":"Evaluating the readability, quality and reliability of online patient education materials on transcutaneuous electrical nerve stimulation (TENS)","volume":"102","author":"Y Erkin","year":"2023","journal-title":"Medicine (Baltimore)"},{"issue":"15","key":"pone.0353355.ref030","doi-asserted-by":"crossref","first-page":"1244","DOI":"10.1001\/jama.1997.03540390074039","article-title":"Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor--Let the reader and viewer beware","volume":"277","author":"WM Silberg","year":"1997","journal-title":"JAMA"},{"issue":"2","key":"pone.0353355.ref031","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1136\/jech.53.2.105","article-title":"DISCERN: An instrument for judging the quality of written consumer health information on treatment choices","volume":"53","author":"D Charnock","year":"1999","journal-title":"J Epidemiol Community Health"},{"key":"pone.0353355.ref032","doi-asserted-by":"crossref","DOI":"10.7717\/peerj.17215","article-title":"Assessing the quality and reliability of YouTube videos as a source of information on inflammatory back pain","volume":"12","author":"M Kara","year":"2024","journal-title":"PeerJ"},{"issue":"4","key":"pone.0353355.ref033","first-page":"588","article-title":"Evaluating the readability, quality, and reliability of online patient education materials on spinal cord stimulation","volume":"34","author":"ME Gunduz","year":"2024","journal-title":"Turk Neurosurg"},{"issue":"1","key":"pone.0353355.ref034","doi-asserted-by":"crossref","first-page":"16","DOI":"10.4103\/injr.injr_56_22","article-title":"Evaluating the readability, quality, and reliability of online information on sjogren\u2019s syndrome","volume":"18","author":"E Ozduran","year":"2023","journal-title":"Indian Journal of Rheumatology"},{"key":"pone.0353355.ref035","doi-asserted-by":"crossref","DOI":"10.7717\/peerj.18847","article-title":"Assessing the readability, quality and reliability of responses produced by ChatGPT, Gemini, and Perplexity regarding most frequently asked keywords about low back pain","volume":"13","author":"E Ozduran","year":"2025","journal-title":"PeerJ"},{"issue":"9","key":"pone.0353355.ref036","doi-asserted-by":"crossref","first-page":"2070","DOI":"10.1111\/j.1572-0241.2007.01325.x","article-title":"A systematic review of patient inflammatory bowel disease information resources on the World Wide Web","volume":"102","author":"A Bernard","year":"2007","journal-title":"Am J Gastroenterol"},{"issue":"10","key":"pone.0353355.ref037","doi-asserted-by":"crossref","first-page":"858","DOI":"10.1016\/j.cjco.2022.07.004","article-title":"Evaluation of online written medication educational resources for people living with heart failure","volume":"4","author":"S Ladhar","year":"2022","journal-title":"CJC Open"},{"issue":"11","key":"pone.0353355.ref038","doi-asserted-by":"crossref","DOI":"10.1097\/MD.0000000000041780","article-title":"Readability, reliability and quality of responses generated by ChatGPT, gemini, and perplexity for the most frequently asked questions about pain","volume":"104","author":"E Ozduran","year":"2025","journal-title":"Medicine (Baltimore)"},{"key":"pone.0353355.ref039","doi-asserted-by":"crossref","first-page":"106997","DOI":"10.1016\/j.ypmed.2022.106997","article-title":"Electronic health literacy and health-related outcomes among older adults: A systematic review","volume":"157","author":"L Xie","year":"2022","journal-title":"Prev Med"},{"issue":"1","key":"pone.0353355.ref040","doi-asserted-by":"crossref","DOI":"10.2196\/jmir.1619","article-title":"eHealth literacy: Extending the digital divide to the realm of health information","volume":"14","author":"E Neter","year":"2012","journal-title":"J Med Internet Res"},{"issue":"3","key":"pone.0353355.ref041","doi-asserted-by":"crossref","DOI":"10.2196\/jmir.6472","article-title":"With some help from my network: Supplementing eHealth literacy with social ties","volume":"19","author":"TZ Hayat","year":"2017","journal-title":"J Med Internet Res"},{"issue":"2","key":"pone.0353355.ref042","doi-asserted-by":"crossref","first-page":"97","DOI":"10.7326\/0003-4819-155-2-201107190-00005","article-title":"Low health literacy and health outcomes: An updated systematic review","volume":"155","author":"ND Berkman","year":"2011","journal-title":"Ann Intern Med"},{"issue":"5","key":"pone.0353355.ref043","doi-asserted-by":"crossref","first-page":"1313","DOI":"10.1016\/j.jos.2023.08.003","article-title":"Readability and quality of online patient resources regarding knee osteoarthritis and lumbar spinal stenosis in Japan","volume":"29","author":"A Yamamura","year":"2024","journal-title":"J Orthop Sci"},{"issue":"3","key":"pone.0353355.ref044","first-page":"185","article-title":"Comprehensiveness, accuracy, quality, credibility and readability of online information about knee osteoarthritis","volume":"52","author":"AJ Goff","year":"2023","journal-title":"Health Inf Manag"},{"issue":"1","key":"pone.0353355.ref045","doi-asserted-by":"crossref","first-page":"14604582251315587","DOI":"10.1177\/14604582251315587","article-title":"Presentation suitability and readability of ChatGPT\u2019s medical responses to patient questions about on knee osteoarthritis","volume":"31","author":"M Yoo","year":"2025","journal-title":"Health Informatics J"},{"issue":"5","key":"pone.0353355.ref046","doi-asserted-by":"crossref","first-page":"495","DOI":"10.3390\/jpm14050495","article-title":"Assessment of the quality and readability of information provided by ChatGPT in relation to the use of platelet-rich plasma therapy for osteoarthritis","volume":"14","author":"S Fahy","year":"2024","journal-title":"J Pers Med"},{"key":"pone.0353355.ref047","doi-asserted-by":"crossref","first-page":"1480381","DOI":"10.3389\/fdgth.2024.1480381","article-title":"Enhancing patient education on the role of tibial osteotomy in the management of knee osteoarthritis using a customized ChatGPT: A readability and quality assessment","volume":"6","author":"S Fahy","year":"2025","journal-title":"Front Digit Health"},{"issue":"7","key":"pone.0353355.ref048","article-title":"Comparing artificial intelligence-generated and clinician-created personalized self-management guidance for patients with knee osteoarthritis: blinded observational study","volume":"27","author":"K Du","year":"2025","journal-title":"J Med Internet Res"},{"issue":"4","key":"pone.0353355.ref049","doi-asserted-by":"crossref","first-page":"10","DOI":"10.55524\/ijircst.2024.12.4.2","article-title":"Comparative study of ChatGPT, Gemini, and Perplexity","volume":"12","author":"M Shukla","year":"2024","journal-title":"IJIRCST"},{"key":"pone.0353355.ref050","doi-asserted-by":"crossref","unstructured":"Arora C, Sayeed AI, Licorish S, Wang F, Treude C. Optimizing LLMs for Code Generation: Which Hyperparameter Settings Yield the Best Results?. In: 2024 31st Asia-Pacific Software Engineering Conference (APSEC), 2024. 281\u201390. https:\/\/doi.org\/10.1109\/apsec65559.2024.00039","DOI":"10.1109\/APSEC65559.2024.00039"},{"key":"pone.0353355.ref051","article-title":"Healthbench: Evaluating large language models towards improved human health","author":"RK Arora","year":"2025"},{"issue":"2","key":"pone.0353355.ref052","first-page":"135","article-title":"Evaluation of the readability of Turkish internet-based patient education materials related to \u201clow back pain\u201d","volume":"36","author":"E \u00d6zduran","year":"2022","journal-title":"Journal of Dokuz Eylul University Medical Faculty"},{"issue":"1","key":"pone.0353355.ref053","first-page":"12","article-title":"Digital guidance: Quality and readability analysis of artificial intelligence-generated spondyloarthropathy texts","volume":"31","author":"\u0130C \u00d6zbek","year":"2025","journal-title":"Turk J Osteoporos"},{"issue":"2","key":"pone.0353355.ref054","first-page":"245","article-title":"Evaluating the readability, quality, and reliability of responses generated by artificial intelligence programs about acupuncture for pain","volume":"29","author":"A Mengi","year":"2026","journal-title":"Journal of Physical Medicine and Rehabilitation Sciences"},{"issue":"1","key":"pone.0353355.ref055","doi-asserted-by":"crossref","first-page":"15","DOI":"10.26650\/jchild.2024.1301406","article-title":"Exploring the reliability and accessibility of youtube for cerebral palsy rehabilitation information","volume":"24","author":"E \u00d6zduran","year":"2024","journal-title":"jchild"},{"key":"pone.0353355.ref056","article-title":"Digital rehabilitation in Parkinson\u2019s disease: The role of artificial intelligence-assisted exercise training","author":"\u0130C \u00d6zbek","year":"2025","journal-title":"Turk J Osteoporos"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0353355","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T17:38:25Z","timestamp":1785346705000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0353355"}},"subtitle":[],"editor":[{"given":"Hamza","family":"K\u00fc\u00e7\u00fck","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,7,29]]},"references-count":56,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2026,7,29]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0353355","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,29]]}}}