{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,11]],"date-time":"2026-11-11T15:38:51Z","timestamp":1794411531276,"version":"4.9.0"},"reference-count":40,"publisher":"Ovid Technologies (Wolters Kluwer Health)","issue":"11","license":[{"start":{"date-parts":[[2025,3,14]],"date-time":"2025-03-14T00:00:00Z","timestamp":1741910400000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,3,14]]},"abstract":"<jats:p>\n                    It is clear that artificial intelligence-based chatbots will be popular applications in the field of healthcare in the near future. It is known that more than 30% of the world\u2019s population suffers from chronic pain and individuals try to access the health information they need through online platforms before applying to the hospital. This study aimed to examine the readability, reliability and quality of the responses given by 3 different artificial intelligence chatbots (ChatGPT, Gemini and Perplexity) to frequently asked questions about pain. In this study, the 25 most frequently used keywords related to pain were determined using Google Trend and asked to every 3 artificial intelligence chatbots. The readability of the response texts was determined by Flesch Reading Ease Score (FRES), Simple Measure of Gobbledygook, Gunning Fog and Flesch-Kincaid Grade Level readability scoring. Reliability assessment was determined by the Journal of American Medical Association (JAMA), DISCERN scales. Global Quality Score and Ensuring Quality Information for Patients (EQIP) score were used in quality assessment. As a result of Google Trend search, the first 3 keywords were determined as \u201cback pain,\u201d \u201cstomach pain,\u201d and \u201cchest pain.\u201d The readability of the answers given by all 3 artificial intelligence applications was determined to be higher than the recommended 6th grade readability level (\n                    <jats:italic toggle=\"yes\">P<\/jats:italic>\n                    \u2005&lt;\u2005.001). In the readability evaluation, the order from easy to difficult was determined as Google Gemini, ChatGPT and Perplexity. Higher GQS scores (\n                    <jats:italic toggle=\"yes\">P<\/jats:italic>\n                    \u2005=\u2005.008) were detected in Gemini compared to other chatbots. Perplexity had higher JAMA, DISCERN and EQIP scores compared to other chatbots, respectively (\n                    <jats:italic toggle=\"yes\">P<\/jats:italic>\n                    \u2005&lt;\u2005.001,\n                    <jats:italic toggle=\"yes\">P<\/jats:italic>\n                    \u2005&lt;\u2005.001,\n                    <jats:italic toggle=\"yes\">P<\/jats:italic>\n                    \u2005&lt;\u2005.05). It has been determined that the answers given by ChatGPT, Gemini, and Perplexity to pain-related questions are difficult to read and their reliability and quality are low. It can be stated that these artificial intelligence chatbots cannot replace a comprehensive medical consultation. In artificial intelligence applications, it may be recommended to facilitate the readability of text content, create texts containing reliable references, and control them by a supervisory expert team.\n                  <\/jats:p>","DOI":"10.1097\/md.0000000000041780","type":"journal-article","created":{"date-parts":[[2025,3,18]],"date-time":"2025-03-18T15:03:56Z","timestamp":1742310236000},"page":"e41780","update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":67,"title":["Readability, reliability and quality of responses generated by ChatGPT, gemini, and perplexity for the most frequently asked questions about pain"],"prefix":"10.1097","volume":"104","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3425-313X","authenticated-orcid":false,"given":"Erkan","family":"Ozduran","sequence":"first","affiliation":[{"name":"Sivas Numune Hospital, Physical Medicine and Rehabilitation, Pain Medicine, Sivas, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3614-5274","authenticated-orcid":false,"given":"Ibrahim","family":"Akkoc","sequence":"additional","affiliation":[{"name":"University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Anesthesiology and Reanimation, Istanbul, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5756-980X","authenticated-orcid":false,"given":"Sibel","family":"B\u00fcy\u00fck\u00e7oban","sequence":"additional","affiliation":[{"name":"Dokuz Eylul University, Anesthesiology and Reanimation, Izmir, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5859-2817","authenticated-orcid":false,"given":"Y\u00fcksel","family":"Erkin","sequence":"additional","affiliation":[{"name":"Dokuz Eylul University, Anesthesiology and Reanimation, Pain Medicine, Izmir, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2227-194X","authenticated-orcid":false,"given":"Volkan","family":"Hanci","sequence":"additional","affiliation":[{"name":"Dokuz Eylul University, Anesthesiology and Reanimation, Critical Care Medicine, Izmir, Turkey."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"276","reference":[{"key":"R1-20260108","doi-asserted-by":"crossref","first-page":"770","DOI":"10.1186\/1471-2458-11-770","article-title":"Pain as a global public health priority.","volume":"11","author":"Goldberg","year":"2011","journal-title":"BMC Public Health"},{"key":"R2-20260108","doi-asserted-by":"crossref","first-page":"e44759","DOI":"10.2196\/44759","article-title":"The current status of the use of internet hospitals for outpatients with pain: retrospective study.","volume":"25","author":"Sang","year":"2023","journal-title":"J Med Internet Res"},{"key":"R3-20260108","doi-asserted-by":"crossref","first-page":"e27214","DOI":"10.2196\/27214","article-title":"Google trends for pain search terms in the world\u2019s most populated regions before and after the first recorded COVID-19 case: infodemiological study.","volume":"23","author":"Szilagyi","year":"2021","journal-title":"J Med Internet Res"},{"key":"R4-20260108","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1186\/s12875-018-0805-7","article-title":"Odds of talking to healthcare providers as the initial source of healthcare information: updated cross-sectional results from the Health Information National Trends Survey (HINTS).","volume":"19","author":"Swoboda","year":"2018","journal-title":"BMC Fam Pract"},{"key":"R5-20260108","doi-asserted-by":"crossref","first-page":"218","DOI":"10.4081\/reumatismo.2020.1325","article-title":"Top 100 cited articles on ankylosing spondylitis.","volume":"72","author":"Bagcier","year":"2021","journal-title":"Reumatismo"},{"key":"R6-20260108","doi-asserted-by":"crossref","first-page":"e38009","DOI":"10.1097\/MD.0000000000038009","article-title":"How artificial intelligence can provide information about subdural hematoma: assessment of readability, reliability, and quality of ChatGPT, BARD, and perplexity responses.","volume":"103","author":"G\u00fcl","year":"2024","journal-title":"Medicine (Baltimore)"},{"key":"R7-20260108","doi-asserted-by":"crossref","first-page":"S36","DOI":"10.1016\/j.metabol.2017.01.011","article-title":"Artificial intelligence in medicine.","volume":"69S","author":"Hamet","year":"2017","journal-title":"Metab Clin Exp"},{"key":"R8-20260108","doi-asserted-by":"crossref","first-page":"pkad010","DOI":"10.1093\/jncics\/pkad010","article-title":"Artificial intelligence chatbots will revolutionize how cancer patients access information: ChatGPT represents a paradigm-shift.","volume":"7","author":"Hopkins","year":"2023","journal-title":"JNCI Cancer Spectr"},{"key":"R9-20260108","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1186\/s43019-024-00218-5","article-title":"Evaluating the accuracy and relevance of ChatGPT responses to frequently asked questions regarding total knee replacement.","volume":"36","author":"Zhang","year":"2024","journal-title":"Knee Surg Relat Res"},{"key":"R10-20260108","doi-asserted-by":"crossref","first-page":"307","DOI":"10.2967\/jnmt.123.266151","article-title":"ChatGPT and patient information in nuclear medicine: GPT-3.5 versus GPT-4.","volume":"51","author":"Currie","year":"2023","journal-title":"J Nucl Med Technol"},{"key":"R12-20260108","doi-asserted-by":"crossref","first-page":"1412","DOI":"10.1038\/s41433-024-02958-w","article-title":"Google DeepMind\u2019s gemini AI versus ChatGPT: a comparative analysis in ophthalmology.","volume":"38","author":"Masalkhi","year":"2024","journal-title":"Eye (Lond)"},{"key":"R13-20260108","first-page":"91","article-title":"A comparative analysis of generative artificial intelligence tools for natural language processing.","volume":"2","author":"Iorliam","year":"2024","journal-title":"J Comp Theor Appl"},{"key":"R14-20260108","doi-asserted-by":"crossref","first-page":"e15412","DOI":"10.7717\/peerj.15412","article-title":"Evaluation of the reliability and quality of YouTube videos as a source of information for transcutaneous electrical nerve stimulation.","volume":"11","author":"Erkin","year":"2023","journal-title":"PeerJ"},{"key":"R15-20260108","doi-asserted-by":"crossref","first-page":"16","DOI":"10.4103\/injr.injr_56_22","article-title":"Evaluating the readability, quality, and reliability of online information on Sjogren\u2019s syndrome.","volume":"18","author":"Ozduran","year":"2023","journal-title":"Indian J Rheumatol"},{"key":"R16-20260108","doi-asserted-by":"crossref","first-page":"49","DOI":"10.4081\/reumatismo.2022.1495","article-title":"Evaluating the readability, quality and reliability of online information on Beh\u00e7et\u2019s disease.","volume":"74","author":"\u00d6zduran","year":"2022","journal-title":"Reumatismo"},{"key":"R17-20260108","doi-asserted-by":"crossref","first-page":"e13686","DOI":"10.7717\/peerj.13686","article-title":"Evaluating the readability, quality and reliability of online patient education materials on post-covid pain.","volume":"10","author":"Ozduran","year":"2022","journal-title":"PeerJ"},{"key":"R18-20260108","first-page":"e35179","article-title":"Artificial Hallucinations in ChatGPT: Implications in Scientific Writing.","volume":"15","author":"Alkaissi","year":"2023","journal-title":"Cureus"},{"key":"R19-20260108","doi-asserted-by":"crossref","first-page":"1190","DOI":"10.1007\/s00167-023-07372-5","article-title":"A deeper dive into ChatGPT: history, use and future perspectives for orthopaedic research.","volume":"31","author":"Ollivier","year":"2023","journal-title":"Knee Surg Sports Traumatol Arthrosc"},{"key":"R20-20260108","doi-asserted-by":"crossref","first-page":"1437","DOI":"10.1001\/jamaoncol.2023.2947","article-title":"Assessment of artificial intelligence Chatbot responses to top searched queries about cancer.","volume":"9","author":"Pan","year":"2023","journal-title":"JAMA Oncol"},{"key":"R21-20260108","first-page":"1","article-title":"Accuracy, readability, and understandability of large language models for prostate cancer information to the public.","author":"Hershenhouse","year":"2024","journal-title":"Prostate Cancer Prostatic Dis"},{"key":"R22-20260108","doi-asserted-by":"crossref","first-page":"e38352","DOI":"10.1097\/MD.0000000000038352","article-title":"Assessing the readability, reliability, and quality of artificial intelligence chatbot responses to the 100 most searched queries about cardiopulmonary resuscitation: An observational study.","volume":"103","author":"\u00d6m\u00fcr Ar\u00e7a","year":"2024","journal-title":"Medicine (Baltimore)"},{"key":"R23-20260108","doi-asserted-by":"crossref","first-page":"e17215","DOI":"10.7717\/peerj.17215","article-title":"Assessing the quality and reliability of YouTube videos as a source of information on inflammatory back pain.","volume":"12","author":"Kara","year":"2024","journal-title":"PeerJ"},{"key":"R24-20260108","doi-asserted-by":"crossref","first-page":"907","DOI":"10.54029\/2023kif","article-title":"Youtube as a source of information about stroke rehabilitation during the COVID-19 pandemic.","volume":"28","author":"Ozduran","year":"2023","journal-title":"Neurol Asia"},{"key":"R25-20260108","doi-asserted-by":"crossref","first-page":"e18847","DOI":"10.7717\/peerj.18847","article-title":"Assessing the readability, quality and reliability of responses produced by ChatGPT, Gemini, and Perplexity regarding most frequently asked keywords about low back pain.","volume":"13","author":"Ozduran","year":"2025","journal-title":"PeerJ"},{"key":"R26-20260108","doi-asserted-by":"crossref","first-page":"e33529","DOI":"10.1097\/MD.0000000000033529","article-title":"Evaluating the readability, quality and reliability of online patient education materials on transcutaneuous electrical nerve stimulation (TENS).","volume":"102","author":"Erkin","year":"2023","journal-title":"Medicine (Baltimore)"},{"key":"R27-20260108","doi-asserted-by":"crossref","first-page":"858","DOI":"10.1016\/j.cjco.2022.07.004","article-title":"Evaluation of online written medication educational resources for people living with heart failure.","volume":"4","author":"Ladhar","year":"2022","journal-title":"CJC Open"},{"key":"R28-20260108","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1111\/j.1369-7625.2004.00273.x","article-title":"Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information.","volume":"7","author":"Moult","year":"2004","journal-title":"Health Expect"},{"key":"R29-20260108","doi-asserted-by":"crossref","first-page":"1728","DOI":"10.1016\/j.jvs.2022.07.022","article-title":"Readability assessment of online peripheral artery disease education materials.","volume":"76","author":"Avra","year":"2022","journal-title":"J Vasc Surg"},{"key":"R30-20260108","doi-asserted-by":"crossref","first-page":"e1078","DOI":"10.1097\/PR9.0000000000001078","article-title":"Online information on chronic pain in 3 countries: an assessment of readability, credibility, and accuracy.","volume":"8","author":"Basnet","year":"2023","journal-title":"Pain Rep"},{"key":"R31-20260108","first-page":"23333928241234863","article-title":"Artificial intelligence and decision-making in healthcare: a thematic analysis of a systematic review of reviews.","volume":"11","author":"Khosravi","year":"2024","journal-title":"Health Serv Res Manag Epidemiol"},{"key":"R32-20260108","first-page":"15589447241247332","article-title":"Evaluation of patient education materials from large-language artificial intelligence models on carpal tunnel release.","author":"Croen","year":"2024","journal-title":"Hand"},{"key":"R33-20260108","doi-asserted-by":"crossref","first-page":"862","DOI":"10.1016\/j.oret.2023.05.022","article-title":"Appropriateness and readability of ChatGPT-4-generated responses for surgical treatment of retinal diseases.","volume":"7","author":"Momenaei","year":"2023","journal-title":"Ophthalmol Retina"},{"key":"R34-20260108","doi-asserted-by":"crossref","first-page":"e381","DOI":"10.1016\/j.jpainsymman.2024.01.008","article-title":"Chatbot performance in defining and differentiating palliative care, supportive care, hospice care.","volume":"67","author":"Kim","year":"2024","journal-title":"J Pain Symptom Manage"},{"key":"R35-20260108","doi-asserted-by":"crossref","first-page":"701","DOI":"10.2214\/AJR.23.29622","article-title":"Use of ChatGPT, GPT-4, and bard to improve readability of ChatGPT\u2019s answers to common questions about lung cancer and lung cancer screening.","volume":"221","author":"Haver","year":"2023","journal-title":"AJR Am J Roentgenol"},{"key":"R36-20260108","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1007\/s10916-024-02056-0","article-title":"Responses of five different artificial intelligence Chatbots to the top searched queries about erectile dysfunction: a comparative analysis.","volume":"48","author":"\u015eahin","year":"2024","journal-title":"J Med Syst"},{"key":"R37-20260108","first-page":"e59898","article-title":"Unlocking health literacy: the ultimate guide to hypertension education from ChatGPT versus Google Gemini.","volume":"16","author":"Lee","year":"2024","journal-title":"Cureus"},{"key":"R38-20260108","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/j.eururo.2023.07.004","article-title":"How well do artificial intelligence Chatbots respond to the top search queries about urological malignancies?","volume":"85","author":"Musheyev","year":"2024","journal-title":"Eur Urol"},{"key":"R39-20260108","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1038\/s41391-023-00705-y","article-title":"Quality of information and appropriateness of ChatGPT outputs for urology patients.","volume":"27","author":"Cocci","year":"2024","journal-title":"Prostate Cancer Prostatic Dis"},{"key":"R40-20260108","doi-asserted-by":"crossref","first-page":"763","DOI":"10.1016\/S0002-9343(02)01473-0","article-title":"Infodemiology: The epidemiology of (mis)information.","volume":"113","author":"Eysenbach","year":"2002","journal-title":"Am J Med"},{"key":"R41-20260108","first-page":"1","article-title":"Drawbacks of artificial intelligence and their potential solutions in the healthcare sector.","author":"Khan","year":"2023","journal-title":"Biomed Mater Devices"}],"container-title":["Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.ovid.com\/10.1097\/MD.0000000000041780","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,6]],"date-time":"2026-08-06T20:30:18Z","timestamp":1786048218000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ovid.com\/10.1097\/MD.0000000000041780"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,14]]},"references-count":40,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2025]]}},"URL":"https:\/\/doi.org\/10.1097\/md.0000000000041780","relation":{},"ISSN":["1536-5964"],"issn-type":[{"value":"1536-5964","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,14]]}}}