{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,28]],"date-time":"2026-10-28T02:49:47Z","timestamp":1793155787425,"version":"4.5.0"},"reference-count":21,"publisher":"American Medical Association (AMA)","issue":"7","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JAMA Netw Open"],"abstract":"<jats:sec id=\"ab-zoi240713-4\"><jats:title>Importance<\/jats:title><jats:p>The mainstream use of chatbots requires a thorough investigation of their readability and quality of information.<\/jats:p><\/jats:sec><jats:sec id=\"ab-zoi240713-5\"><jats:title>Objective<\/jats:title><jats:p>To identify readability and quality differences in information between a free and paywalled chatbot cancer-related responses, and to explore if more precise prompting can mitigate any observed differences.<\/jats:p><\/jats:sec><jats:sec id=\"ab-zoi240713-6\"><jats:title>Design, Setting, and Participants<\/jats:title><jats:p>This cross-sectional study compared readability and information quality of a chatbot\u2019s free vs paywalled responses with Google Trends\u2019 top 5 search queries associated with breast, lung, prostate, colorectal, and skin cancers from January 1, 2021, to January 1, 2023. Data were extracted from the search tracker, and responses were produced by free and paywalled ChatGPT. Data were analyzed from December 20, 2023, to January 15, 2024.<\/jats:p><\/jats:sec><jats:sec id=\"ab-zoi240713-7\"><jats:title>Exposures<\/jats:title><jats:p>Free vs paywalled chatbot outputs with and without prompt: \u201cExplain the following at a sixth grade reading level: [nonprompted input].\u201d<\/jats:p><\/jats:sec><jats:sec id=\"ab-zoi240713-8\"><jats:title>Main Outcomes and Measures<\/jats:title><jats:p>The primary outcome measured the readability of a chatbot\u2019s responses using Flesch Reading Ease scores (0 [graduate reading level] to 100 [easy fifth grade reading level]). Secondary outcomes included assessing consumer health information quality with the validated DISCERN instrument (overall score from 1 [low quality] to 5 [high quality]) for each response. Scores were compared between the 2 chatbot models with and without prompting.<\/jats:p><\/jats:sec><jats:sec id=\"ab-zoi240713-9\"><jats:title>Results<\/jats:title><jats:p>This study evaluated 100 chatbot responses. Nonprompted free chatbot responses had lower readability (median [IQR] Flesh Reading ease scores, 52.60 [44.54-61.46]) than nonprompted paywalled chatbot responses (62.48 [54.83-68.40]) (<jats:italic>P<\/jats:italic>\u2009&amp;amp;lt;\u2009.05). However, prompting the free chatbot to reword responses at a sixth grade reading level was associated with increased reading ease scores than the paywalled chatbot nonprompted responses (median [IQR], 71.55 [68.20-78.99]) (<jats:italic>P<\/jats:italic>\u2009&amp;amp;lt;\u2009.001). Prompting was associated with increases in reading ease in both free (median [IQR], 71.55 [68.20-78.99]; <jats:italic>P<\/jats:italic>\u2009&amp;amp;lt;\u2009.001)and paywalled versions (median [IQR], 75.64 [70.53-81.12]; <jats:italic>P<\/jats:italic>\u2009&amp;amp;lt;\u2009.001). There was no significant difference in overall DISCERN scores between the chatbot models, with and without prompting.<\/jats:p><\/jats:sec><jats:sec id=\"ab-zoi240713-10\"><jats:title>Conclusions and Relevance<\/jats:title><jats:p>In this cross-sectional study, paying for the chatbot was found to provide easier-to-read responses, but prompting the free version of the chatbot was associated with increased response readability without changing information quality. Educating the public on how to prompt chatbots may help promote equitable access to health information.<\/jats:p><\/jats:sec>","DOI":"10.1001\/jamanetworkopen.2024.22275","type":"journal-article","created":{"date-parts":[[2024,7,26]],"date-time":"2024-07-26T15:33:20Z","timestamp":1722008000000},"page":"e2422275","source":"Crossref","is-referenced-by-count":45,"title":["Readability and Information Quality in Cancer Information From a Free vs Paid Chatbot"],"prefix":"10.1001","volume":"7","author":[{"given":"David","family":"Musheyev","sequence":"first","affiliation":[{"name":"Department of Urology, State University of New York Downstate Health Sciences University, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexander","family":"Pan","sequence":"additional","affiliation":[{"name":"Department of Urology, State University of New York Downstate Health Sciences University, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Preston","family":"Gross","sequence":"additional","affiliation":[{"name":"Department of Urology, State University of New York Downstate Health Sciences University, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Kamyab","sequence":"additional","affiliation":[{"name":"Department of Urology, State University of New York Downstate Health Sciences University, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter","family":"Kaplinsky","sequence":"additional","affiliation":[{"name":"Department of Urology, State University of New York Downstate Health Sciences University, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mark","family":"Spivak","sequence":"additional","affiliation":[{"name":"Department of Urology, State University of New York Downstate Health Sciences University, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marie A.","family":"Bragg","sequence":"additional","affiliation":[{"name":"Department of Urology, New York University and Manhattan Veterans Affairs, New York"},{"name":"Marketing Department, Stern School of Business, New York University, New York"},{"name":"Department of Population Health, New York University, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stacy","family":"Loeb","sequence":"additional","affiliation":[{"name":"Department of Urology, New York University and Manhattan Veterans Affairs, New York"},{"name":"Department of Population Health, New York University, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abdo E.","family":"Kabarriti","sequence":"additional","affiliation":[{"name":"Department of Urology, State University of New York Downstate Health Sciences University, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10","published-online":{"date-parts":[[2024,7,26]]},"reference":[{"issue":"12","key":"zoi240713r1","doi-asserted-by":"publisher","first-page":"4090","DOI":"10.1007\/s00261-020-02657-6","article-title":"Health disparities and inequities in the utilization of diagnostic imaging for prostate cancer.","volume":"45","author":"Washington","year":"2020","journal-title":"Abdom Radiol (NY)"},{"issue":"1","key":"zoi240713r2","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1016\/j.ogc.2021.11.010","article-title":"Disparities in breast cancer.","volume":"49","author":"Grabinski","year":"2022","journal-title":"Obstet Gynecol Clin North Am"},{"issue":"1","key":"zoi240713r3","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1200\/JCO.2017.74.2049","article-title":"Racial and ethnic disparities in cancer survival: the contribution of tumor, sociodemographic, institutional, and neighborhood characteristics.","volume":"36","author":"Ellis","year":"2018","journal-title":"J Clin Oncol"},{"issue":"5","key":"zoi240713r4","doi-asserted-by":"publisher","first-page":"e1","DOI":"10.1093\/heapro\/day062","article-title":"Does health literacy mediate the relationship between socioeconomic status and health disparities: integrative review.","volume":"34","author":"Stormacq","year":"2019","journal-title":"Health Promot Int"},{"key":"zoi240713r5","doi-asserted-by":"publisher","DOI":"10.1016\/j.ypmed.2021.106927","article-title":"The association between adherence to cancer screening programs and health literacy: a systematic review and meta-analysis.","volume":"155","author":"Baccolini","year":"2022","journal-title":"Prev Med"},{"issue":"8","key":"zoi240713r6","doi-asserted-by":"publisher","first-page":"682.e11","DOI":"10.1016\/j.urolonc.2020.04.007","article-title":"Examining the association of health literacy and numeracy with prostate-related knowledge and prostate cancer treatment regret.","volume":"38","author":"Joyce","year":"2020","journal-title":"Urol Oncol"},{"key":"zoi240713r8","doi-asserted-by":"publisher","first-page":"1591","DOI":"10.2147\/IJGM.S408208","article-title":"The use of chatbots in oncological care: a narrative review.","volume":"16","author":"Wang","year":"2023","journal-title":"Int J Gen Med"},{"issue":"5","key":"zoi240713r9","doi-asserted-by":"publisher","first-page":"1665","DOI":"10.1177\/1557988318780864","article-title":"Readability of prostate cancer information online: a cross-sectional study.","volume":"12","author":"Basch","year":"2018","journal-title":"Am J Mens Health"},{"issue":"10","key":"zoi240713r10","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2023.38050","article-title":"Assessing biases in medical decisions via clinician and AI chatbot responses to patient vignettes.","volume":"6","author":"Kim","year":"2023","journal-title":"JAMA Netw Open"},{"issue":"10","key":"zoi240713r11","doi-asserted-by":"publisher","first-page":"1437","DOI":"10.1001\/jamaoncol.2023.2947","article-title":"Assessment of artificial intelligence chatbot responses to top searched queries about cancer.","volume":"9","author":"Pan","year":"2023","journal-title":"JAMA Oncol"},{"issue":"3","key":"zoi240713r12","doi-asserted-by":"publisher","first-page":"721","DOI":"10.3350\/cmh.2023.0089","article-title":"Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma.","volume":"29","author":"Yeo","year":"2023","journal-title":"Clin Mol Hepatol"},{"key":"zoi240713r13","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1186\/s41205-023-00186-8","article-title":"Generative AI for medical 3D printing: a comparison of ChatGPT outputs to reference standard education.","volume":"9","author":"Sriwastwa","year":"2023","journal-title":"3D Print Med"},{"issue":"4","key":"zoi240713r14","doi-asserted-by":"publisher","first-page":"344","DOI":"10.1016\/j.jclinepi.2007.11.008","article-title":"The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.","volume":"61","author":"von Elm","year":"2008","journal-title":"J Clin Epidemiol"},{"issue":"3","key":"zoi240713r15","doi-asserted-by":"publisher","first-page":"233","DOI":"10.3322\/caac.v73.3","article-title":"Colorectal cancer statistics, 2023.","volume":"73","author":"Siegel","year":"2023","journal-title":"CA Cancer J Clin"},{"issue":"2","key":"zoi240713r16","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1136\/jech.53.2.105","article-title":"DISCERN: an instrument for judging the quality of written consumer health information on treatment choices.","volume":"53","author":"Charnock","year":"1999","journal-title":"J Epidemiol Community Health"},{"issue":"1","key":"zoi240713r17","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1080\/13548509608400003","article-title":"The use of readability formulas in health care.","volume":"1","author":"Ley","year":"1996","journal-title":"Psychol Health Med"},{"issue":"10","key":"zoi240713r19","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.46662","article-title":"How efficient is ChatGPT in accessing accurate and quality health-related information","volume":"15","author":"Ulusoy","year":"2023","journal-title":"Cureus"},{"issue":"4","key":"zoi240713r20","doi-asserted-by":"publisher","first-page":"573","DOI":"10.1007\/s11606-023-08469-w","article-title":"New frontiers in health literacy: using ChatGPT to simplify health information for people in the community.","volume":"39","author":"Ayre","year":"2024","journal-title":"J Gen Intern Med"},{"issue":"2","key":"zoi240713r21","doi-asserted-by":"publisher","DOI":"10.1093\/jncics\/pkad015","article-title":"Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information.","volume":"7","author":"Johnson","year":"2023","journal-title":"J Natl Cancer Inst Cancer Spectr"},{"key":"zoi240713r7","unstructured":"Weiss? BD. Health literacy: a manual for clinicians. American Medical Association Foundation. Accessed June 27, 2024. http:\/\/lib.ncfh.org\/pdfs\/6617.pdf"},{"key":"zoi240713r18","doi-asserted-by":"crossref","unstructured":"Kincaid? JP, Fishburne? RP, Rogers? RL, Chissom? BS. Derivation of new readability formulas (automated readability index, fog count, and Flesch reading ease formula) for Navy enlisted personnel: Research Branch Report 8-75. Chief of Naval Technical Training: Naval Air Station Memphis; 1975. Accessed June 13, 2024. https:\/\/stars.library.ucf.edu\/istlibrary\/56\/","DOI":"10.21236\/ADA006655"}],"container-title":["JAMA Network Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/articlepdf\/2821558\/musheyev_2024_oi_240713_1721230302.98424.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,26]],"date-time":"2024-07-26T15:33:22Z","timestamp":1722008002000},"score":1,"resource":{"primary":{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/fullarticle\/2821558"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,26]]},"references-count":21,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2024,7,1]]}},"URL":"https:\/\/doi.org\/10.1001\/jamanetworkopen.2024.22275","relation":{},"ISSN":["2574-3805"],"issn-type":[{"value":"2574-3805","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,26]]}}}