{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,9]],"date-time":"2026-11-09T21:14:32Z","timestamp":1794258872003,"version":"4.9.0"},"reference-count":10,"publisher":"American Medical Association (AMA)","issue":"10","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JAMA Oncol"],"published-print":{"date-parts":[[2023,10,1]]},"abstract":"\n Importance<\/jats:title>\n Consumers are increasingly using artificial intelligence (AI) chatbots as a source of information. However, the quality of the cancer information generated by these chatbots has not yet been evaluated using validated instruments.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n To characterize the quality of information and presence of misinformation about skin, lung, breast, colorectal, and prostate cancers generated by 4 AI chatbots.<\/jats:p>\n <\/jats:sec>\n \n Design, Setting, and Participants<\/jats:title>\n This cross-sectional study assessed AI chatbots\u2019 text responses to the 5 most commonly searched queries related to the 5 most common cancers using validated instruments. Search data were extracted from the publicly available Google Trends platform and identical prompts were used to generate responses from 4 AI chatbots: ChatGPT version 3.5 (OpenAI), Perplexity (Perplexity.AI), Chatsonic (Writesonic), and Bing AI (Microsoft).<\/jats:p>\n <\/jats:sec>\n \n Exposures<\/jats:title>\n Google Trends\u2019 top 5 search queries related to skin, lung, breast, colorectal, and prostate cancer from January 1, 2021, to January 1, 2023, were input into 4 AI chatbots.<\/jats:p>\n <\/jats:sec>\n \n Main Outcomes and Measures<\/jats:title>\n The primary outcomes were the quality of consumer health information based on the validated DISCERN instrument (scores from 1 [low] to 5 [high] for quality of information) and the understandability and actionability of this information based on the understandability and actionability domains of the Patient Education Materials Assessment Tool (PEMAT) (scores of 0%-100%, with higher scores indicating a higher level of understandability and actionability). Secondary outcomes included misinformation scored using a 5-item Likert scale (scores from 1 [no misinformation] to 5 [high misinformation]) and readability assessed using the Flesch-Kincaid Grade Level readability score.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n The analysis included 100 responses from 4 chatbots about the 5 most common search queries for skin, lung, breast, colorectal, and prostate cancer. The quality of text responses generated by the 4 AI chatbots was good (median [range] DISCERN score, 5 [2-5]) and no misinformation was identified. Understandability was moderate (median [range] PEMAT Understandability score, 66.7% [33.3%-90.1%]), and actionability was poor (median [range] PEMAT Actionability score, 20.0% [0%-40.0%]). The responses were written at the college level based on the Flesch-Kincaid Grade Level score.<\/jats:p>\n <\/jats:sec>\n \n Conclusions and Relevance<\/jats:title>\n Findings of this cross-sectional study suggest that AI chatbots generally produce accurate information for the top cancer-related search queries, but the responses are not readily actionable and are written at a college reading level. These limitations suggest that AI chatbots should be used supplementarily and not as a primary source for medical information.<\/jats:p>\n <\/jats:sec>","DOI":"10.1001\/jamaoncol.2023.2947","type":"journal-article","created":{"date-parts":[[2023,8,24]],"date-time":"2023-08-24T11:32:22Z","timestamp":1692876742000},"page":"1437","source":"Crossref","is-referenced-by-count":299,"title":["Assessment of Artificial Intelligence Chatbot Responses to Top Searched Queries About Cancer"],"prefix":"10.1001","volume":"9","author":[{"given":"Alexander","family":"Pan","sequence":"first","affiliation":[{"name":"Department of Urology, State University of New York Downstate Health Sciences University, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Musheyev","sequence":"additional","affiliation":[{"name":"Department of Urology, State University of New York Downstate Health Sciences University, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Bockelman","sequence":"additional","affiliation":[{"name":"Department of Urology, State University of New York Downstate Health Sciences University, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stacy","family":"Loeb","sequence":"additional","affiliation":[{"name":"Department of Urology, New York University School of Medicine, New York"},{"name":"Department of Population Health, New York University School of Medicine, New York"},{"name":"Department of Surgery, VA New York Harbor Health Care, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abdo E.","family":"Kabarriti","sequence":"additional","affiliation":[{"name":"Department of Urology, State University of New York Downstate Health Sciences University, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10","reference":[{"issue":"2","key":"cbr230013r2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models.","volume":"2","author":"Kung","year":"2023","journal-title":"PLOS Digit Health"},{"issue":"1","key":"cbr230013r3","doi-asserted-by":"publisher","first-page":"17","DOI":"10.3322\/caac.v73.1","article-title":"Cancer statistics, 2023.","volume":"73","author":"Siegel","year":"2023","journal-title":"CA Cancer J Clin"},{"issue":"2","key":"cbr230013r4","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1136\/jech.53.2.105","article-title":"DISCERN: an instrument for judging the quality of written consumer health information on treatment choices.","volume":"53","author":"Charnock","year":"1999","journal-title":"J Epidemiol Community Health"},{"issue":"3","key":"cbr230013r6","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1016\/j.pec.2014.05.027","article-title":"Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information.","volume":"96","author":"Shoemaker","year":"2014","journal-title":"Patient Educ Couns"},{"issue":"4","key":"cbr230013r7","doi-asserted-by":"publisher","first-page":"564","DOI":"10.1016\/j.eururo.2018.10.056","article-title":"Dissemination of misinformative and biased information about prostate cancer on YouTube.","volume":"75","author":"Loeb","year":"2019","journal-title":"Eur Urol"},{"key":"cbr230013r9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/13548509608400003","article-title":"The use of readability formulas in health care.","volume":"1","author":"Ley","year":"1996","journal-title":"Psychol Health Med"},{"issue":"7","key":"cbr230013r10","doi-asserted-by":"publisher","first-page":"1036","DOI":"10.1093\/jnci\/djab141","article-title":"Cancer misinformation and harmful information on Facebook and other social media: a brief report.","volume":"114","author":"Johnson","year":"2022","journal-title":"J Natl Cancer Inst"},{"key":"cbr230013r1","doi-asserted-by":"crossref","unstructured":"Johnson? D, Goodman? R, Patrinely? J, . Assessing the accuracy and reliability of AI-generated medical responses: an evaluation of the Chat-GPT model.? Res Sq. Preprint posted online February 28, 2023. doi:10.21203\/rs.3.rs-2566942\/v1?","DOI":"10.21203\/rs.3.rs-2566942\/v1"},{"key":"cbr230013r5","unstructured":"Agency for Healthcare Research and Quality. The Patient Education Materials Assessment Tool (PEMAT) and user\u2019s guide. Accessed April 4, 2023. https:\/\/www.ahrq.gov\/health-literacy\/patient-education\/pemat.html"},{"key":"cbr230013r8","unstructured":"National Comprehensive Cancer Network. National Comprehensive Cancer Network (NCCN) guidelines. Accessed April 4, 2023. http:\/\/www.nccn.org"}],"container-title":["JAMA Oncology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/jamanetwork.com\/journals\/jamaoncology\/articlepdf\/2808733\/jamaoncology_pan_2023_br_230013_1697125931.23609.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,19]],"date-time":"2023-10-19T11:08:13Z","timestamp":1697713693000},"score":1,"resource":{"primary":{"URL":"https:\/\/jamanetwork.com\/journals\/jamaoncology\/fullarticle\/2808733"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,1]]},"references-count":10,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2023,10,1]]}},"URL":"https:\/\/doi.org\/10.1001\/jamaoncol.2023.2947","relation":{},"ISSN":["2374-2437"],"issn-type":[{"value":"2374-2437","type":"print"}],"subject":[],"published":{"date-parts":[[2023,10,1]]}}}