{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T22:19:33Z","timestamp":1791238773853,"version":"4.1.0"},"reference-count":12,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Aging"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Artificial intelligence (AI) such as ChatGPT by OpenAI holds great promise to improve the quality of life of patients with dementia and their caregivers by providing high-quality responses to their questions about typical dementia behaviors. So far, however, evidence on the quality of such ChatGPT responses is limited. A few recent publications have investigated the quality of ChatGPT responses in other health conditions. Our study is the first to assess ChatGPT using real-world questions asked by dementia caregivers themselves.<\/jats:p>\n <\/jats:sec>\n \n Objectives<\/jats:title>\n This pilot study examines the potential of ChatGPT-3.5 to provide high-quality information that may enhance dementia care and patient-caregiver education.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n Our interprofessional team used a formal rating scale (scoring range: 0-5; the higher the score, the better the quality) to evaluate ChatGPT responses to real-world questions posed by dementia caregivers. We selected 60 posts by dementia caregivers from Reddit, a popular social media platform. These posts were verified by 3 interdisciplinary dementia clinicians as representing dementia caregivers\u2019 desire for information in the areas of memory loss and confusion, aggression, and driving. Word count for posts in the memory loss and confusion category ranged from 71 to 531 (mean 218; median 188), aggression posts ranged from 58 to 602 words (mean 254; median 200), and driving posts ranged from 93 to 550 words (mean 272; median 276).<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n ChatGPT\u2019s response quality scores ranged from 3 to 5. Of the 60 responses, 26 (43%) received 5 points, 21 (35%) received 4 points, and 13 (22%) received 3 points, suggesting high quality. ChatGPT obtained consistently high scores in synthesizing information to provide follow-up recommendations (n=58, 96%), with the lowest scores in the area of comprehensiveness (n=38, 63%).<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n ChatGPT provided high-quality responses to complex questions posted by dementia caregivers, but it did have limitations. ChatGPT was unable to anticipate future problems that a human professional might recognize and address in a clinical encounter. At other times, ChatGPT recommended a strategy that the caregiver had already explicitly tried. This pilot study indicates the potential of AI to provide high-quality information to enhance dementia care and patient-caregiver education in tandem with information provided by licensed health care professionals. Evaluating the quality of responses is necessary to ensure that caregivers can make informed decisions. ChatGPT has the potential to transform health care practice by shaping how caregivers receive health information.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/53019","type":"journal-article","created":{"date-parts":[[2024,3,9]],"date-time":"2024-03-09T10:02:10Z","timestamp":1709978530000},"page":"e53019-e53019","source":"Crossref","is-referenced-by-count":29,"title":["Assessing the Quality of ChatGPT Responses to Dementia Caregivers\u2019 Questions: Qualitative Analysis"],"prefix":"10.2196","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7522-6704","authenticated-orcid":false,"given":"Alyssa","family":"Aguirre","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8386-1508","authenticated-orcid":false,"given":"Robin","family":"Hilsabeck","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7512-4919","authenticated-orcid":false,"given":"Tawny","family":"Smith","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6016-6008","authenticated-orcid":false,"given":"Bo","family":"Xie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4645-8696","authenticated-orcid":false,"given":"Daqing","family":"He","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-5204-5792","authenticated-orcid":false,"given":"Zhendong","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2287-0392","authenticated-orcid":false,"given":"Ning","family":"Zou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2024,5,6]]},"reference":[{"issue":"1","key":"R1","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1093\/workar\/waac026","article-title":"Internet-enabled behaviors in older adults during the pandemic: patterns of use, psychosocial impacts, and plans for continued utilization","volume":"10","author":"Benge","journal-title":"Work Aging Retire"},{"issue":"3","key":"R2","doi-asserted-by":"publisher","first-page":"766","DOI":"10.1177\/1471301218786568","article-title":"Seeking relational information sources in the digital age: a study into information source preferences amongst family and friends of those with dementia","volume":"19","author":"Allen","journal-title":"Dementia (London)"},{"issue":"8","key":"R3","doi-asserted-by":"publisher","DOI":"10.2196\/18189","article-title":"Artificial intelligence for caregivers of persons with Alzheimer\u2019s disease and related dementias: systematic literature review","volume":"8","author":"Xie","journal-title":"JMIR Med Inform"},{"issue":"Suppl 1","key":"R4","doi-asserted-by":"publisher","first-page":"S41","DOI":"10.1093\/geront\/gnz178","article-title":"Technology and caregiving: emerging interventions and directions for research","volume":"60","author":"Lindeman","journal-title":"Gerontologist"},{"key":"R5","unstructured":"ChatGPT. OpenAI. 2023. URL: https:\/\/openai.com\/chatgpt [Accessed 21-09-2023]"},{"issue":"9","key":"R6","doi-asserted-by":"publisher","first-page":"286","DOI":"10.3390\/fi15090286","article-title":"Generative AI in medicine and healthcare: promises, opportunities and challenges","volume":"15","author":"Zhang","journal-title":"Future Internet"},{"key":"R7","doi-asserted-by":"publisher","DOI":"10.3389\/fpubh.2023.1166120","article-title":"ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health","volume":"11","author":"De Angelis","journal-title":"Front Public Health"},{"issue":"10","key":"R8","doi-asserted-by":"publisher","first-page":"842","DOI":"10.1001\/jama.2023.1044","article-title":"Appropriateness of cardiovascular disease prevention recommendations obtained from a popular online chat-based artificial intelligence model","volume":"329","author":"Sarraju","journal-title":"JAMA"},{"issue":"3","key":"R9","doi-asserted-by":"publisher","first-page":"721","DOI":"10.3350\/cmh.2023.0089","article-title":"Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma","volume":"29","author":"Yeo","journal-title":"Clin Mol Hepatol"},{"issue":"6","key":"R10","doi-asserted-by":"publisher","first-page":"1790","DOI":"10.1007\/s11695-023-06603-5","article-title":"Assessing the accuracy of responses by the language model ChatGPT to questions regarding bariatric surgery","volume":"33","author":"Samaan","journal-title":"Obes Surg"},{"issue":"4","key":"R11","doi-asserted-by":"publisher","first-page":"396","DOI":"10.1177\/0163278712446639","article-title":"Measuring clinical decision making: do key features problems measure higher level cognitive processes?","volume":"35","author":"Hurtz","journal-title":"Eval Health Prof"},{"issue":"4","key":"R12","doi-asserted-by":"publisher","first-page":"1598","DOI":"10.1002\/alz.13016","article-title":"2023 Alzheimer\u2019s disease facts and figures","volume":"19","journal-title":"Alzheimers Dement"}],"container-title":["JMIR Aging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/aging.jmir.org\/2024\/1\/e53019","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:38:17Z","timestamp":1782841097000},"score":1,"resource":{"primary":{"URL":"https:\/\/aging.jmir.org\/2024\/1\/e53019"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,6]]},"references-count":12,"URL":"https:\/\/doi.org\/10.2196\/53019","relation":{},"ISSN":["2561-7605"],"issn-type":[{"value":"2561-7605","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,6]]},"article-number":"v7i1e53019"}}