{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T14:50:14Z","timestamp":1774968614420,"version":"3.50.1"},"reference-count":39,"publisher":"Oxford University Press (OUP)","issue":"10","license":[{"start":{"date-parts":[[2023,7,19]],"date-time":"2023-07-19T00:00:00Z","timestamp":1689724800000},"content-version":"vor","delay-in-days":1,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Institute of Health"},{"DOI":"10.13039\/100000057","name":"NIGMS","doi-asserted-by":"publisher","award":["R00GM135488"],"award-info":[{"award-number":["R00GM135488"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100014041","name":"Alzheimer\u2019s Disease Neuroimaging Initiative","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100014041","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["U01 AG024904"],"award-info":[{"award-number":["U01 AG024904"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000005","name":"Department of Defense","doi-asserted-by":"publisher","award":["W81XWH-12-2-0012"],"award-info":[{"award-number":["W81XWH-12-2-0012"]}],"id":[{"id":"10.13039\/100000005","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000049","name":"National Institute on Aging","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000049","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000070","name":"National Institute of Biomedical Imaging and Bioengineering","doi-asserted-by":"publisher","award":["AbbVie"],"award-info":[{"award-number":["AbbVie"]}],"id":[{"id":"10.13039\/100000070","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000957","name":"Alzheimer\u2019s Association","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100000957","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Alzheimer\u2019s Drug Discovery Foundation"},{"name":"Araclon Biotech"},{"DOI":"10.13039\/100007742","name":"BioClinica, Inc.","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007742","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100002491","name":"Bristol-Myers Squibb Company","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100002491","id-type":"DOI","asserted-by":"publisher"}]},{"name":"CereSpir, Inc."},{"name":"Elan Pharmaceuticals, Inc."},{"DOI":"10.13039\/100004312","name":"Eli Lilly and Company","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100004312","id-type":"DOI","asserted-by":"publisher"}]},{"name":"EuroImmun; F. Hoffmann-La Roche Ltd"},{"name":"Janssen Alzheimer Immunotherapy Research & Development, LLC"},{"DOI":"10.13039\/501100000024","name":"The Canadian Institutes of Health Research","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100000024","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,9,25]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Alzheimer\u2019s disease (AD) is a progressive neurological disorder with no specific curative medications. Sophisticated clinical skills are crucial to optimize treatment regimens given the multiple coexisting comorbidities in the patient population.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Here, we propose a study to leverage reinforcement learning (RL) to learn the clinicians\u2019 decisions for AD patients based on the longitude data from electronic health records.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>In this study, we selected 1736 patients from the Alzheimer\u2019s Disease Neuroimaging Initiative (ADNI) database. We focused on the two most frequent concomitant diseases\u2014depression, and hypertension, thus creating 5 data cohorts (ie, Whole Data, AD, AD-Hypertension, AD-Depression, and AD-Depression-Hypertension). We modeled the treatment learning into an RL problem by defining states, actions, and rewards. We built a regression model and decision tree to generate multiple states, used six combinations of medications (ie, cholinesterase inhibitors, memantine, memantine-cholinesterase inhibitors, hypertension drugs, supplements, or no drugs) as actions, and Mini-Mental State Exam (MMSE) scores as rewards.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Given the proper dataset, the RL model can generate an optimal policy (regimen plan) that outperforms the clinician\u2019s treatment regimen. Optimal policies (ie, policy iteration and Q-learning) had lower rewards than the clinician\u2019s policy (mean \u22123.03 and \u22122.93 vs. \u22122.93, respectively) for smaller datasets but had higher rewards for larger datasets (mean \u22124.68 and \u22122.82 vs. \u22124.57, respectively).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Our results highlight the potential of using RL to generate the optimal treatment based on the patients\u2019 longitude records. Our work can lead the path towards developing RL-based decision support systems that could help manage AD with comorbidities.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocad135","type":"journal-article","created":{"date-parts":[[2023,7,12]],"date-time":"2023-07-12T15:58:29Z","timestamp":1689177509000},"page":"1645-1656","source":"Crossref","is-referenced-by-count":10,"title":["Using artificial intelligence to learn optimal regimen plan for Alzheimer\u2019s disease"],"prefix":"10.1093","volume":"30","author":[{"given":"Kritib","family":"Bhattarai","sequence":"first","affiliation":[{"name":"Luther College , Decorah, Iowa, USA"}]},{"given":"Sivaraman","family":"Rajaganapathy","sequence":"additional","affiliation":[{"name":"Mayo Clinic , Rochester, Minnesota, USA"}]},{"given":"Trisha","family":"Das","sequence":"additional","affiliation":[{"name":"University of Illinois Urbana-Champaign , Champaign, Illinois, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7815-6310","authenticated-orcid":false,"given":"Yejin","family":"Kim","sequence":"additional","affiliation":[{"name":"University of Texas Health Science Center , Houston, Texas, USA"}]},{"given":"Yongbin","family":"Chen","sequence":"additional","affiliation":[{"name":"Mayo Clinic , Rochester, Minnesota, USA"}]},{"name":"The Alzheimer\u2019s Disease Neuroimaging Initiative","sequence":"additional","affiliation":[]},{"name":"The Australian Imaging Biomarkers and Lifestyle Flagship Study of Ageing","sequence":"additional","affiliation":[]},{"given":"Qiying","family":"Dai","sequence":"additional","affiliation":[{"name":"Mayo Clinic , Rochester, Minnesota, USA"}]},{"given":"Xiaoyang","family":"Li","sequence":"additional","affiliation":[{"name":"Mayo Clinic , Rochester, Minnesota, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9933-2205","authenticated-orcid":false,"given":"Xiaoqian","family":"Jiang","sequence":"additional","affiliation":[{"name":"University of Texas Health Science Center , Houston, Texas, USA"}]},{"given":"Nansu","family":"Zong","sequence":"additional","affiliation":[{"name":"Mayo Clinic , Rochester, Minnesota, USA"}]}],"member":"286","published-online":{"date-parts":[[2023,7,18]]},"reference":[{"key":"2023092719593798100_ocad135-B1"},{"key":"2023092719593798100_ocad135-B2","doi-asserted-by":"crossref","first-page":"117996","DOI":"10.1016\/j.lfs.2020.117996","article-title":"Alzheimer\u2019s disease; a review of the pathophysiological basis and therapeutic interventions","volume":"256","author":"Abeysinghe","year":"2020","journal-title":"Life Sci"},{"key":"2023092719593798100_ocad135-B3","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.neuroscience.2016.03.051","article-title":"Analysis of genetics and risk factors of Alzheimer\u2019s disease","volume":"325","author":"Panpalli Ates","year":"2016","journal-title":"Neuroscience"},{"key":"2023092719593798100_ocad135-B4","doi-asserted-by":"crossref","first-page":"146","DOI":"10.3389\/fnagi.2019.00146","article-title":"Modifiable risk factors for Alzheimer\u2019s disease","volume":"11","author":"Edwards Iii","year":"2019","journal-title":"Front Aging Neurosci"},{"key":"2023092719593798100_ocad135-B5","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1186\/s13195-016-0207-9","article-title":"Drug development in Alzheimer\u2019s disease: the path to 2025","volume":"8","author":"Cummings","year":"2016","journal-title":"Alzheimers Res Ther"},{"key":"2023092719593798100_ocad135-B6","doi-asserted-by":"crossref","first-page":"173554","DOI":"10.1016\/j.ejphar.2020.173554","article-title":"Alzheimer\u2019s disease: recent treatment strategies","volume":"887","author":"Vaz","year":"2020","journal-title":"Eur J Pharmacol"},{"issue":"3","key":"2023092719593798100_ocad135-B7","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1097\/WAD.0b013e31816653bc","article-title":"Long-term course and effectiveness of combination therapy in Alzheimer disease","volume":"22","author":"Atri","year":"2008","journal-title":"Alzheimer Dis Assoc Disord"},{"issue":"S165","key":"2023092719593798100_ocad135-B8","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1111\/j.1600-0404.1996.tb05870.x","article-title":"Epidemiology of Alzheimer\u2019s disease and current possibilities for prevention","volume":"94","author":"Fratiglioni","year":"1996","journal-title":"Acta Neurol Scand"},{"issue":"3","key":"2023092719593798100_ocad135-B9","first-page":"460","article-title":"Personalized management and treatment of Alzheimer\u2019s disease","volume":"12","author":"Cacabelos","year":"2022","journal-title":"Life Basel Switz"},{"issue":"3","key":"2023092719593798100_ocad135-B10","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1016\/S0006-3223(02)01348-3","article-title":"Depression in Alzheimer's disease: overview and treatment","volume":"52","author":"Lyketsos","year":"2002","journal-title":"Biol Psychiatry"},{"issue":"1","key":"2023092719593798100_ocad135-B11","doi-asserted-by":"crossref","first-page":"171","DOI":"10.3233\/ADR-210294","article-title":"The relationship between anxiety and Alzheimer\u2019s disease","volume":"5","author":"Mendez","year":"2021","journal-title":"J Alzheimers Dis Rep"},{"issue":"2","key":"2023092719593798100_ocad135-B12","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1007\/BF03327433","article-title":"Pharmacological treatment of Alzheimer\u2019s disease","volume":"18","author":"Bianchetti","year":"2006","journal-title":"Aging Clin Exp Res"},{"issue":"1","key":"2023092719593798100_ocad135-B13","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1038\/s41746-020-00341-z","article-title":"Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines","volume":"3","author":"Huang","year":"2020","journal-title":"NPJ Digit Med"},{"key":"2023092719593798100_ocad135-B14","doi-asserted-by":"crossref","first-page":"102332","DOI":"10.1016\/j.artmed.2022.102332","article-title":"Deep learning for Alzheimer\u2019s disease diagnosis: a survey","volume":"130","author":"Khojaste-Sarakhsi","year":"2022","journal-title":"Artif Intell Med"},{"issue":"1","key":"2023092719593798100_ocad135-B15","doi-asserted-by":"crossref","first-page":"1952","DOI":"10.1038\/s41598-018-37769-z","article-title":"Predicting Alzheimer\u2019s disease progression using multi-modal deep learning approach","volume":"9","author":"Lee","year":"2019","journal-title":"Sci Rep"},{"key":"2023092719593798100_ocad135-B16","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1016\/j.neuroimage.2019.01.031","article-title":"A parameter-efficient deep learning approach to predict conversion from mild cognitive impairment to Alzheimer\u2019s disease","volume":"189","author":"Spasov","year":"2019","journal-title":"NeuroImage"},{"key":"2023092719593798100_ocad135-B17","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1016\/j.neuroscience.2022.03.026","article-title":"A single model deep learning approach for Alzheimer\u2019s disease diagnosis","volume":"491","author":"Zhang","year":"2022","journal-title":"Neuroscience"},{"key":"2023092719593798100_ocad135-B18","first-page":"20903","author":"Saboo","year":"2021"},{"issue":"6","key":"2023092719593798100_ocad135-B19","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1042\/ETLS20210249","article-title":"Applied machine learning in Alzheimer\u2019s disease research: omics, imaging, and clinical data","volume":"5","author":"Li","year":"2021","journal-title":"Emerg Top Life Sci"},{"key":"2023092719593798100_ocad135-B20","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1109\/Confluence51648.2021.9377188","author":"Garg","year":"2021"},{"issue":"1","key":"2023092719593798100_ocad135-B21","doi-asserted-by":"crossref","first-page":"9313","DOI":"10.1038\/s41598-021-88619-4","article-title":"Computational medication regimen for Parkinson\u2019s disease using reinforcement learning","volume":"11","author":"Kim","year":"2021","journal-title":"Sci Rep"},{"issue":"11","key":"2023092719593798100_ocad135-B22","doi-asserted-by":"crossref","first-page":"1716","DOI":"10.1038\/s41591-018-0213-5","article-title":"The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care","volume":"24","author":"Komorowski","year":"2018","journal-title":"Nat Med"},{"key":"2023092719593798100_ocad135-B23","author":"ADNI"},{"issue":"4","key":"2023092719593798100_ocad135-B24","first-page":"672","article-title":"The Australian Imaging, Biomarkers and Lifestyle (AIBL) study of aging: methodology and baseline characteristics of 1112 individuals recruited for a longitudinal study of Alzheimer's disease","volume":"21","author":"Ellis","year":"2009"},{"issue":"10","key":"2023092719593798100_ocad135-B25","doi-asserted-by":"crossref","first-page":"896","DOI":"10.1212\/WNL.0b013e3182a35228","article-title":"Antihypertensive drugs decrease risk of Alzheimer disease: Ginkgo evaluation of memory study","volume":"81","author":"Yasar","year":"2013","journal-title":"Neurology"},{"issue":"1","key":"2023092719593798100_ocad135-B26","doi-asserted-by":"crossref","first-page":"e12060","DOI":"10.1002\/trc2.12060","article-title":"Antihypertensive medications ameliorate Alzheimer\u2019s disease pathology by slowing its propagation","volume":"6","author":"Affleck","year":"2020","journal-title":"Alzheimers Dement (N Y)"},{"issue":"3","key":"2023092719593798100_ocad135-B27","first-page":"CD010783","article-title":"Mini-Mental State Examination (MMSE) for the detection of Alzheimer\u2019s disease and other dementias in people with mild cognitive impairment (MCI)","volume":"2015","author":"Arevalo-Rodriguez","year":"2015","journal-title":"Cochrane Database Syst Rev"},{"issue":"3","key":"2023092719593798100_ocad135-B28","doi-asserted-by":"crossref","first-page":"1065","DOI":"10.3233\/JAD-150852","article-title":"Describing the sequence of cognitive decline in Alzheimer\u2019s disease patients: results from an observational study","volume":"52","author":"Henneges","year":"2016","journal-title":"J Alzheimers Dis"},{"key":"2023092719593798100_ocad135-B29"},{"key":"2023092719593798100_ocad135-B30","author":"Sutton"},{"key":"2023092719593798100_ocad135-B31","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1145\/3460231.3474245","author":"Saito","year":"2021"},{"key":"2023092719593798100_ocad135-B32","first-page":"2","article-title":"Model selection for offline reinforcement learning: practical considerations for healthcare settings","volume":"149","author":"Tang","year":"2021","journal-title":"Proc Mach Learn Res"},{"key":"2023092719593798100_ocad135-B33","doi-asserted-by":"crossref","DOI":"10.1002\/9781118631980","volume-title":"Simulation and the Monte Carlo Method","author":"Rubinstein","year":"2016."},{"key":"2023092719593798100_ocad135-B34","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1007\/978-3-7091-6303-0_4","volume-title":"Rendering Techniques 2000","author":"Bekaert","year":"2000"},{"issue":"1","key":"2023092719593798100_ocad135-B35","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1038\/s41398-020-0839-1","article-title":"Depression\u2014an underrecognized target for prevention of dementia in Alzheimer\u2019s disease","volume":"10","author":"Dafsari","year":"2020","journal-title":"Transl Psychiatry"},{"issue":"2","key":"2023092719593798100_ocad135-B36","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1007\/s40120-019-00148-5","article-title":"Diagnosing and treating depression in patients with Alzheimer\u2019s disease","volume":"8","author":"Burke","year":"2019","journal-title":"Neurol Ther"},{"issue":"3","key":"2023092719593798100_ocad135-B37","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1016\/S0006-3223(02)01348-3","article-title":"Depression in Alzheimer\u2019s disease: overview and treatment","volume":"52","author":"Lyketsos","year":"2002","journal-title":"Biol Psychiatry"},{"issue":"7","key":"2023092719593798100_ocad135-B38","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1016\/j.biopsych.2013.11.020","article-title":"A focus on structural brain imaging in the Alzheimer\u2019s Disease Neuroimaging Initiative","volume":"75","author":"Braskie","year":"2014","journal-title":"Biol Psychiatry"},{"key":"2023092719593798100_ocad135-B39","first-page":"1179","author":"Kumar"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/30\/10\/1645\/51770138\/ocad135.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/30\/10\/1645\/51770138\/ocad135.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T16:27:23Z","timestamp":1695832043000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/30\/10\/1645\/7226148"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,18]]},"references-count":39,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2023,7,18]]},"published-print":{"date-parts":[[2023,9,25]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocad135","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2023.01.26.23285064","asserted-by":"object"}]},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,10,1]]},"published":{"date-parts":[[2023,7,18]]}}}