{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T06:38:59Z","timestamp":1792219139195,"version":"4.4.0"},"reference-count":48,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2026,1,20]],"date-time":"2026-01-20T00:00:00Z","timestamp":1768867200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/100000027","name":"National Institute on Alcohol Abuse and Alcoholism","doi-asserted-by":"publisher","award":["AA029831"],"award-info":[{"award-number":["AA029831"]}],"id":[{"id":"10.13039\/100000027","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["UG1DA049435, CTN-0114"],"award-info":[{"award-number":["UG1DA049435, CTN-0114"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000049","name":"National Institute on Aging","doi-asserted-by":"publisher","award":["AG057557, AG061388, AG062272, AG07664"],"award-info":[{"award-number":["AG057557, AG061388, AG062272, AG07664"]}],"id":[{"id":"10.13039\/100000049","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Alzheimer\u2019s Disease"],"published-print":{"date-parts":[[2026,3]]},"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Large language models (LLMs) are increasingly used in the biomedical field for information retrieval, information extraction and knowledge discovery. However, their potential in retrieving and discovering drug combinations for diseases remains underexplored.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aims to evaluate the effectiveness of LLMs in retrieving known drug combinations and to identify novel drug combinations for treating Alzheimer's disease (AD).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We developed a series of prompts to guide LLMs in retrieving drug combinations. Their performance was evaluated using both FDA-approved combinations and combinations identified through PubMed literature mining. We then assessed the feasibility of identifying novel drug combination candidates for AD. In collaboration with domain experts, we performed pathway enrichment analyses to evaluate their potential mechanisms of action within the context of AD.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>In a comparative evaluation of multiple LLMs, GPT-5 demonstrated the strongest overall performance, achieving an accuracy of 0.95 and a balanced F1 score of 0.95 in identifying FDA-approved drug combinations. Among the top 10 drug-combination candidates for AD treatment suggested by GPT-5, the combination of donepezil and memantine is already FDA-approved. Three other combinations have been tested in AD clinical trials, and three have supporting evidence in the literature. We also identified 10 off-label drug combinations, with pathway enrichment analyses indicating that several target key AD-related biological pathways.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>LLMs is effective in retrieving drug combinations for a given disease and the performance varies among different language models with best performance for GPT-5. However, the suggestions from LLM models require further validation to be considered reliable.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/13872877261415635","type":"journal-article","created":{"date-parts":[[2026,1,20]],"date-time":"2026-01-20T08:24:01Z","timestamp":1768897441000},"page":"619-627","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":2,"title":["Comparative evaluation of large language models in retrieving known and predicting novel drug combinations"],"prefix":"10.1177","volume":"110","author":[{"given":"Evan J.","family":"Wang","sequence":"first","affiliation":[{"name":"Center for Artificial Intelligence in Drug Discovery, School of Medicine, Case Western Reserve University, Cleveland, OH, USA"},{"name":"College of Art and Sciences, Emory University, Atlanta, GA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"\u00c7era\u011f","family":"O\u011fuzt\u00fcz\u00fcn","sequence":"additional","affiliation":[{"name":"Center for Artificial Intelligence in Drug Discovery, School of Medicine, Case Western Reserve University, Cleveland, OH, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3127-4795","authenticated-orcid":false,"given":"Rong","family":"Xu","sequence":"additional","affiliation":[{"name":"Center for Artificial Intelligence in Drug Discovery, School of Medicine, Case Western Reserve University, Cleveland, OH, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2223-5092","authenticated-orcid":false,"given":"Zhenxiang","family":"Gao","sequence":"additional","affiliation":[{"name":"Center for Artificial Intelligence in Drug Discovery, School of Medicine, Case Western Reserve University, Cleveland, OH, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2026,1,20]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1111\/hel.13115"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.lanmic.2024.07.017"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1002\/hcs2.61"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.apjo.2024.100084"},{"key":"e_1_3_3_6_2","article-title":"ChatGPT for digital pathology research","volume":"6","author":"Omar M","year":"2024","unstructured":"Omar M, Ullanat V, Loda M, et al. ChatGPT for digital pathology research. Lancet Digit Health 2024; 6: e595\u2013e600.","journal-title":"Lancet Digit Health"},{"key":"e_1_3_3_7_2","unstructured":"Scherbakov D Hubig N Jansari V et al. The emergence of large language models as a tool in literature reviews: an LLM-automated systematic review. arXiv 2024; arXiv:2409.04600."},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-45563-x"},{"key":"e_1_3_3_9_2","first-page":"875","article-title":"Contribution and performance of ChatGPT and other large language models for scientific and research advancements: a double-edged sword","volume":"5","author":"Rane NL","year":"2023","unstructured":"Rane NL, Tawde A, Choudhary SP, et al. Contribution and performance of ChatGPT and other large language models for scientific and research advancements: a double-edged sword. Int Res J Mod Eng Technol Sci 2023; 5: 875\u2013899.","journal-title":"Int Res J Mod Eng Technol Sci"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.isci.2024.109713"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0303231"},{"key":"e_1_3_3_12_2","unstructured":"Zheng Y Koh HY Yang M et al. Large language models in drug discovery and development: from disease mechanisms to clinical trials. arXiv 2024; arXiv:2409.04481."},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.3389\/frdem.2024.1385303"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10439-023-03385-w"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-022-04437-2"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41579-018-0141-x"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.3748\/wjg.v31.i9.100607"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.csbj.2022.05.055"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1038\/nrd1804"},{"key":"e_1_3_3_20_2","article-title":"DrugCombDB: a comprehensive database of drug combinations toward the discovery of combinatorial therapy","volume":"48","author":"Liu H","year":"2020","unstructured":"Liu H, Zhang W, Zou B, et al. DrugCombDB: a comprehensive database of drug combinations toward the discovery of combinatorial therapy. Nucleic Acids Res 2020; 48: D871\u2013D881.","journal-title":"Nucleic Acids Res"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btt359"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-15-105"},{"key":"e_1_3_3_23_2","first-page":"15","volume-title":"A review of evaluation metrics in machine learning algorithmsComputer Science On-line Conference","author":"Naidu G","unstructured":"Naidu G, Zuva T, Sibanda EM. A review of evaluation metrics in machine learning algorithms. In: Computer Science On-line Conference, Cham, 2023, pp.15\u201325: Springer International Publishing."},{"key":"e_1_3_3_24_2","unstructured":"Han T Adams LC Papaioannou JM et al. MedAlpaca\u2014an open-source collection of medical conversational AI models and training data. arXiv 2023; arXiv:2304.08247."},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbac409"},{"key":"e_1_3_3_26_2","unstructured":"Toma A Lawler PR Ba J et al. Clinical Camel: an open expert-level medical language model with dialogue-based knowledge encoding. arXiv 2023; arXiv:2305.12031."},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1093\/hmg\/ddq160"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.3390\/ijms161125961"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1002\/alz.13016"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1212\/WNL.0000000000000240"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.2196\/53559"},{"key":"e_1_3_3_32_2","article-title":"The DisGeNET knowledge platform for disease genomics: 2019 update","volume":"48","author":"Pi\u00f1ero J","year":"2020","unstructured":"Pi\u00f1ero J, Ram\u00edrez-Anguita JM, Sa\u00fcch-Pitarch J, et al. The DisGeNET knowledge platform for disease genomics: 2019 update. Nucleic Acids Res 2020; 48: D845\u2013D855.","journal-title":"Nucleic Acids Res"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkad976"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkv1277"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1111\/add.16168"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0323761"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1111\/add.16715"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.2174\/1568026622666220907114443"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa2100708"},{"key":"e_1_3_3_40_2","volume-title":"Donepezil, galantamine, rivastigmine and memantine for the treatment of Alzheimer\u2019s disease","author":"National Institute for Health and Clinical Excellence","year":"2011","unstructured":"National Institute for Health and Clinical Excellence. Donepezil, galantamine, rivastigmine and memantine for the treatment of Alzheimer\u2019s disease. London: NICE, 2011."},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1002\/alz.14313"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1016\/S1474-4422(20)30231-3"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/S1474-4422(15)70016-5"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.3233\/JAD-170585"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2013.10.008"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.4103\/1673-5374.274329"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neures.2024.06.003"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.iotcps.2023.04.003"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10439-023-03284-0"}],"container-title":["Journal of Alzheimer\u2019s Disease"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/13872877261415635","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/13872877261415635","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/13872877261415635","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,13]],"date-time":"2026-05-13T10:20:26Z","timestamp":1778667626000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/13872877261415635"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,20]]},"references-count":48,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,3]]}},"alternative-id":["10.1177\/13872877261415635"],"URL":"https:\/\/doi.org\/10.1177\/13872877261415635","relation":{},"ISSN":["1387-2877","1875-8908"],"issn-type":[{"value":"1387-2877","type":"print"},{"value":"1875-8908","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,20]]}}}