{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T22:44:29Z","timestamp":1777502669571,"version":"3.51.4"},"reference-count":72,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,1,21]],"date-time":"2025-01-21T00:00:00Z","timestamp":1737417600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,1,21]],"date-time":"2025-01-21T00:00:00Z","timestamp":1737417600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100012688","name":"Universit\u00e4t Rostock","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100012688","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Drug development is known to be a costly and time-consuming process, which is prone to high failure rates. Drug repurposing allows drug discovery by reusing already approved compounds. The outcomes of past clinical trials can be used to predict novel drug-disease associations by leveraging drug- and disease-related similarities. To tackle this classification problem, collaborative filtering with implicit feedback (and potentially additional data on drugs and diseases) has become popular. It can handle large imbalances between negative and positive known associations and known and unknown associations. However, properly evaluating the improvement over the state of the art is challenging, as there is no consensus approach to compare models. We propose a reproducible methodology for comparing collaborative filtering-based drug repurposing. We illustrate this method by comparing 11 models from the literature on eight diverse drug repurposing datasets. Based on this benchmark, we derive guidelines to ensure a fair and comprehensive evaluation of the performance of those models. In particular, an uncontrolled bias on unknown associations might lead to severe data leakage and a misestimation of the model\u2019s true performance. Moreover, in drug repurposing, the ability of a model to extrapolate beyond its training distribution is crucial and should also be assessed. Finally, we identified a subcategory of collaborative filtering that seems efficient and robust to distribution shifts. Benchmarks constitute an essential step towards increased reproducibility and more accessible development of competitive drug repurposing methods.<\/jats:p>","DOI":"10.1038\/s41598-025-85927-x","type":"journal-article","created":{"date-parts":[[2025,1,21]],"date-time":"2025-01-21T09:49:15Z","timestamp":1737452955000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Comprehensive evaluation of pure and hybrid collaborative filtering in drug repurposing"],"prefix":"10.1038","volume":"15","author":[{"given":"Cl\u00e9mence","family":"R\u00e9da","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jill-J\u00eann","family":"Vie","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Olaf","family":"Wolkenhauer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,1,21]]},"reference":[{"key":"85927_CR1","doi-asserted-by":"publisher","first-page":"192","DOI":"10.1089\/genedge.5.1.39","volume":"5","author":"A Philippidis","year":"2023","unstructured":"Philippidis, A. The unbearable cost of drug development: Deloitte report shows 15% jump in r &d to \\$2.3 billion: A separate study published by british researchers shows biopharma giants spent 57% more on operating costs than research from 1999\u20132018. GEN Edge 5, 192\u2013198 (2023).","journal-title":"GEN Edge"},{"key":"85927_CR2","doi-asserted-by":"publisher","first-page":"3049","DOI":"10.1016\/j.apsb.2022.02.002","volume":"12","author":"D Sun","year":"2022","unstructured":"Sun, D., Gao, W., Hu, H. & Zhou, S. Why 90% of clinical drug development fails and how to improve it?. Acta Pharmaceutica Sinica B 12, 3049\u20133062 (2022).","journal-title":"Acta Pharmaceutica Sinica B"},{"key":"85927_CR3","doi-asserted-by":"publisher","first-page":"852","DOI":"10.1056\/NEJMsa1012065","volume":"364","author":"DA Zarin","year":"2011","unstructured":"Zarin, D. A., Tse, T., Williams, R. J., Califf, R. M. & Ide, N. C. The ClinicalTrials.gov results database-update and key issues. N. Engl. J. Med. 364, 852\u2013860 (2011).","journal-title":"N. Engl. J. Med."},{"key":"85927_CR4","doi-asserted-by":"publisher","first-page":"1437","DOI":"10.1016\/j.cell.2017.10.049","volume":"171","author":"A Subramanian","year":"2017","unstructured":"Subramanian, A. et al. A next generation connectivity map: L1000 platform and the first 1,000,000 profiles. Cell 171, 1437\u20131452 (2017).","journal-title":"Cell"},{"key":"85927_CR5","doi-asserted-by":"publisher","first-page":"D955","DOI":"10.1093\/nar\/gks1111","volume":"41","author":"W Yang","year":"2012","unstructured":"Yang, W. et al. Genomics of drug sensitivity in cancer (GDSC): A resource for therapeutic biomarker discovery in cancer cells. Nucleic Acids Res. 41, D955\u2013D961 (2012).","journal-title":"Nucleic Acids Res."},{"key":"85927_CR6","doi-asserted-by":"publisher","first-page":"186","DOI":"10.1002\/wsbm.1337","volume":"8","author":"RA Hodos","year":"2016","unstructured":"Hodos, R. A., Kidd, B. A., Shameer, K., Readhead, B. P. & Dudley, J. T. In silico methods for drug repurposing and pharmacology. Wiley Interdiscip. Rev. Syst. Biol. Med. 8, 186\u2013210 (2016).","journal-title":"Wiley Interdiscip. Rev. Syst. Biol. Med."},{"key":"85927_CR7","doi-asserted-by":"publisher","first-page":"18911","DOI":"10.1038\/s41598-019-54849-w","volume":"9","author":"AD Hingorani","year":"2019","unstructured":"Hingorani, A. D. et al. Improving the odds of drug development success through human genomics: Modelling study. Sci. Rep. 9, 18911 (2019).","journal-title":"Sci. Rep."},{"key":"85927_CR8","doi-asserted-by":"publisher","first-page":"186","DOI":"10.1002\/wsbm.1337","volume":"8","author":"RA Hodos","year":"2016","unstructured":"Hodos, R. A., Kidd, B. A., Khader, S., Readhead, B. P. & Dudley, J. T. Computational approaches to drug repurposing and pharmacology. Wiley Interdiscip. Rev. Syst. Biol. Med. 8, 186 (2016).","journal-title":"Wiley Interdiscip. Rev. Syst. Biol. Med."},{"key":"85927_CR9","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1038\/nrd.2018.168","volume":"18","author":"S Pushpakom","year":"2019","unstructured":"Pushpakom, S. et al. Drug repurposing: Progress, challenges and recommendations. Nat. Rev. Drug Discov. 18, 41\u201358 (2019).","journal-title":"Nat. Rev. Drug Discov."},{"key":"85927_CR10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/sdata.2017.29","volume":"4","author":"AS Brown","year":"2017","unstructured":"Brown, A. S. & Patel, C. J. A standard database for drug repositioning. Sci. Data 4, 1\u20137 (2017).","journal-title":"Sci. Data"},{"key":"85927_CR11","doi-asserted-by":"publisher","first-page":"590","DOI":"10.1038\/ncpneuro0618","volume":"3","author":"RT Johnson","year":"2007","unstructured":"Johnson, R. T. & Dickersin, K. Publication bias against negative results from clinical trials: Three of the seven deadly sins. Nat. Clin. Practice Neurol. 3, 590\u2013591 (2007).","journal-title":"Nat. Clin. Practice Neurol."},{"key":"85927_CR12","doi-asserted-by":"publisher","first-page":"453","DOI":"10.4103\/ija.IJA_361_17","volume":"61","author":"SB Bhaskar","year":"2017","unstructured":"Bhaskar, S. B. Concealing research outcomes: Missing data, negative results and missed publications. Indian J. Anaesthesia 61, 453 (2017).","journal-title":"Indian J. Anaesthesia"},{"key":"85927_CR13","doi-asserted-by":"publisher","unstructured":"R\u00e9da, C. TRANSCRIPT drug repurposing dataset (2.0.0), https:\/\/doi.org\/10.5281\/zenodo.7982976 (2023). Dataset.","DOI":"10.5281\/zenodo.7982976"},{"key":"85927_CR14","doi-asserted-by":"publisher","unstructured":"R\u00e9da, C. PREDICT drug repurposing dataset (2.0.1), https:\/\/doi.org\/10.5281\/zenodo.7983090 (2023). Dataset.","DOI":"10.5281\/zenodo.7983090"},{"key":"85927_CR15","unstructured":"Niu, G., du Plessis, M.\u00a0C., Sakai, T., Ma, Y. & Sugiyama, M. Theoretical comparisons of positive-unlabeled learning against positive-negative learning. Adv. Neural Inform. Process. Syst. 29 (2016)."},{"key":"85927_CR16","unstructured":"Spezzano, F., Chen, W. & Xiao, X. Proceedings of the 2019 IEEE\/ACM International Conference on Advances in Social Networks Analysis and Mining (ACM, 2019)."},{"key":"85927_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12859-019-2983-2","volume":"20","author":"X Yang","year":"2019","unstructured":"Yang, X., Zamit, l, Liu, Y. & He, J. Additional neural matrix factorization model for computational drug repositioning. BMC Bioinform. 20, 1\u201311 (2019).","journal-title":"BMC Bioinform."},{"key":"85927_CR18","doi-asserted-by":"publisher","first-page":"i455","DOI":"10.1093\/bioinformatics\/btz331","volume":"35","author":"M Yang","year":"2019","unstructured":"Yang, M., Luo, H., Li, Y. & Wang, J. Drug repositioning based on bounded nuclear norm regularization. Bioinformatics 35, i455\u2013i463 (2019).","journal-title":"Bioinformatics"},{"key":"85927_CR19","doi-asserted-by":"crossref","unstructured":"Tak\u00e1cs, G., Pil\u00e1szy, I., N\u00e9meth, B. & Tikk, D. Matrix factorization and neighbor-based algorithms for the Netflix prize problem. In Proceedings of the 2008 ACM Conference on Recommender Systems, 267\u2013274 (2008).","DOI":"10.1145\/1454008.1454049"},{"key":"85927_CR20","unstructured":"Fix, E. Discriminatory analysis: nonparametric discrimination, consistency properties, vol.\u00a01 (USAF school of Aviation Medicine, 1985)."},{"key":"85927_CR21","doi-asserted-by":"crossref","unstructured":"Rendle, S. Factorization machines. In 2010 IEEE International conference on data mining, 995\u20131000 (IEEE, 2010).","DOI":"10.1109\/ICDM.2010.127"},{"key":"85927_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12859-019-3288-1","volume":"20","author":"H Liu","year":"2019","unstructured":"Liu, H. et al. Predicting effective drug combinations using gradient tree boosting based on features extracted from drug-protein heterogeneous network. BMC Bioinform. 20, 1\u201312 (2019).","journal-title":"BMC Bioinform."},{"key":"85927_CR23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12859-020-03898-4","volume":"21","author":"J He","year":"2020","unstructured":"He, J. et al. Hybrid attentional memory network for computational drug repositioning. BMC Bioinform. 21, 1\u201317 (2020).","journal-title":"BMC Bioinform."},{"key":"85927_CR24","doi-asserted-by":"crossref","unstructured":"Yang, X., Yang, G. & Chu, J. The computational drug repositioning without negative sampling. IEEE\/ACM Transactions on Computational Biology and Bioinformatics (2022).","DOI":"10.1109\/TCBB.2022.3212051"},{"key":"85927_CR25","doi-asserted-by":"crossref","unstructured":"Yang, X., Yang, G. & Chu, J. Self-supervised learning for label sparsity in computational drug repositioning. IEEE\/ACM Transactions on Computational Biology and Bioinformatics (2023).","DOI":"10.1109\/TCBB.2023.3254163"},{"key":"85927_CR26","first-page":"8784","volume":"35","author":"C Luo","year":"2021","unstructured":"Luo, C. et al. PULNS: Positive-unlabeled learning with effective negative sample selector. Proc. AAAI Conf. Artif. Intell. 35, 8784\u20138792 (2021).","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"85927_CR27","unstructured":"McDermott, M. et al. Reproducibility in machine learning for health. arXiv preprint[SPACE]arXiv:1907.01463 (2019)."},{"key":"85927_CR28","first-page":"29304","volume":"34","author":"R Agarwal","year":"2021","unstructured":"Agarwal, R., Schwarzer, M., Castro, P. S., Courville, A. C. & Bellemare, M. Deep reinforcement learning at the edge of the statistical precipice. Adv. Neural Inform. Process. Syst. 34, 29304\u201329320 (2021).","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"85927_CR29","doi-asserted-by":"crossref","unstructured":"Kapoor, S. & Narayanan, A. Leakage and the reproducibility crisis in machine-learning-based science. Patterns 4 (2023).","DOI":"10.1016\/j.patter.2023.100804"},{"key":"85927_CR30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2597627","volume":"14","author":"T Hassner","year":"2015","unstructured":"Hassner, T. & Bayaz, I. Teaching computer vision: Bringing research benchmarks to the classroom. ACM Trans. Comput. Educ. (TOCE) 14, 1\u201317 (2015).","journal-title":"ACM Trans. Comput. Educ. (TOCE)"},{"key":"85927_CR31","unstructured":"K\u00f6pf, A. et al. Openassistant conversations-democratizing large language model alignment. Adv. Neural Inform. Process. Syst. 36 (2024)."},{"key":"85927_CR32","doi-asserted-by":"publisher","first-page":"5973","DOI":"10.21105\/joss.05973","volume":"9","author":"C R\u00e9da","year":"2024","unstructured":"R\u00e9da, C., Vie, J.-J. & Wolkenhauer, O. stanscofi and benchscofi: A new standard for drug repurposing by collaborative filtering. J. Open Source Softw. 9, 5973 (2024).","journal-title":"J. Open Source Softw."},{"key":"85927_CR33","doi-asserted-by":"publisher","first-page":"2664","DOI":"10.1093\/bioinformatics\/btw228","volume":"32","author":"H Luo","year":"2016","unstructured":"Luo, H. et al. Drug repositioning based on comprehensive similarity measures and bi-random walk algorithm. Bioinformatics 32, 2664\u20132671 (2016).","journal-title":"Bioinformatics"},{"key":"85927_CR34","doi-asserted-by":"publisher","first-page":"496","DOI":"10.1038\/msb.2011.26","volume":"7","author":"A Gottlieb","year":"2011","unstructured":"Gottlieb, A., Stein, G. Y., Ruppin, E. & Sharan, R. PREDICT: A method for inferring novel drug indications with application to personalized medicine. Mol. Syst. Biol. 7, 496 (2011).","journal-title":"Mol. Syst. Biol."},{"key":"85927_CR35","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1016\/j.artmed.2014.11.003","volume":"63","author":"V Martinez","year":"2015","unstructured":"Martinez, V., Navarro, C., Cano, C., Fajardo, W. & Blanco, A. DrugNet: Network-based drug-disease prioritization by integrating heterogeneous data. Artif. Intell. Med. 63, 41\u201349 (2015).","journal-title":"Artif. Intell. Med."},{"key":"85927_CR36","doi-asserted-by":"publisher","DOI":"10.3389\/fphar.2021.784171","volume":"12","author":"C-Q Gao","year":"2022","unstructured":"Gao, C.-Q., Zhou, Y.-K., Xin, X.-H., Min, H. & Du, P.-F. DDA-SKF: Predicting drug-disease associations using similarity kernel fusion. Front. Pharmacol. 12, 784171 (2022).","journal-title":"Front. Pharmacol."},{"key":"85927_CR37","doi-asserted-by":"publisher","first-page":"1187","DOI":"10.1093\/bioinformatics\/btw770","volume":"33","author":"X Liang","year":"2017","unstructured":"Liang, X. et al. LRSSL: Predict and interpret drug-disease associations based on data integration using sparse subspace learning. Bioinformatics 33, 1187\u20131196 (2017).","journal-title":"Bioinformatics"},{"key":"85927_CR38","unstructured":"Hsieh, Y.-G., Niu, G. & Sugiyama, M. Classification from positive, unlabeled and biased negative data. In International Conference on Machine Learning, 2820\u20132829 (PMLR, 2019)."},{"key":"85927_CR39","doi-asserted-by":"publisher","first-page":"D901","DOI":"10.1093\/nar\/gkm958","volume":"36","author":"DS Wishart","year":"2008","unstructured":"Wishart, D. S. et al. DrugBank: A knowledgebase for drugs, drug actions and drug targets. Nucleic Acids Res. 36, D901\u2013D906 (2008).","journal-title":"Nucleic Acids Res."},{"key":"85927_CR40","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1002\/(SICI)1098-1004(200001)15:1<57::AID-HUMU12>3.0.CO;2-G","volume":"15","author":"A Hamosh","year":"2000","unstructured":"Hamosh, A., Scott, A. F., Amberger, J., Valle, D. & McKusick, V. A. Online mendelian inheritance in man (OMIM). Human Mutation 15, 57\u201361 (2000).","journal-title":"Human Mutation"},{"key":"85927_CR41","first-page":"296","volume":"98","author":"D Lin","year":"1998","unstructured":"Lin, D. et al. An information-theoretic definition of similarity. ICML 98, 296\u2013304 (1998).","journal-title":"ICML"},{"key":"85927_CR42","doi-asserted-by":"publisher","first-page":"D940","DOI":"10.1093\/nar\/gkr972","volume":"40","author":"LM Schriml","year":"2012","unstructured":"Schriml, L. M. et al. Disease ontology: A backbone for disease semantic integration. Nucleic Acids Res. 40, D940\u2013D946 (2012).","journal-title":"Nucleic Acids Res."},{"key":"85927_CR43","doi-asserted-by":"publisher","first-page":"1644","DOI":"10.1093\/bioinformatics\/btq241","volume":"26","author":"D Wang","year":"2010","unstructured":"Wang, D., Wang, J., Lu, M., Song, F. & Cui, Q. Inferring the human microrna functional similarity and functional network based on microrna-associated diseases. Bioinformatics 26, 1644\u20131650 (2010).","journal-title":"Bioinformatics"},{"key":"85927_CR44","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1038\/75556","volume":"25","author":"M Ashburner","year":"2000","unstructured":"Ashburner, M. et al. Gene ontology: Tool for the unification of biology. Nat. Genet. 25, 25\u201329 (2000).","journal-title":"Nat. Genet."},{"key":"85927_CR45","doi-asserted-by":"publisher","first-page":"12846","DOI":"10.1038\/ncomms12846","volume":"7","author":"Z Wang","year":"2016","unstructured":"Wang, Z. et al. Extraction and analysis of signatures from the gene expression omnibus by the crowd. Nat. Commun. 7, 12846 (2016).","journal-title":"Nat. Commun."},{"key":"85927_CR46","doi-asserted-by":"publisher","unstructured":"R\u00e9da, C. Drug repurposing datasets for collaborative filtering methods (2.0.0), https:\/\/doi.org\/10.5281\/zenodo.8014775 (2023). Notebooks.","DOI":"10.5281\/zenodo.8014775"},{"key":"85927_CR47","doi-asserted-by":"crossref","unstructured":"Bell, Koren & Volinsky. Modeling relationships at multiple scales to improve accuracy of large recommender systems. In Proceedings of the 13th ACM SIGKDD (2007).","DOI":"10.1145\/1281192.1281206"},{"key":"85927_CR48","first-page":"1","volume":"17","author":"W-S Chin","year":"2016","unstructured":"Chin, W.-S. et al. LIBMF: A library for parallel matrix factorization in shared-memory systems. J. Machine Learn. Res. 17, 1\u20135 (2016).","journal-title":"J. Machine Learn. Res."},{"key":"85927_CR49","first-page":"1","volume":"27","author":"CC Johnson","year":"2014","unstructured":"Johnson, C. C. et al. Logistic matrix factorization for implicit feedback data. Adv. Neural Inform. Process. Syst. 27, 1\u20139 (2014).","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"85927_CR50","unstructured":"Mnih, A. & Salakhutdinov, R.\u00a0R. Probabilistic matrix factorization. Adv. Neural Informat. Process. Syst. 20 (2007)."},{"key":"85927_CR51","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2021.107135","volume":"103","author":"Y Meng","year":"2021","unstructured":"Meng, Y., Jin, M., Tang, X. & Xu, J. Drug repositioning based on similarity constrained probabilistic matrix factorization: COVID-19 as a case study. Appl. Soft Computing 103, 107135 (2021).","journal-title":"Appl. Soft Computing"},{"key":"85927_CR52","unstructured":"Howard et al. fastai. https:\/\/github.com\/fastai\/fastai (2018)."},{"key":"85927_CR53","doi-asserted-by":"publisher","first-page":"2538","DOI":"10.1093\/bioinformatics\/btz965","volume":"36","author":"J Li","year":"2020","unstructured":"Li, J. et al. Neural inductive matrix completion with graph convolutional networks for mirna-disease association prediction. Bioinformatics 36, 2538\u20132546 (2020).","journal-title":"Bioinformatics"},{"key":"85927_CR54","doi-asserted-by":"publisher","first-page":"1904","DOI":"10.1093\/bioinformatics\/bty013","volume":"34","author":"H Luo","year":"2018","unstructured":"Luo, H. et al. Computational drug repositioning using low-rank matrix approximation and randomized algorithms. Bioinformatics 34, 1904\u20131912 (2018).","journal-title":"Bioinformatics"},{"key":"85927_CR55","doi-asserted-by":"crossref","unstructured":"Wang, X. et al. Heterogeneous graph attention network. In The world wide web conference, 2022\u20132032 (2019).","DOI":"10.1145\/3308558.3313562"},{"key":"85927_CR56","doi-asserted-by":"crossref","unstructured":"Damak, K., Khenissi, S. & Nasraoui, O. Debiased explainable pairwise ranking from implicit feedback. In Proceedings of the 15th ACM Conference on Recommender Systems, 321\u2013331 (2021).","DOI":"10.1145\/3460231.3474274"},{"key":"85927_CR57","unstructured":"Agarwal, A., Dahleh, M., Shah, D. & Shen, D. Causal matrix completion. In The thirty sixth annual conference on learning theory, 3821\u20133826 (PMLR, 2023)."},{"key":"85927_CR58","first-page":"1","volume":"41","author":"J Chen","year":"2023","unstructured":"Chen, J. et al. Bias and debias in recommender system: A survey and future directions. ACM Trans. Inform. Syst. 41, 1\u201339 (2023).","journal-title":"ACM Trans. Inform. Syst."},{"key":"85927_CR59","unstructured":"Schnabel, T., Swaminathan, A., Singh, A., Chandak, N. & Joachims, T. Recommendations as treatments: Debiasing learning and evaluation. In international conference on machine learning, 1670\u20131679 (PMLR, 2016)."},{"key":"85927_CR60","doi-asserted-by":"crossref","unstructured":"Yu, H.-F., Bilenko, M. & Lin, C.-J. Selection of negative samples for one-class matrix factorization. In Proceedings of the 2017 SIAM International Conference on Data Mining, 363\u2013371 (SIAM, 2017).","DOI":"10.1137\/1.9781611974973.41"},{"key":"85927_CR61","doi-asserted-by":"crossref","unstructured":"Bekkar, M., Djemaa, H.\u00a0K. & Alitouche, T.\u00a0A. Evaluation measures for models assessment over imbalanced data sets. J. Inform. Eng. Appl. 3 (2013).","DOI":"10.5121\/ijdkp.2013.3402"},{"key":"85927_CR62","doi-asserted-by":"publisher","first-page":"164","DOI":"10.1126\/science.adg8538","volume":"383","author":"AM Chekroud","year":"2024","unstructured":"Chekroud, A. M. et al. Illusory generalizability of clinical prediction models. Science 383, 164\u2013167 (2024).","journal-title":"Science"},{"key":"85927_CR63","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","volume":"57","author":"Y Benjamini","year":"1995","unstructured":"Benjamini, Y. & Hochberg, Y. Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. R. Stat. Soc. Series B (Methodol.) 57, 289\u2013300 (1995).","journal-title":"J. R. Stat. Soc. Series B (Methodol.)"},{"key":"85927_CR64","first-page":"18583","volume":"33","author":"R Taori","year":"2020","unstructured":"Taori, R. et al. Measuring robustness to natural distribution shifts in image classification. Adv. Neural Inform. Process. Syst. 33, 18583\u201318599 (2020).","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"85927_CR65","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1038\/s43856-023-00356-z","volume":"3","author":"T Shadbahr","year":"2023","unstructured":"Shadbahr, T. et al. The impact of imputation quality on machine learning classifiers for datasets with missing values. Commun. Med. 3, 139 (2023).","journal-title":"Commun. Med."},{"key":"85927_CR66","unstructured":"Morvan, M.\u00a0L. & Varoquaux, G. Imputation for prediction: beware of diminishing returns. arXiv preprint[SPACE]arXiv:2407.19804 (2024)."},{"key":"85927_CR67","doi-asserted-by":"publisher","first-page":"3091","DOI":"10.1214\/21-AOS2137","volume":"50","author":"W Fithian","year":"2022","unstructured":"Fithian, W. & Lei, L. Conditional calibration for false discovery rate control under dependence. Ann. Stat. 50, 3091\u20133118 (2022).","journal-title":"Ann. Stat."},{"key":"85927_CR68","unstructured":"Pedregosa, F. et al. Scikit-learn: Machine learning in python. J. Machine Learn. Res. 12, 2825\u20132830 (2011)."},{"key":"85927_CR69","unstructured":"Bethesda (MD): National Library of Medicine\u00a0(US), N. C. f. B.\u00a0I. National center for biotechnology information (NCBI). https:\/\/www.ncbi.nlm.nih.gov (1988). Cited 2024 Nov 08."},{"key":"85927_CR70","unstructured":"McKusick-Nathans Institute of Genetic Medicine, M., Johns Hopkins University\u00a0(Baltimore. Online mendelian inheritance in man, OMIM. https:\/\/omim.org (1998). Cited 2024 Nov 08."},{"key":"85927_CR71","doi-asserted-by":"publisher","first-page":"D1265","DOI":"10.1093\/nar\/gkad976","volume":"52","author":"C Knox","year":"2024","unstructured":"Knox, C. et al. DrugBank 6.0: The DrugBank knowledgebase for 2024. Nucleic Acids Res. 52, D1265\u2013D1275 (2024).","journal-title":"Nucleic Acids Res."},{"key":"85927_CR72","doi-asserted-by":"publisher","first-page":"D1373","DOI":"10.1093\/nar\/gkac956","volume":"51","author":"S Kim","year":"2023","unstructured":"Kim, S. et al. PubChem 2023 update. Nucleic Acids Res. 51, D1373\u2013D1380 (2023).","journal-title":"Nucleic Acids Res."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-85927-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-85927-x","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-85927-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,21]],"date-time":"2025-01-21T11:16:11Z","timestamp":1737458171000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-85927-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,21]]},"references-count":72,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["85927"],"URL":"https:\/\/doi.org\/10.1038\/s41598-025-85927-x","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-4649327\/v1","asserted-by":"object"}]},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,21]]},"assertion":[{"value":"27 June 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 January 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 January 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"2711"}}