{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T22:41:26Z","timestamp":1764715286046,"version":"3.38.0"},"reference-count":72,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,3,5]],"date-time":"2025-03-05T00:00:00Z","timestamp":1741132800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,3,5]],"date-time":"2025-03-05T00:00:00Z","timestamp":1741132800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100019180","name":"HORIZON EUROPE European Research Council","doi-asserted-by":"publisher","award":["956832","956832","956832"],"award-info":[{"award-number":["956832","956832","956832"]}],"id":[{"id":"10.13039\/100019180","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004497","name":"Onderzoeksraad, KU Leuven","doi-asserted-by":"publisher","award":["C14\/22\/125","C14\/22\/125"],"award-info":[{"award-number":["C14\/22\/125","C14\/22\/125"]}],"id":[{"id":"10.13039\/501100004497","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011878","name":"Vlaamse regering","doi-asserted-by":"publisher","award":["S003422N","S003422N"],"award-info":[{"award-number":["S003422N","S003422N"]}],"id":[{"id":"10.13039\/501100011878","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cheminform"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>In the drug discovery process, where experiments can be costly and time-consuming, computational models that predict drug-target interactions are valuable tools to accelerate the development of new therapeutic agents. Estimating the uncertainty inherent in these neural network predictions provides valuable information that facilitates optimal decision-making when risk assessment is crucial. However, such models can be poorly calibrated, which results in unreliable uncertainty estimates that do not reflect the true predictive uncertainty. In this study, we compare different metrics, including accuracy and calibration scores, used for model hyperparameter tuning to investigate which model selection strategy achieves well-calibrated models. Furthermore, we propose to use a computationally efficient Bayesian uncertainty estimation method named HMC Bayesian Last Layer (HBLL), which generates Hamiltonian Monte Carlo (HMC) trajectories to obtain samples for the parameters of a Bayesian logistic regression fitted to the hidden layer of the baseline neural network. We report that this approach improves model calibration and achieves the performance of common uncertainty quantification methods by combining the benefits of uncertainty estimation and probability calibration methods. Finally, we show that combining post hoc calibration method with well-performing uncertainty quantification approaches can boost model accuracy and calibration. <\/jats:p>","DOI":"10.1186\/s13321-025-00964-y","type":"journal-article","created":{"date-parts":[[2025,3,5]],"date-time":"2025-03-05T10:13:31Z","timestamp":1741169611000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Achieving well-informed decision-making in drug discovery: a comprehensive calibration study using neural network-based structure-activity models"],"prefix":"10.1186","volume":"17","author":[{"given":"Hannah Rosa","family":"Friesacher","sequence":"first","affiliation":[]},{"given":"Ola","family":"Engkvist","sequence":"additional","affiliation":[]},{"given":"Lewis","family":"Mervin","sequence":"additional","affiliation":[]},{"given":"Yves","family":"Moreau","sequence":"additional","affiliation":[]},{"given":"Adam","family":"Arany","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,3,5]]},"reference":[{"key":"964_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s40273-021-01065-y","volume":"39","author":"M Schlander","year":"2021","unstructured":"Schlander M, Hernandez-Villafuerte K, Cheng C-Y, Mestre-Ferrandiz J, Baumann M (2021) How much does it cost to research and develop a new drug? a systematic review and assessment. Phar Econom 39:1\u201327. https:\/\/doi.org\/10.1007\/s40273-021-01065-y","journal-title":"Phar Econom"},{"issue":"9","key":"964_CR2","doi-asserted-by":"publisher","first-page":"844","DOI":"10.1001\/jama.2020.1166","volume":"323","author":"OJ Wouters","year":"2020","unstructured":"Wouters OJ, McKee M, Luyten J (2020) Estimated Research and Development Investment Needed to Bring a New Medicine to Market, 2009\u20132018. JAMA 323(9):844\u2013853. https:\/\/doi.org\/10.1001\/jama.2020.1166","journal-title":"JAMA"},{"issue":"6","key":"964_CR3","doi-asserted-by":"publisher","first-page":"1241","DOI":"10.1016\/j.drudis.2018.01.039","volume":"23","author":"H Chen","year":"2018","unstructured":"Chen H, Engkvist O, Wang Y, Olivecrona M, Blaschke T (2018) The rise of deep learning in drug discovery. Drug Disc Today 23(6):1241\u20131250. https:\/\/doi.org\/10.1016\/j.drudis.2018.01.039","journal-title":"Drug Disc Today"},{"key":"964_CR4","doi-asserted-by":"publisher","first-page":"895","DOI":"10.1007\/s12257-020-0049-y","volume":"25","author":"H Kim","year":"2020","unstructured":"Kim H, Kim E, Lee I, Bae B, Park M, Nam H (2020) Artificial intelligence in drug discovery: a comprehensive review of data-driven and machine learning approaches. Biotechnol Biopro Eng 25:895\u2013930. https:\/\/doi.org\/10.1007\/s12257-020-0049-y","journal-title":"Biotechnol Biopro Eng"},{"key":"964_CR5","doi-asserted-by":"publisher","DOI":"10.3390\/ijms24032026","author":"C Sarkar","year":"2023","unstructured":"Sarkar C, Das B, Rawat VS, Wahlang JB, Nongpiur A, Tiewsoh I, Lyngdoh NM, Das D, Bidarolli M, Sony HT (2023) Artificial intelligence and machine learning technology driven modern drug discovery and development. Int J Mol Sci. https:\/\/doi.org\/10.3390\/ijms24032026","journal-title":"Int J Mol Sci"},{"key":"964_CR6","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1016\/j.inffus.2021.05.008","volume":"76","author":"M Abdar","year":"2021","unstructured":"Abdar M, Pourpanah F, Hussain S, Rezazadegan D, Liu L, Ghavamzadeh M, Fieguth P, Cao X, Khosravi A, Acharya UR, Makarenkov V, Nahavandi S (2021) A review of uncertainty quantification in deep learning: techniques, applications and challenges. Infor Fusion 76:243\u2013297. https:\/\/doi.org\/10.1016\/j.inffus.2021.05.008","journal-title":"Infor Fusion"},{"issue":"2","key":"964_CR7","doi-asserted-by":"publisher","first-page":"474","DOI":"10.1016\/j.drudis.2020.11.027","volume":"26","author":"LH Mervin","year":"2021","unstructured":"Mervin LH, Johansson S, Semenova E, Giblin KA, Engkvist O (2021) Uncertainty quantification in drug design. Drug Disc Today 26(2):474\u2013489. https:\/\/doi.org\/10.1016\/j.drudis.2020.11.027","journal-title":"Drug Disc Today"},{"issue":"1","key":"964_CR8","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1038\/s42256-018-0004-1","volume":"1","author":"E Begoli","year":"2019","unstructured":"Begoli E, Bhattacharya T, Kusnezov D (2019) The need for uncertainty quantification in machine-assisted medical decision making. Nat Machine Intell 1(1):20\u201323. https:\/\/doi.org\/10.1038\/s42256-018-0004-1","journal-title":"Nat Machine Intell"},{"issue":"1","key":"964_CR9","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1109\/TMI.2020.3025065","volume":"40","author":"V Edupuganti","year":"2021","unstructured":"Edupuganti V, Mardani M, Vasanawala S, Pauly J (2021) Uncertainty quantification in deep MRI reconstruction. IEEE Transac Med Imag 40(1):239\u2013250. https:\/\/doi.org\/10.1109\/TMI.2020.3025065","journal-title":"IEEE Transac Med Imag"},{"key":"964_CR10","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2020.117366","volume":"225","author":"R Tanno","year":"2021","unstructured":"Tanno R, Worrall DE, Kaden E, Ghosh A, Grussu F, Bizzi A, Sotiropoulos SN, Criminisi A, Alexander DC (2021) Uncertainty modelling in deep learning for safer neuroimage enhancement: Demonstration in diffusion MRI. NeuroImage 225:117366. https:\/\/doi.org\/10.1016\/j.neuroimage.2020.117366","journal-title":"NeuroImage"},{"key":"964_CR11","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2024.127339","volume":"576","author":"T Blasco","year":"2024","unstructured":"Blasco T, S\u00e1nchez JS, Garc\u00eda V (2024) A survey on uncertainty quantification in deep learning for financial time series prediction. Neurocomputing 576:127339. https:\/\/doi.org\/10.1016\/j.neucom.2024.127339","journal-title":"Neurocomputing"},{"key":"964_CR12","doi-asserted-by":"publisher","unstructured":"Michelmore R, Wicker M, Laurenti L, Cardelli L, Gal Y, Kwiatkowska M (2020) Uncertainty quantification with statistical guarantees in end-to-end autonomous driving control. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 7344\u20137350. https:\/\/doi.org\/10.1109\/ICRA40945.2020.9196844","DOI":"10.1109\/ICRA40945.2020.9196844"},{"key":"964_CR13","unstructured":"Gruber C, Schenk PO, Schierholz M, Kreuter F, Kauermann G (2023) Sources of Uncertainty in Machine Learning \u2013 A Statisticians\u2019 View. Preprint at https:\/\/arxiv.org\/abs\/2305.16703"},{"issue":"3","key":"964_CR14","doi-asserted-by":"publisher","first-page":"457","DOI":"10.1007\/s10994-021-05946-3","volume":"110","author":"E H\u00fcllermeier","year":"2021","unstructured":"H\u00fcllermeier E, Waegeman W (2021) Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Machine Learn 110(3):457\u2013506. https:\/\/doi.org\/10.1007\/s10994-021-05946-3","journal-title":"Machine Learn"},{"key":"964_CR15","unstructured":"Guo C, Pleiss G, Sun Y, Weinberger KQ (2017) On calibration of modern neural networks. In: International Conference on Machine Learning, pp. 1321\u20131330. PMLR"},{"issue":"10","key":"964_CR16","doi-asserted-by":"publisher","first-page":"4546","DOI":"10.1021\/acs.jcim.0c00476","volume":"60","author":"LH Mervin","year":"2020","unstructured":"Mervin LH, Afzal AM, Engkvist O, Bender A (2020) Comparison of scaling methods to obtain calibrated probabilities of activity for protein-ligand predictions. J Chem Infor Model 60(10):4546\u20134559. https:\/\/doi.org\/10.1021\/acs.jcim.0c00476","journal-title":"J Chem Infor Model"},{"key":"964_CR17","first-page":"15682","volume":"34","author":"M Minderer","year":"2021","unstructured":"Minderer M, Djolonga J, Romijnders R, Hubis F, Zhai X, Houlsby N, Tran D, Lucic M (2021) Revisiting the calibration of modern neural networks. Adv Neural Infor Proc Syst 34:15682\u201315694","journal-title":"Adv Neural Infor Proc Syst"},{"key":"964_CR18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13321-021-00551-x","volume":"13","author":"D Wang","year":"2021","unstructured":"Wang D, Yu J, Chen L, Li X, Jiang H, Chen K, Zheng M, Luo X (2021) A hybrid framework for improving uncertainty quantification in deep learning-based QSAR regression modeling. J Cheminformat 13:1\u201317","journal-title":"J Cheminformat"},{"key":"964_CR19","unstructured":"Wang C. Calibration in Deep Learning: A Survey of the State-of-the-Art. Preprint at https:\/\/arxiv.org\/abs\/2308.01222"},{"key":"964_CR20","unstructured":"Ovadia Y, Fertig E, Ren J, Nado Z, Sculley D, Nowozin S, Dillon J, Lakshminarayanan B, Snoek J (2019) Can you trust your model\u2019s uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems 32"},{"issue":"24","key":"964_CR21","doi-asserted-by":"publisher","first-page":"7699","DOI":"10.1021\/acs.jcim.3c01241","volume":"63","author":"R Wang","year":"2023","unstructured":"Wang R, Liu Z, Gong J, Zhou Q, Guan X, Ge G (2023) An uncertainty-guided deep learning method facilitates rapid screening of cyp3a4 inhibitors. J Chem Infor Model 63(24):7699\u20137710","journal-title":"J Chem Infor Model"},{"key":"964_CR22","doi-asserted-by":"crossref","unstructured":"Fan Z, Yu J, Zhang X, Chen Y, Sun S, Zhang Y, Chen M, Xiao F, Wu W, Li X et al (2024) Reducing overconfident errors in molecular property classification using posterior network. Patterns","DOI":"10.1016\/j.patter.2024.100991"},{"key":"964_CR23","doi-asserted-by":"crossref","unstructured":"Svensson E, Friesacher HR, Winiwarter S, Mervin L, Arany A, Engkvist O (2024) Enhancing uncertainty quantification in drug discovery with censored regression labels. arXiv preprint arXiv:2409.04313","DOI":"10.1016\/j.ailsci.2025.100128"},{"issue":"1","key":"964_CR24","doi-asserted-by":"publisher","DOI":"10.1016\/j.aichem.2023.100004","volume":"1","author":"YJ Fan","year":"2023","unstructured":"Fan YJ, Allen JE, McLoughlin KS, Shi D, Bennion BJ, Zhang X, Lightstone FC (2023) Evaluating point-prediction uncertainties in neural networks for protein-ligand binding prediction. Artif Intell Chem 1(1):100004","journal-title":"Artif Intell Chem"},{"key":"964_CR25","unstructured":"Han K, Lakshminarayanan B, Liu J (2021) Reliable graph neural networks for drug discovery under distributional shift. arXiv preprint arXiv:2111.12951"},{"issue":"6","key":"964_CR26","doi-asserted-by":"publisher","first-page":"2697","DOI":"10.1021\/acs.jcim.9b00975","volume":"60","author":"G Scalia","year":"2020","unstructured":"Scalia G, Grambow CA, Pernici B, Li Y-P, Green WH (2020) Evaluating scalable uncertainty estimation methods for deep learning-based molecular property prediction. J Chem Infor Model 60(6):2697\u20132717","journal-title":"J Chem Infor Model"},{"issue":"20","key":"964_CR27","doi-asserted-by":"publisher","first-page":"3428","DOI":"10.1093\/bioinformatics\/btab346","volume":"37","author":"Q Kim","year":"2021","unstructured":"Kim Q, Ko J-H, Kim S, Park N, Jhe W (2021) Bayesian neural network with pretrained protein embedding enhances prediction accuracy of drug-protein interaction. Bioinformatics 37(20):3428\u20133435","journal-title":"Bioinformatics"},{"key":"964_CR28","first-page":"19446","volume":"36","author":"K Schweighofer","year":"2023","unstructured":"Schweighofer K, Aichberger L, Ielanskyi M, Klambauer G, Hochreiter S (2023) Quantification of uncertainty with adversarial models. Adv Neural Infor Proc Syst 36:19446\u201319484","journal-title":"Adv Neural Infor Proc Syst"},{"issue":"1","key":"964_CR29","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1186\/s13321-023-00709-9","volume":"15","author":"T-M Dutschmann","year":"2023","unstructured":"Dutschmann T-M, Kinzel L, Ter Laak A, Baumann K (2023) Large-scale evaluation of k-fold cross-validation ensembles for uncertainty estimation. J Cheminfor 15(1):49","journal-title":"J Cheminfor"},{"issue":"4","key":"964_CR30","doi-asserted-by":"publisher","first-page":"202300292","DOI":"10.1002\/minf.202300292","volume":"43","author":"M Rayka","year":"2024","unstructured":"Rayka M, Mirzaei M, Mohammad Latifi A (2024) An ensemble-based approach to estimate confidence of predicted protein-ligand binding affinity values. Mol Infor 43(4):202300292","journal-title":"Mol Infor"},{"key":"964_CR31","unstructured":"Dheur V, Taieb SB (2023) A large-scale study of probabilistic calibration in neural network regression. In: International Conference on Machine Learning, pp. 7813\u20137836. PMLR"},{"key":"964_CR32","doi-asserted-by":"crossref","unstructured":"Fiedler F, Lucia S (2023) Improved uncertainty quantification for neural networks with bayesian last layer. IEEE Access","DOI":"10.1109\/ACCESS.2023.3329685"},{"key":"964_CR33","unstructured":"Brunzema P, Jordahn M, Willes J, Trimpe S, Snoek J, Harrison J (2024) Variational last layers for bayesian optimization. In: NeurIPS 2024 Workshop on Bayesian Decision-making and Uncertainty"},{"key":"964_CR34","unstructured":"Ober SW, Rasmussen CE (2019) Benchmarking the Neural Linear Model for Regression. https:\/\/arxiv.org\/abs\/1912.08416"},{"key":"964_CR35","unstructured":"Snoek J, Rippel O, Swersky K, Kiros R, Satish N, Sundaram N, Patwary M, Prabhat M, Adams R (2015) Scalable bayesian optimization using deep neural networks. In: International Conference on Machine Learning, pp. 2171\u20132180. PMLR"},{"key":"964_CR36","unstructured":"Watson J, Lin JA, Klink P, Pajarinen J, Peters J (2021) Latent derivative bayesian last layer networks. In: International Conference on Artificial Intelligence and Statistics, pp. 1198\u20131206. PMLR"},{"key":"964_CR37","unstructured":"Bengs V, H\u00fcllermeier E, Waegeman W (2021) On second-order scoring rules for epistemic uncertainty quantification. In: International Conference on Machine Learning, pp. 2078\u20132091. PMLR"},{"key":"964_CR38","first-page":"54","volume":"10","author":"J Platt","year":"1999","unstructured":"Platt J (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv. Large Margin Classif 10:54","journal-title":"Adv. Large Margin Classif"},{"key":"964_CR39","first-page":"1050","volume":"48","author":"Y Gal","year":"2016","unstructured":"Gal Y, Ghahramani Z (2016) Dropout as a Bayesian approximation: representing model uncertainty in deep learning. Proc Machine Learn Res 48:1050\u20131059","journal-title":"Proc Machine Learn Res"},{"issue":"1","key":"964_CR40","first-page":"1929","volume":"15","author":"N Srivastava","year":"2014","unstructured":"Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Machine Learn Res 15(1):1929\u20131958","journal-title":"J Machine Learn Res"},{"issue":"1","key":"964_CR41","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1186\/s13321-023-00753-5","volume":"15","author":"T Yin","year":"2023","unstructured":"Yin T, Panapitiya G, Coda ED, Saldanha EG (2023) Evaluating uncertainty-based active learning for accelerating the generalization of molecular property prediction. J Cheminfor 15(1):105","journal-title":"J Cheminfor"},{"key":"964_CR42","first-page":"33","volume":"30","author":"B Lakshminarayanan","year":"2017","unstructured":"Lakshminarayanan B, Pritzel A, Blundell C (2017) Simple and scalable predictive uncertainty estimation using deep ensembles. Adv Neural Infor Proc Syst 30:33","journal-title":"Adv Neural Infor Proc Syst"},{"key":"964_CR43","unstructured":"Fort S, Hu H, Lakshminarayanan B (2020) Deep Ensembles: A Loss Landscape Perspective. https:\/\/arxiv.org\/abs\/1912.02757"},{"key":"964_CR44","unstructured":"Harrison J, Willes J, Snoek J (2020) Variational Bayesian Last Layers. https:\/\/arxiv.org\/abs\/2404.11599"},{"key":"964_CR45","doi-asserted-by":"publisher","DOI":"10.1201\/b10905","volume-title":"Handbook of Markov Chain Monte Carlo","author":"S Brooks","year":"2011","unstructured":"Brooks S, Gelman A, Jones G, Meng X-L (2011) Handbook of Markov Chain Monte Carlo. CRC Press, New York"},{"key":"964_CR46","unstructured":"Betancourt M (2018) A Conceptual Introduction to Hamiltonian Monte Carlo. Preprint at https:\/\/arxiv.org\/abs\/1701.02434"},{"key":"964_CR47","unstructured":"Izmailov P, Vikram S, Hoffman MD, Wilson AGG (2021) What are bayesian neural network posteriors really like? In: International Conference on Machine Learning, pp. 4629\u20134640. PMLR"},{"issue":"D1","key":"964_CR48","doi-asserted-by":"publisher","first-page":"930","DOI":"10.1093\/nar\/gky1075","volume":"47","author":"D Mendez","year":"2018","unstructured":"Mendez D, Gaulton A, Bento AP, Chambers J, De Veij M, F\u00e9lix E, Magari\u00f1os M, Mosquera J, Mutowo P, Nowotka M, Gordillo-Mara\u00f1\u00f3n M, Hunter F, Junco L, Mugumbate G, Rodriguez-Lopez M, Atkinson F, Bosc N, Radoux C, Segura-Cabrera A, Hersey A, Leach A (2018) ChEMBL: towards direct deposition of bioassay data. Nucleic Acids Research 47(D1):930\u2013940. https:\/\/doi.org\/10.1093\/nar\/gky1075","journal-title":"Nucleic Acids Research"},{"issue":"9","key":"964_CR49","doi-asserted-by":"publisher","first-page":"846","DOI":"10.3390\/pharmaceutics12090846","volume":"12","author":"M Deodhar","year":"2020","unstructured":"Deodhar M, Al Rihani SB, Arwood MJ, Darakjian L, Dow P, Turgeon J, Michaud V (2020) Mechanisms of cyp450 inhibition: understanding drug-drug interactions due to mechanism-based inhibition in clinical practice. Pharmaceutics 12(9):846","journal-title":"Pharmaceutics"},{"key":"964_CR50","volume-title":"Cytochromes P450: Metabolic and Toxicological Aspects","author":"C Ioannides","year":"1996","unstructured":"Ioannides C (1996) Cytochromes P450: Metabolic and Toxicological Aspects. Crc Press, ???"},{"issue":"5262","key":"964_CR51","doi-asserted-by":"publisher","first-page":"681","DOI":"10.1126\/science.272.5262.681","volume":"272","author":"MT Keating","year":"1996","unstructured":"Keating MT, Sanguinetti MC (1996) Molecular genetic insights into cardiovascular disease. Science 272(5262):681\u2013685","journal-title":"Science"},{"key":"964_CR52","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.6961488","author":"G Landrum","year":"2006","unstructured":"Landrum G (2006) RDKit: Open-Source Cheminformatics. Zenodo. https:\/\/doi.org\/10.5281\/zenodo.6961488","journal-title":"Zenodo"},{"issue":"23","key":"964_CR53","doi-asserted-by":"publisher","first-page":"5938","DOI":"10.1021\/acs.jcim.2c01073","volume":"62","author":"D Van Tilborg","year":"2022","unstructured":"Van Tilborg D, Alenicheva A, Grisoni F (2022) Exposing the limitations of molecular machine learning with activity cliffs. J Chem Infor Model 62(23):5938\u20135951","journal-title":"J Chem Infor Model"},{"issue":"5","key":"964_CR54","doi-asserted-by":"publisher","first-page":"611","DOI":"10.1016\/j.chembiol.2018.01.015","volume":"25","author":"J Simm","year":"2018","unstructured":"Simm J, Klambauer G, Arany A, Steijaert M, Wegner JK, Gustin E, Chupakhin V, Chong YT, Vialard J, Buijnsters P, Velter I, Vapirev A, Singh S, Carpenter AE, Wuyts R, Hochreiter S, Moreau Y, Ceulemans H (2018) Repurposing high-throughput image assays enables biological activity prediction for drug discovery. Cell Chem Biol 25(5):611\u20136183. https:\/\/doi.org\/10.1016\/j.chembiol.2018.01.015","journal-title":"Cell Chem Biol"},{"key":"964_CR55","unstructured":"Arany A, Simm J, Oldenhof M, Moreau Y (2022) Sparsechem: Fast and accurate machine learning model for small molecules. arXiv preprint arXiv:2203.04676"},{"issue":"7","key":"964_CR56","doi-asserted-by":"publisher","first-page":"2331","DOI":"10.1021\/acs.jcim.3c00799","volume":"64","author":"W Heyndrickx","year":"2023","unstructured":"Heyndrickx W, Mervin L, Morawietz T, Sturm N, Friedrich L, Zalewski A, Pentina A, Humbeck L, Oldenhof M, Niwayama R et al (2023) Melloddy: Cross-pharma federated learning at unprecedented scale unlocks benefits in QSAR without compromising proprietary information. J Chem Infor Model 64(7):2331\u20132344","journal-title":"J Chem Infor Model"},{"key":"964_CR57","unstructured":"Sommer E, Wimmer L, Papamarkou T, Bothmann L, Bischl B, R\u00fcgamer D (2024) Connecting the Dots: Is Mode-Connectedness the Key to Feasible Sample-Based Inference in Bayesian Neural Networks?. https:\/\/arxiv.org\/abs\/2402.01484"},{"key":"964_CR58","first-page":"33","volume":"32","author":"A Paszke","year":"2019","unstructured":"Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L et al (2019) Pytorch: an imperative style, high-performance deep learning library. Adv Neural Infor Proc Syst 32:33","journal-title":"Adv Neural Infor Proc Syst"},{"key":"964_CR59","unstructured":"Cobb AD, Baydin AG, Jalaian B (2023) hamiltorch. GitHub. https:\/\/github.com\/AdamCobb\/hamiltorch"},{"key":"964_CR60","first-page":"685","volume":"675","author":"AD Cobb","year":"2021","unstructured":"Cobb AD, Jalaian B (2021) Scaling hamiltonian monte carlo inference for bayesian neural networks with symmetric splitting. Uncertainty Artif Intell 675:685","journal-title":"Uncertainty Artif Intell"},{"issue":"1","key":"964_CR61","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1175\/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2","volume":"78","author":"GW Brier","year":"1950","unstructured":"Brier GW (1950) Verification of forecasts expressed in terms of probability. Monthly Weather Rev 78(1):1\u20133","journal-title":"Monthly Weather Rev"},{"issue":"4","key":"964_CR62","doi-asserted-by":"publisher","first-page":"595","DOI":"10.1175\/1520-0450(1973)012<0595:ANVPOT>2.0.CO;2","volume":"12","author":"AH Murphy","year":"1973","unstructured":"Murphy AH (1973) A new vector partition of the probability score. J Appl Meteorol Climatol 12(4):595\u2013600","journal-title":"J Appl Meteorol Climatol"},{"issue":"1\u20132","key":"964_CR63","doi-asserted-by":"publisher","first-page":"12","DOI":"10.2307\/2987588","volume":"32","author":"MH DeGroot","year":"1983","unstructured":"DeGroot MH, Fienberg SE (1983) The comparison and evaluation of forecasters. J Royal Stat Soc Series D (The Statistician) 32(1\u20132):12\u201322. https:\/\/doi.org\/10.2307\/2987588","journal-title":"J Royal Stat Soc Series D (The Statistician)"},{"key":"964_CR64","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v29i1.9602","author":"M Pakdaman Naeini","year":"2015","unstructured":"Pakdaman Naeini M, Cooper G, Hauskrecht M (2015) Obtaining well calibrated probabilities using bayesian binning. Proc AAAI Conf Artif Intell. https:\/\/doi.org\/10.1609\/aaai.v29i1.9602","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"964_CR65","unstructured":"Nixon J, Dusenberry MW, Zhang L, Jerfel G, Tran D (2019) Measuring calibration in deep learning. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops"},{"issue":"477","key":"964_CR66","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1198\/016214506000001437","volume":"102","author":"T Gneiting","year":"2007","unstructured":"Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. J Am Stat Assoc 102(477):359\u2013378. https:\/\/doi.org\/10.1198\/016214506000001437","journal-title":"J Am Stat Assoc"},{"issue":"1","key":"964_CR67","doi-asserted-by":"publisher","first-page":"6536","DOI":"10.1038\/s41598-024-57135-6","volume":"14","author":"JP Roth","year":"2024","unstructured":"Roth JP, Bajorath J (2024) Relationship between prediction accuracy and uncertainty in compound potency prediction using deep neural networks and control models. Sci Rep 14(1):6536. https:\/\/doi.org\/10.1038\/s41598-024-57135-6","journal-title":"Sci Rep"},{"key":"964_CR68","first-page":"20063","volume":"34","author":"R Rahaman","year":"2021","unstructured":"Rahaman R et al (2021) Uncertainty quantification and deep ensembles. Adv Neural Infor Proc Syst 34:20063\u201320075","journal-title":"Adv Neural Infor Proc Syst"},{"key":"964_CR69","unstructured":"Popordanoska T, Gruber SG, Tiulpin A, Buettner F, Blaschko MB (2024) Consistent and asymptotically unbiased estimation of proper calibration errors. In: International Conference on Artificial Intelligence and Statistics, pp. 3466\u20133474. PMLR"},{"issue":"643","key":"964_CR70","doi-asserted-by":"publisher","first-page":"1512","DOI":"10.1002\/qj.456","volume":"135","author":"J Br\u00f6cker","year":"2009","unstructured":"Br\u00f6cker J (2009) Reliability, sufficiency, and the decomposition of proper scores. Quart J Royal Meteorol Soc 135(643):1512\u20131519","journal-title":"Quart J Royal Meteorol Soc"},{"key":"964_CR71","first-page":"7933","volume":"35","author":"T Popordanoska","year":"2022","unstructured":"Popordanoska T, Sayer R, Blaschko M (2022) A consistent and differentiable LP canonical calibration error estimator. Adv Neural Infor Proc Syst 35:7933\u20137946","journal-title":"Adv Neural Infor Proc Syst"},{"key":"964_CR72","volume-title":"Pattern Recog Machine Learn","author":"CM Bishop","year":"2006","unstructured":"Bishop CM, Nasrabadi NM (2006) Pattern Recog Machine Learn, vol 4. Springer, ???"}],"container-title":["Journal of Cheminformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13321-025-00964-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13321-025-00964-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13321-025-00964-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,5]],"date-time":"2025-03-05T10:13:43Z","timestamp":1741169623000},"score":1,"resource":{"primary":{"URL":"https:\/\/jcheminf.biomedcentral.com\/articles\/10.1186\/s13321-025-00964-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,5]]},"references-count":72,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["964"],"URL":"https:\/\/doi.org\/10.1186\/s13321-025-00964-y","relation":{},"ISSN":["1758-2946"],"issn-type":[{"value":"1758-2946","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,5]]},"assertion":[{"value":"12 June 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 January 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 March 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"29"}}