{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,4]],"date-time":"2026-08-04T05:58:18Z","timestamp":1785823098262,"version":"3.56.0"},"reference-count":53,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2025,1,28]],"date-time":"2025-01-28T00:00:00Z","timestamp":1738022400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/S024093\/1"],"award-info":[{"award-number":["EP\/S024093\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,2,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>Machine learning-based scoring functions (MLBSFs) have been found to exhibit inconsistent performance on different benchmarks and be prone to learning dataset bias. For the field to develop MLBSFs that learn a generalizable understanding of physics, a more rigorous understanding of how they perform is required.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>In this work, we compared the performance of a diverse set of popular MLBSFs (RFScore, SIGN, OnionNet-2, Pafnucy, and PointVS) to our proposed baseline models that can only learn dataset biases on a range of benchmarks. We found that these baseline models were competitive in accuracy to these MLBSFs in almost all proposed benchmarks, indicating these models only learn dataset biases. Our tests and provided platform, ToolBoxSF, will enable researchers to robustly interrogate MLBSF performance and determine the effect of dataset biases on their predictions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability and implementation<\/jats:title>\n                    <jats:p>https:\/\/github.com\/guydurant\/toolboxsf.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btaf040","type":"journal-article","created":{"date-parts":[[2025,1,24]],"date-time":"2025-01-24T15:20:16Z","timestamp":1737732016000},"source":"Crossref","is-referenced-by-count":16,"title":["Robustly interrogating machine learning-based scoring functions: what are they learning?"],"prefix":"10.1093","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-3249-3793","authenticated-orcid":false,"given":"Guy","family":"Durant","sequence":"first","affiliation":[{"name":"Department of Statistics, University of Oxford, St Giles' , Oxford OX1 3LB,","place":["United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4185-1229","authenticated-orcid":false,"given":"Fergus","family":"Boyles","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of Oxford, St Giles' , Oxford OX1 3LB,","place":["United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9992-2555","authenticated-orcid":false,"given":"Kristian","family":"Birchall","sequence":"additional","affiliation":[{"name":"LifeArc , Stevenage SG1 2FX,","place":["United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1937-4091","authenticated-orcid":false,"given":"Brian","family":"Marsden","sequence":"additional","affiliation":[{"name":"Nuffield Department of Medicine, University of Oxford , Oxford OX3 7BN,","place":["United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1388-2252","authenticated-orcid":false,"given":"Charlotte M","family":"Deane","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of Oxford, St Giles' , Oxford OX1 3LB,","place":["United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,1,28]]},"reference":[{"key":"2025030422271853600_btaf040-B1","doi-asserted-by":"crossref","first-page":"1169","DOI":"10.1093\/bioinformatics\/btq112","article-title":"A machine learning approach to predicting protein\u2013ligand binding affinity with applications to molecular docking","volume":"26","author":"Ballester","year":"2010","journal-title":"Bioinformatics"},{"key":"2025030422271853600_btaf040-B2","doi-asserted-by":"crossref","first-page":"944","DOI":"10.1021\/ci500091r","article-title":"Does a more precise chemical description of protein\u2013ligand complexes lead to more accurate prediction of binding affinity?","volume":"54","author":"Ballester","year":"2014","journal-title":"J Chem Inf Model"},{"key":"2025030422271853600_btaf040-B3","doi-asserted-by":"crossref","first-page":"4759","DOI":"10.1021\/jm001044l","article-title":"Protein-based virtual screening of chemical databases. 1. Evaluation of different docking\/scoring combinations","volume":"43","author":"Bissantz","year":"2000","journal-title":"J Med Chem"},{"key":"2025030422271853600_btaf040-B4","doi-asserted-by":"crossref","first-page":"758","DOI":"10.1093\/bioinformatics\/btz665","article-title":"Learning from the ligand: using ligand-based features to improve binding affinity prediction","volume":"36","author":"Boyles","year":"2020","journal-title":"Bioinformatics"},{"key":"2025030422271853600_btaf040-B5","doi-asserted-by":"crossref","first-page":"5329","DOI":"10.1021\/acs.jcim.1c00096","article-title":"Learning from docked ligands: ligand-based features rescue structure-based scoring functions when trained on docked poses","volume":"62","author":"Boyles","year":"2022","journal-title":"J Chem Inf Model"},{"key":"2025030422271853600_btaf040-B6","doi-asserted-by":"crossref","first-page":"420","DOI":"10.1016\/j.drudis.2009.01.012","article-title":"Healthy skepticism: assessing realistic model performance","volume":"14","author":"Brown","year":"2009","journal-title":"Drug Discov Today"},{"key":"2025030422271853600_btaf040-B7","doi-asserted-by":"crossref","first-page":"1063","DOI":"10.1021\/acs.jcim.5b00523","article-title":"CSAR 2014: a benchmark exercise using unpublished data from pharma","volume":"56","author":"Carlson","year":"2016","journal-title":"J Chem Inf Model"},{"key":"2025030422271853600_btaf040-B8","doi-asserted-by":"crossref","first-page":"e11955","DOI":"10.1371\/journal.pone.0011955","article-title":"Virtual screening for HIV protease inhibitors: a comparison of autodock 4 and vina","volume":"5","author":"Chang","year":"2010","journal-title":"PLoS One"},{"key":"2025030422271853600_btaf040-B9","doi-asserted-by":"crossref","first-page":"2897","DOI":"10.1021\/ci2003889","article-title":"NNScore 2.0: a neural-network receptor-ligand scoring function","volume":"51","author":"Durrant","year":"2011","journal-title":"J Chem Inf Model"},{"key":"2025030422271853600_btaf040-B10","doi-asserted-by":"crossref","first-page":"D222","DOI":"10.1093\/nar\/gkt1223","article-title":"Pfam: the protein families database","volume":"42","author":"Finn","year":"2014","journal-title":"Nucleic Acids Res"},{"key":"2025030422271853600_btaf040-B11","doi-asserted-by":"crossref","first-page":"4200","DOI":"10.1021\/acs.jcim.0c00411","article-title":"Three-dimensional convolutional neural networks and a crossdocked data set for structure-based drug design","volume":"60","author":"Francoeur","year":"2020","journal-title":"J Chem Inf Model"},{"key":"2025030422271853600_btaf040-B12","doi-asserted-by":"crossref","first-page":"1739","DOI":"10.1021\/jm0306430","article-title":"Glide: a new approach for rapid, accurate docking and scoring. 1. Method and assessment of docking accuracy","volume":"47","author":"Friesner","year":"2004","journal-title":"J Med Chem"},{"key":"2025030422271853600_btaf040-B13","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1006\/jmbi.1999.3371","article-title":"Knowledge-based scoring function to predict protein\u2013ligand interactions","volume":"295","author":"Gohlke","year":"2000","journal-title":"J Mol Biol"},{"key":"2025030422271853600_btaf040-B14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/(SICI)1099-1352(199601)9:1<1::AID-JMR241>3.0.CO;2-6","article-title":"Automated docking of flexible ligands: applications of AutoDock","volume":"9","author":"Goodsell","year":"1996","journal-title":"J Mol Recognit"},{"key":"2025030422271853600_btaf040-B15","doi-asserted-by":"crossref","first-page":"114382","DOI":"10.1016\/j.ejmech.2022.114382","article-title":"Discovery of novel non-steroidal selective glucocorticoid receptor modulators by structure- and ign-based virtual screening, structural optimization, and biological evaluation","volume":"237","author":"Hu","year":"2022","journal-title":"Eur J Med Chem"},{"key":"2025030422271853600_btaf040-B16","doi-asserted-by":"crossref","first-page":"5166","DOI":"10.1039\/B608269F","article-title":"Molecular mechanics methods for predicting protein\u2013ligand binding","author":"Huang","year":"2006","journal-title":"Article Phys Chem Chem Phys"},{"key":"2025030422271853600_btaf040-B17","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1021\/acs.jcim.7b00650","article-title":"KDEEP: protein\u2013ligand absolute binding affinity prediction via 3D-Convolutional neural networks","volume":"58","author":"Jim\u00e9nez","year":"2018","journal-title":"J Chem Inf Model"},{"key":"2025030422271853600_btaf040-B18","doi-asserted-by":"crossref","first-page":"e61007","DOI":"10.1371\/journal.pone.0061007","article-title":"Comparability of mixed IC50 data\u2014a statistical analysis","volume":"8","author":"Kalliokoski","year":"2013","journal-title":"PLoS One"},{"key":"2025030422271853600_btaf040-B19","doi-asserted-by":"crossref","first-page":"5150","DOI":"10.1021\/acsomega.9b04162","article-title":"GraphDelta: MPNN scoring function for the affinity prediction of protein\u2013ligand complexes","volume":"5","author":"Karlov","year":"2020","journal-title":"ACS Omega"},{"key":"2025030422271853600_btaf040-B20","doi-asserted-by":"crossref","first-page":"1893","DOI":"10.1021\/ci300604z","article-title":"Lessons learned in empirical scoring with smina from the CSAR 2011 benchmarking exercise","volume":"53","author":"Koes","year":"2013","journal-title":"J Chem Inf Model"},{"key":"2025030422271853600_btaf040-B21","doi-asserted-by":"crossref","first-page":"1961","DOI":"10.1021\/ci100264e","article-title":"Leave-cluster-out cross-validation is appropriate for scoring functions derived from diverse protein data sets","volume":"50","author":"Kramer","year":"2010","journal-title":"J Chem Inf Model"},{"key":"2025030422271853600_btaf040-B22","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1016\/j.jmgm.2004.11.007","article-title":"LigScore: a novel scoring function for predicting binding affinities","volume":"23","author":"Krammer","year":"2005","journal-title":"J Mol Graph Model"},{"key":"2025030422271853600_btaf040-B23","author":"Landrum","year":"2023"},{"key":"2025030422271853600_btaf040-B24","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1002\/minf.201400132","article-title":"Improving AutoDock vina using Random Forest: the growing accuracy of binding affinity prediction by the effective exploitation of larger data sets","volume":"34","author":"Li","year":"2015","journal-title":"Mol Inform"},{"key":"2025030422271853600_btaf040-B25","first-page":"975","article-title":"Structure-aware interactive graph neural networks for the prediction of protein\u2013ligand binding affinity; structure-aware interactive graph neural networks for the prediction of protein\u2013ligand binding affinity","volume":"21","author":"Li","year":"2021","journal-title":"KDD"},{"key":"2025030422271853600_btaf040-B26","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1093\/bioinformatics\/btu626","article-title":"PDB-wide collection of binding data: current status of the pdbbind database","volume":"31","author":"Liu","year":"2014","journal-title":"Bioinformatics"},{"key":"2025030422271853600_btaf040-B27","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1016\/j.str.2009.11.012","article-title":"The structural basis of peptide\u2013protein binding strategies","volume":"18","author":"London","year":"2010","journal-title":"Structure"},{"key":"2025030422271853600_btaf040-B28","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1186\/s13321-021-00522-2","article-title":"GNINA 1.0: molecular docking with deep learning","volume":"13","author":"McNutt","year":"2021","journal-title":"J Cheminform"},{"key":"2025030422271853600_btaf040-B29","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1186\/s13321-021-00536-w","article-title":"Learning protein\u2013ligand binding affinity with atomic environment vectors","volume":"13","author":"Meli","year":"2021","journal-title":"J Cheminform"},{"key":"2025030422271853600_btaf040-B30","doi-asserted-by":"crossref","first-page":"885983","DOI":"10.3389\/fbinf.2022.885983","article-title":"Scoring functions for protein\u2013ligand binding affinity prediction using structure-based deep learning: a review","volume":"2","author":"Meli","year":"2022","journal-title":"Front Bioinform"},{"key":"2025030422271853600_btaf040-B31","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1146\/annurev-biophys-070816-033654","article-title":"Predicting binding free energies: frontiers and benchmarks","volume":"46","author":"Mobley","year":"2017","journal-title":"Annu Rev Biophys"},{"key":"2025030422271853600_btaf040-B32","doi-asserted-by":"crossref","first-page":"3661","DOI":"10.1039\/D1SC06946B","article-title":"PIGNet: a physics-informed deep learning model toward generalized drug\u2013target interaction predictions","volume":"13","author":"Moon","year":"2022","journal-title":"Chem Sci"},{"key":"2025030422271853600_btaf040-B33","doi-asserted-by":"crossref","first-page":"2785","DOI":"10.1002\/jcc.21256","article-title":"AutoDock4 and AutoDockTools4: automated docking with selective receptor flexibility","volume":"30","author":"Morris","year":"2009","journal-title":"J Comput Chem"},{"key":"2025030422271853600_btaf040-B34","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1186\/1758-2946-3-33","article-title":"Open babel: an open chemical toolbox","volume":"3","author":"O'Boyle","year":"2011","journal-title":"J Cheminform"},{"key":"2025030422271853600_btaf040-B35","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1186\/1758-2946-4-39","article-title":"InChIKey collision resistance: an experimental testing","volume":"4","author":"Pletnev","year":"2012","journal-title":"J Cheminform"},{"key":"2025030422271853600_btaf040-B36","doi-asserted-by":"crossref","first-page":"2562","DOI":"10.1021\/acs.jcim.5b00654","article-title":"Better informed distance geometry: using what We know to improve conformation generation","volume":"55","author":"Riniker","year":"2015","journal-title":"J Chem Inf Model"},{"key":"2025030422271853600_btaf040-B37","first-page":"2960","author":"Scantlebury","year":"2023"},{"key":"2025030422271853600_btaf040-B38","doi-asserted-by":"crossref","first-page":"105920","DOI":"10.1016\/j.isci.2022.105920","article-title":"How good are AlphaFold models for docking-based virtual screening?","volume":"26","author":"Scardino","year":"2023","journal-title":"iScience"},{"key":"2025030422271853600_btaf040-B39","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1186\/s13321-021-00560-w","article-title":"The impact of cross-docked poses on performance of machine learning classifier for protein\u2013ligand binding pose prediction","volume":"13","author":"Shen","year":"2021","journal-title":"J Cheminform"},{"key":"2025030422271853600_btaf040-B40","doi-asserted-by":"crossref","first-page":"947","DOI":"10.1021\/acs.jcim.8b00712","article-title":"In need of bias control: evaluating chemical data for machine learning in structure-based virtual screening","volume":"59","author":"Sieg","year":"2019","journal-title":"J Chem Inf Model"},{"key":"2025030422271853600_btaf040-B41","doi-asserted-by":"crossref","first-page":"3666","DOI":"10.1093\/bioinformatics\/bty374","article-title":"Development and evaluation of a deep learning model for protein\u2013ligand binding affinity prediction","volume":"34","author":"Stepniewska-Dziubinska","year":"2018","journal-title":"Bioinformatics"},{"key":"2025030422271853600_btaf040-B42","doi-asserted-by":"crossref","first-page":"895","DOI":"10.1021\/acs.jcim.8b00545","article-title":"Comparative assessment of scoring functions: the CASF-2016 update","volume":"59","author":"Su","year":"2019","journal-title":"J Chem Inf Model"},{"key":"2025030422271853600_btaf040-B43","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1002\/jcc.21334","article-title":"AutoDock vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization and multithreading","volume":"31","author":"Trott","year":"2010","journal-title":"J Comput Chem"},{"key":"2025030422271853600_btaf040-B44","doi-asserted-by":"crossref","first-page":"609","DOI":"10.1002\/prot.10465","article-title":"Improved protein\u2013ligand docking using GOLD","volume":"52","author":"Verdonk","year":"2003","journal-title":"Proteins"},{"key":"2025030422271853600_btaf040-B45","doi-asserted-by":"crossref","first-page":"7946","DOI":"10.1021\/acs.jmedchem.2c00487","article-title":"On the frustration to predict binding affinities from protein\u2013ligand structures with deep neural networks","volume":"65","author":"Volkov","year":"2022","journal-title":"J Med Chem"},{"key":"2025030422271853600_btaf040-B46","doi-asserted-by":"crossref","first-page":"916","DOI":"10.1021\/acs.jcim.7b00403","article-title":"Most ligand-based classification benchmarks reward memorization rather than generalization","volume":"58","author":"Wallach","year":"2018","journal-title":"J Chem Inf Model"},{"key":"2025030422271853600_btaf040-B47","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1002\/jcc.24667","article-title":"Improving Scoring-Docking-Screening powers of protein\u2013ligand scoring functions using Random Forest","volume":"38","author":"Wang","year":"2017","journal-title":"J Comput Chem"},{"key":"2025030422271853600_btaf040-B48","first-page":"434","volume-title":"Proc Mach Learn Syst","author":"Wang","year":"2021"},{"key":"2025030422271853600_btaf040-B49","doi-asserted-by":"crossref","first-page":"2977","DOI":"10.1021\/jm030580l","article-title":"The PDBbind database: collection of binding affinities for protein\u2013ligand complexes with known three-dimensional structures","volume":"47","author":"Wang","year":"2004","journal-title":"J Med Chem"},{"key":"2025030422271853600_btaf040-B50","doi-asserted-by":"crossref","first-page":"753002","DOI":"10.3389\/fchem.2021.753002","article-title":"OnionNet-2: a convolutional neural network model for predicting protein\u2013ligand binding affinity based on residue-atom contacting shells","volume":"9","author":"Wang","year":"2021","journal-title":"Front Chem"},{"key":"2025030422271853600_btaf040-B51","doi-asserted-by":"crossref","first-page":"e11081","DOI":"10.15252\/msb.202211081","article-title":"Benchmarking AlphaFold-enabled molecular docking predictions for antibiotic discovery","volume":"18","author":"Wong","year":"2022","journal-title":"Mol Syst Biol"},{"key":"2025030422271853600_btaf040-B52","doi-asserted-by":"crossref","first-page":"5485","DOI":"10.1021\/acs.jcim.2c01149","article-title":"Assessment of the generalization abilities of machine-learning scoring functions for structure-based virtual screening","volume":"62","author":"Zhu","year":"2022","journal-title":"J Chem Inf Model"},{"key":"2025030422271853600_btaf040-B53","doi-asserted-by":"crossref","first-page":"1923","DOI":"10.1021\/ci400120b","article-title":"SFCscore(RF): a random forest-based scoring function for improved affinity prediction of protein\u2013ligand complexes","volume":"53","author":"Zilian","year":"2013","journal-title":"J Chem Inf Model"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btaf040\/61664395\/btaf040.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/41\/2\/btaf040\/61664395\/btaf040.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/41\/2\/btaf040\/61664395\/btaf040.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,4]],"date-time":"2025-03-04T17:27:39Z","timestamp":1741109259000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/doi\/10.1093\/bioinformatics\/btaf040\/7985708"}},"subtitle":[],"editor":[{"given":"Janet","family":"Kelso","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2025,1,28]]},"references-count":53,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,2,4]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btaf040","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2023.10.30.564251","asserted-by":"object"}]},"ISSN":["1367-4811"],"issn-type":[{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,2]]},"published":{"date-parts":[[2025,1,28]]},"article-number":"btaf040"}}