{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T15:24:04Z","timestamp":1792596244429,"version":"4.5.0"},"reference-count":122,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T00:00:00Z","timestamp":1792195200000},"content-version":"vor","delay-in-days":289,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,1,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>Effective computational tool development depends on our ability to measure progress with rigorous benchmarking, allowing developers to showcase advances and users to select the best of the available options. Computational biology has developed a robust infrastructure of benchmarking standards, tools\/resources, and best practices to allow for rigorous and unbiased assessment. This infrastructure, however, faces growing challenges as modern artificial intelligence (AI) methods move to the forefront of computational analysis. The field has imperfectly adapted to issues problematic for benchmarking AI methods, including data entanglement, data leakage, problem-specific performance, limited theoretical guarantees, elusive ground-truth, opaque and often closed models that make reproducibility of results difficult, and dependence on impermanent data sources.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>In this paper, we consider how the field should tackle benchmarking in the AI era. We revisit why benchmarking matters and its history in computational biology, examine emerging AI-specific challenges, and outline practical paths forward. We argue that benchmarks should be treated as long-lived community infrastructure rather than one-off artifacts bundled with individual papers. We envision a future of critical assessment i.e. dynamic and responsive to evolving use cases, sustaining progress for advancing science in the age of AI.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability and implementation<\/jats:title>\n                    <jats:p>N\/A<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioadv\/vbag290","type":"journal-article","created":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T11:57:29Z","timestamp":1790769449000},"source":"Crossref","is-referenced-by-count":1,"title":["Critical assessment in the age of AI: challenges and opportunities for computational biology"],"prefix":"10.1093","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4970-2252","authenticated-orcid":false,"given":"Russell","family":"Schwartz","sequence":"first","affiliation":[{"name":"Ray & Stephanie Lane Computational Biology Department, Carnegie Mellon University , Pittsburgh, PA 15213,","place":["United States"]},{"name":"Department of Biological Sciences, Carnegie Mellon University , Pittsburgh, PA 15213,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9684-5643","authenticated-orcid":false,"given":"Xuegong","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Automation\/BNRIST and School of Life Sciences, Tsinghua University , Beijing 100084,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jian","family":"Ma","sequence":"additional","affiliation":[{"name":"Ray & Stephanie Lane Computational Biology Department, Carnegie Mellon University , Pittsburgh, PA 15213,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yana","family":"Bromberg","sequence":"additional","affiliation":[{"name":"Department of Biology and Department of Computer Science, Emory University , Atlanta, GA 30322,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0358-901X","authenticated-orcid":false,"given":"Robert F","family":"Murphy","sequence":"additional","affiliation":[{"name":"Ray & Stephanie Lane Computational Biology Department, Carnegie Mellon University , Pittsburgh, PA 15213,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1455-3370","authenticated-orcid":false,"given":"Lawrence E","family":"Hunter","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, University of Chicago , Chicago, IL 60637,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7872-5074","authenticated-orcid":false,"given":"Laxmi","family":"Parida","sequence":"additional","affiliation":[{"name":"IBM Research , Yorktown Heights, NY 10598,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8552-8976","authenticated-orcid":false,"given":"Julio","family":"Saez-Rodriguez","sequence":"additional","affiliation":[{"name":"European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI) , Hinxton, Cambridgeshire,","place":["United Kingdom"]},{"name":"Faculty of Medicine, and Heidelberg University Hospital, Institute for Computational Biomedicine, Heidelberg University , Heidelberg,","place":["Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gustavo","family":"Stolovitzky","sequence":"additional","affiliation":[{"name":"Department of Pathology, NYU Grossman School of Engineering , New York, NY 10016,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9617-5304","authenticated-orcid":false,"given":"Mihai","family":"Pop","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Maryland , College Park, MD 20742,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8290-813X","authenticated-orcid":false,"given":"Shoba","family":"Ranganathan","sequence":"additional","affiliation":[{"name":"Applied Biosciences, Macquarie University , Sydney, NSW 2109,","place":["Australia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3357-437X","authenticated-orcid":false,"given":"Donna K","family":"Slonim","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Tufts University , Medford, MA 02155,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Terry","family":"Gaasterland","sequence":"additional","affiliation":[{"name":"Bioinformatics & Systems Biology Program, UC San Diego , La Jolla, CA 92093,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,10,16]]},"reference":[{"key":"2026101621370453000_vbag290-B1","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1038\/s41570-022-00363-z","article-title":"CACHE (critical assessment of computational hit-finding experiments): a public-private partnership benchmarking initiative to enable the development of computational methods for hit-finding","volume":"6","author":"Ackloo","year":"2022","journal-title":"Nat Rev Chem"},{"key":"2026101621370453000_vbag290-B34","first-page":"36","article-title":"DARPA\u2019s role in machine learning","volume":"42","author":"Alspector","year":"2020","journal-title":"AI Mag"},{"key":"2026101621370453000_vbag290-B2","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1038\/75556","article-title":"Gene ontology: tool for the unification of biology","volume":"25","author":"Ashburner","year":"2000","journal-title":"Nat Genet"},{"key":"2026101621370453000_vbag290-B3","doi-asserted-by":"publisher","first-page":"1222","DOI":"10.1038\/s41592-023-01909-9","article-title":"MultiVI: deep generative model for the integration of multimodal data","volume":"20","author":"Ashuach","year":"2023","journal-title":"Nat Methods"},{"key":"2026101621370453000_vbag290-B4","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbaf355","article-title":"A technical review of multi-omics data integration methods: from classical statistical to deep generative approaches","volume":"26","author":"Bai\u00e3o","year":"2025","journal-title":"Brief Bioinform"},{"key":"2026101621370453000_vbag290-B5","doi-asserted-by":"publisher","first-page":"412","DOI":"10.1093\/bioinformatics\/16.5.412","article-title":"Assessing the accuracy of prediction algorithms for classification: an overview","volume":"16","author":"Baldi","year":"2000","journal-title":"Bioinformatics"},{"key":"2026101621370453000_vbag290-B6","doi-asserted-by":"publisher","first-page":"D693","DOI":"10.1093\/nar\/gkab1016","article-title":"Rhea, the reaction knowledgebase in 2022","volume":"50","author":"Bansal","year":"2022","journal-title":"Nucleic Acids Res"},{"key":"2026101621370453000_vbag290-B7","author":"BEACON: The Benchmarking, Evaluation, and Assessment Consortium for Science","year":"2026"},{"key":"2026101621370453000_vbag290-B8","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1093\/nar\/28.1.235","article-title":"The protein data bank","volume":"28","author":"Berman","year":"2000","journal-title":"Nucleic Acids Res"},{"key":"2026101621370453000_vbag290-B10","doi-asserted-by":"publisher","first-page":"nwae340","DOI":"10.1093\/nsr\/nwae340","article-title":"General-purpose pre-trained large cellular models for single-cell transcriptomics","volume":"11","author":"Bian","year":"2024","journal-title":"Natl Sci Rev"},{"key":"2026101621370453000_vbag290-B11","doi-asserted-by":"publisher","first-page":"1352","DOI":"10.1038\/s41592-021-01264-7","article-title":"Deep learning and alignment of spatially resolved single-cell transcriptomes with Tangram","volume":"18","author":"Biancalani","year":"2021","journal-title":"Nat Methods"},{"key":"2026101621370453000_vbag290-B12","author":"Blum","year":"2015"},{"key":"2026101621370453000_vbag290-B13","doi-asserted-by":"publisher","first-page":"394","DOI":"10.1038\/s41580-025-00918-0","article-title":"Advancing single-cell omics and cell-based therapeutics with quantum computing","volume":"27","author":"Bose","year":"2026","journal-title":"Nat Rev Mol Cell Biol"},{"key":"2026101621370453000_vbag290-B14","doi-asserted-by":"publisher","first-page":"14255","DOI":"10.1073\/pnas.1216613110","article-title":"Neutral and weakly nonneutral sequence variants may define individuality","volume":"110","author":"Bromberg","year":"2013","journal-title":"Proc Natl Acad Sci USA"},{"key":"2026101621370453000_vbag290-B15","doi-asserted-by":"publisher","first-page":"326","DOI":"10.1038\/s41576-023-00679-6","article-title":"Challenges and best practices in omics benchmarking","volume":"25","author":"Brooks","year":"2024","journal-title":"Nat Rev Genet"},{"key":"2026101621370453000_vbag290-B16","doi-asserted-by":"publisher","first-page":"vbae166","DOI":"10.1093\/bioadv\/vbae166","article-title":"The ISCB competency framework v. 3: a revised and extended standard for bioinformatics education and training","volume":"4","author":"Brooksbank","year":"2024","journal-title":"Bioinform Adv"},{"key":"2026101621370453000_vbag290-B17","doi-asserted-by":"publisher","first-page":"679","DOI":"10.1109\/TPAMI.1986.4767851","article-title":"A computational approach to edge detection","volume":"8","author":"Canny","year":"1986","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2026101621370453000_vbag290-B18","doi-asserted-by":"publisher","first-page":"3856","DOI":"10.1038\/s41467-025-59231-1","article-title":"Science acceleration and accessibility with self-driving labs","volume":"16","author":"Canty","year":"2025","journal-title":"Nat Commun"},{"key":"2026101621370453000_vbag290-B19","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbaf380","article-title":"The current landscape and emerging challenges of benchmarking single-cell methods","volume":"26","author":"Cao","year":"2025","journal-title":"Brief Bioinform"},{"key":"2026101621370453000_vbag290-B20","doi-asserted-by":"publisher","first-page":"4919","DOI":"10.1038\/s41467-022-32550-3","article-title":"Generalization in quantum machine learning from few training data","volume":"13","author":"Caro","year":"2022","journal-title":"Nat Commun"},{"key":"2026101621370453000_vbag290-B21","doi-asserted-by":"publisher","first-page":"2111","DOI":"10.1016\/j.drudis.2019.06.020","article-title":"Target 2035: probing the human proteome","volume":"24","author":"Carter","year":"2019","journal-title":"Drug Discov Today"},{"key":"2026101621370453000_vbag290-B22","doi-asserted-by":"publisher","first-page":"7296","DOI":"10.1038\/s41467-024-51801-z","article-title":"AlphaFold predictions of fold-switched conformations are driven by structure memorization","volume":"15","author":"Chakravarty","year":"2024","journal-title":"Nat Commun"},{"key":"2026101621370453000_vbag290-B23","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1038\/s41467-025-66870-x","article-title":"Rapid estimation of protein folding pathways from sequence alone using AlphaFold2","volume":"17","author":"Chang","year":"2025","journal-title":"Nat Commun"},{"key":"2026101621370453000_vbag290-B24","doi-asserted-by":"publisher","first-page":"jkaa056","DOI":"10.1093\/g3journal\/jkaa056","article-title":"Revisiting the genome-wide significance threshold for common variant GWAS","author":"Chen","year":"2021"},{"key":"2026101621370453000_vbag290-B25","doi-asserted-by":"publisher","first-page":"1454","DOI":"10.1038\/s41592-024-02359-7","article-title":"Applying interpretable machine learning in computational biology-pitfalls, recommendations and opportunities for new developments","volume":"21","author":"Chen","year":"2024","journal-title":"Nat Methods"},{"key":"2026101621370453000_vbag290-B26","doi-asserted-by":"publisher","first-page":"3908","DOI":"10.1016\/j.csbj.2025.08.028","article-title":"TBM preferred to AlphaFold 3 for functional models of insect odorant receptors","volume":"27","author":"Chidambara Thanu","year":"2025","journal-title":"Comput Struct Biotechnol J"},{"key":"2026101621370453000_vbag290-B27","doi-asserted-by":"publisher","first-page":"1030","DOI":"10.12688\/f1000research.7118.1","article-title":"DREAMTools: a python package for scoring collaborative challenges","volume":"4","author":"Cokelaer","year":"2015","journal-title":"F1000Res"},{"key":"2026101621370453000_vbag290-B28","author":"Cormen","year":"2022"},{"key":"2026101621370453000_vbag290-B29","doi-asserted-by":"publisher","first-page":"3021","DOI":"10.1073\/pnas.0830968100","article-title":"Revisiting the independence of the publicly and privately funded drafts of the human genome","volume":"100","author":"Cozzarelli","year":"2003","journal-title":"Proc Natl Acad Sci USA"},{"key":"2026101621370453000_vbag290-B30","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1186\/s13059-023-03113-6","article-title":"CAGI, the critical assessment of genome interpretation, establishes progress and prospects for computational genetic variant interpretation methods","volume":"25","author":"Critical Assessment of Genome Interpretation Consortium","year":"2024","journal-title":"Genome Biol"},{"key":"2026101621370453000_vbag290-B31","doi-asserted-by":"publisher","first-page":"1470","DOI":"10.1038\/s41592-024-02201-0","article-title":"scGPT: toward building a foundation model for single-cell multi-omics using generative AI","volume":"21","author":"Cui","year":"2024","journal-title":"Nat Methods"},{"key":"2026101621370453000_vbag290-B32","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1146\/annurev.biochem.77.062906.171838","article-title":"Macromolecular modeling with rosetta","volume":"77","author":"Das","year":"2008","journal-title":"Annu Rev Biochem"},{"key":"2026101621370453000_vbag290-B33","doi-asserted-by":"publisher","first-page":"831","DOI":"10.1016\/j.str.2004.02.031","article-title":"Heterogeneity and inaccuracy in protein structures solved by X-ray crystallography","volume":"12","author":"DePristo","year":"2004","journal-title":"Structure"},{"key":"2026101621370453000_vbag290-B35","first-page":"287","article-title":"A note on two problems in connexion with graphs","volume-title":"Numer Math","author":"Dijkstra","year":"2022"},{"key":"2026101621370453000_vbag290-B36","doi-asserted-by":"publisher","first-page":"636","DOI":"10.1126\/science.aaa9375","article-title":"STATISTICS. The reusable holdout: preserving validity in adaptive data analysis","volume":"349","author":"Dwork","year":"2015","journal-title":"Science"},{"key":"2026101621370453000_vbag290-B37","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1145\/3051088","article-title":"Guilt-free data reuse","volume":"60","author":"Dwork","year":"2017","journal-title":"Commun ACM"},{"key":"2026101621370453000_vbag290-B38","doi-asserted-by":"publisher","first-page":"634","DOI":"10.1038\/s41570-025-00737-z","article-title":"Protein-ligand data at scale to support machine learning","volume":"9","author":"Edwards","year":"2025","journal-title":"Nat Rev Chem"},{"key":"2026101621370453000_vbag290-B39","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3777382","article-title":"Towards trustworthy AI: a review of ethical and robust large language models","volume":"58","author":"Ferdaus","year":"2026","journal-title":"ACM Comput Surv"},{"key":"2026101621370453000_vbag290-B105","doi-asserted-by":"crossref","first-page":"e13283","DOI":"10.1371\/journal.pone.0013283","article-title":"When the optimal is not the best: parameter estimation in complex biological models","volume":"5","author":"Fernandez Slezak D, Su\u00e1rez C, Cecchi GA","year":"2010","journal-title":"PLoS One"},{"key":"2026101621370453000_vbag290-B40","author":"Gan","year":"2023"},{"key":"2026101621370453000_vbag290-B41","first-page":"29","article-title":"Protein contact prediction from amino acid co-evolution using convolutional networks for graph-valued images","author":"Golkov","year":"2016","journal-title":"Adv Neural Inf Process Syst"},{"key":"2026101621370453000_vbag290-B42","doi-asserted-by":"publisher","first-page":"1481","DOI":"10.1038\/s41592-024-02305-7","article-title":"Large-scale foundation model on single-cell transcriptomics","volume":"21","author":"Hao","year":"2024","journal-title":"Nat Methods"},{"key":"2026101621370453000_vbag290-B43","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1038\/s42003-023-05640-1","article-title":"STEM enables mapping of single-cell and spatial transcriptomics data with transfer learning","volume":"7","author":"Hao","year":"2024","journal-title":"Commun Biol"},{"key":"2026101621370453000_vbag290-B44","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1002\/qub2.65","article-title":"Current opinions on large cellular models","volume":"12","author":"Hao","year":"2024","journal-title":"Quant Biol"},{"key":"2026101621370453000_vbag290-B45","author":"Harman","year":"1992"},{"key":"2026101621370453000_vbag290-B46","doi-asserted-by":"publisher","first-page":"550","DOI":"10.1038\/s41576-023-00586-w","article-title":"Best practices for single-cell analysis across modalities","volume":"24","author":"Heumos","year":"2023","journal-title":"Nat Rev Genet"},{"key":"2026101621370453000_vbag290-B47","doi-asserted-by":"publisher","first-page":"1039","DOI":"10.1002\/humu.23290","article-title":"Reports from CAGI: the critical assessment of genome interpretation","volume":"38","author":"Hoskins","year":"2017","journal-title":"Hum Mutat"},{"key":"2026101621370453000_vbag290-B48","doi-asserted-by":"publisher","first-page":"2182","DOI":"10.1038\/s41592-024-02429-w","article-title":"Benchmarking algorithms for single-cell multi-omics prediction and integration","volume":"21","author":"Hu","year":"2024","journal-title":"Nat Methods"},{"key":"2026101621370453000_vbag290-B49","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1007\/s40484-018-0164-3","article-title":"A case study on the detailed reproducibility of a human cell atlas project","volume":"7","author":"Hua","year":"2019","journal-title":"Quant Biol"},{"key":"2026101621370453000_vbag290-B50","doi-asserted-by":"publisher","first-page":"29084","DOI":"10.1038\/s41598-024-80094-x","article-title":"Template-based modeling of insect odorant receptors outperforms AlphaFold3 for ligand binding predictions","volume":"14","author":"Jabeen","year":"2024","journal-title":"Sci Rep"},{"key":"2026101621370453000_vbag290-B51","doi-asserted-by":"publisher","first-page":"268","DOI":"10.1016\/j.jspi.2004.02.014","article-title":"An efficient algorithm for constructing optimal design of computer experiments","volume":"134","author":"Jin","year":"2005","journal-title":"J Stat Plan Inference"},{"key":"2026101621370453000_vbag290-B52","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1038\/s41586-021-03819-2","article-title":"Highly accurate protein structure prediction with AlphaFold","volume":"596","author":"Jumper","year":"2021","journal-title":"Nature"},{"key":"2026101621370453000_vbag290-B53","doi-asserted-by":"publisher","first-page":"100804","DOI":"10.1016\/j.patter.2023.100804","article-title":"Leakage and the reproducibility crisis in machine-learning-based science","volume":"4","author":"Kapoor","year":"2023","journal-title":"Patterns (N Y)"},{"key":"2026101621370453000_vbag290-B55","doi-asserted-by":"publisher","first-page":"1607","DOI":"10.1002\/prot.26237","article-title":"Critical assessment of methods of protein structure prediction (CASP)-round XIV","volume":"89","author":"Kryshtafovych","year":"2021","journal-title":"Proteins"},{"key":"2026101621370453000_vbag290-B56","author":"Labaj","year":"2023"},{"key":"2026101621370453000_vbag290-B57","author":"Liang","year":"2024"},{"key":"2026101621370453000_vbag290-B58","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1038\/nrg3768","article-title":"Single-cell and multivariate approaches in genetic perturbation screens","volume":"16","author":"Liberali","year":"2015","journal-title":"Nat Rev Genet"},{"key":"2026101621370453000_vbag290-B54","doi-asserted-by":"crossref","first-page":"18","DOI":"10.3390\/e23010018","article-title":"Explainable AI: a review of machine learning interpretability methods","volume":"23","author":"Linardatos P, Papastefanopoulos V, Kotsiantis S.","year":"2020","journal-title":"Entropy"},{"key":"2026101621370453000_vbag290-B59","doi-asserted-by":"publisher","first-page":"849","DOI":"10.1038\/nbt0807-849","article-title":"Novel leverage of structural genomics","volume":"25","author":"Liu","year":"2007","journal-title":"Nat Biotechnol"},{"key":"2026101621370453000_vbag290-B60","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1007\/s44163-024-00149-w","article-title":"Green AI: exploring carbon footprints, mitigation strategies, and trade offs in large language model training","volume":"4","author":"Liu","year":"2024","journal-title":"Discov Artif Intell"},{"key":"2026101621370453000_vbag290-B61","doi-asserted-by":"publisher","first-page":"1162","DOI":"10.1126\/science.252.5009.1162","article-title":"Predicting coiled coils from protein sequences","volume":"252","author":"Lupas","year":"1991","journal-title":"Science"},{"key":"2026101621370453000_vbag290-B62","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1002\/gepi.1370010206","article-title":"Genetic analysis workshop II: summary","volume":"1","author":"Maccluer","year":"1984","journal-title":"Genet Epidemiol"},{"key":"2026101621370453000_vbag290-B63","doi-asserted-by":"publisher","first-page":"1608","DOI":"10.1038\/s41598-017-01054-2","article-title":"Common sequence variants affect molecular function more than rare variants?","volume":"7","author":"Mahlich","year":"2017","journal-title":"Sci Rep"},{"key":"2026101621370453000_vbag290-B64","doi-asserted-by":"publisher","first-page":"e1013658","DOI":"10.1371\/journal.pcbi.1013658","article-title":"Building a continuous benchmarking ecosystem in bioinformatics","volume":"21","author":"Mallona","year":"2025","journal-title":"PLoS Comput Biol"},{"key":"2026101621370453000_vbag290-B65","doi-asserted-by":"publisher","first-page":"1393","DOI":"10.1038\/s41467-019-09406-4","article-title":"Systematic benchmarking of omics computational tools","volume":"10","author":"Mangul","year":"2019","journal-title":"Nat Commun"},{"key":"2026101621370453000_vbag290-B66","doi-asserted-by":"publisher","first-page":"e2002050","DOI":"10.1371\/journal.pbio.2002050","article-title":"All biology is computational biology","volume":"15","author":"Markowetz","year":"2017","journal-title":"PLoS Biol"},{"key":"2026101621370453000_vbag290-B67","doi-asserted-by":"publisher","first-page":"1072","DOI":"10.1038\/nbt.2419","article-title":"Protein structure prediction from sequence variation","volume":"30","author":"Marks","year":"2012","journal-title":"Nat Biotechnol"},{"key":"2026101621370453000_vbag290-B68","first-page":"187","article-title":"Theory of edge detection","volume":"207","author":"Marr","year":"1980","journal-title":"Proc Royal Soc London Ser B. Biol Sci"},{"key":"2026101621370453000_vbag290-B69","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1038\/s41592-022-01431-4","article-title":"Critical assessment of metagenome interpretation: the second round of challenges","volume":"19","author":"Meyer","year":"2022","journal-title":"Nat Methods"},{"key":"2026101621370453000_vbag290-B70","doi-asserted-by":"publisher","first-page":"636","DOI":"10.1016\/j.cels.2021.05.015","article-title":"Advances in systems biology modeling: 10 years of crowdsourcing DREAM challenges","volume":"12","author":"Meyer","year":"2021","journal-title":"Cell Syst"},{"key":"2026101621370453000_vbag290-B71","author":"Meyes","year":"2019"},{"key":"2026101621370453000_vbag290-B72","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1038\/s41586-023-05881-4","article-title":"Foundation models for generalist medical artificial intelligence","volume":"616","author":"Moor","year":"2023","journal-title":"Nature"},{"key":"2026101621370453000_vbag290-B73","volume-title":"The Tyranny of Metrics","author":"Muller","year":"2018"},{"key":"2026101621370453000_vbag290-B74","doi-asserted-by":"publisher","first-page":"256","DOI":"10.1038\/d41586-025-03506-6","article-title":"Major AI conference flooded with peer reviews written fully by AI","volume":"648","author":"Naddaf","year":"2025","journal-title":"Nature"},{"key":"2026101621370453000_vbag290-B75","doi-asserted-by":"publisher","first-page":"e10047","DOI":"10.7554\/eLife.10047","article-title":"Active machine learning-driven experimentation to determine compound effects on protein patterns","volume":"5","author":"Naik","year":"2016","journal-title":"Elife"},{"key":"2026101621370453000_vbag290-B76","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1007\/s10969-008-9055-6","article-title":"Structural genomics is the largest contributor of novel structural leverage","volume":"10","author":"Nair","year":"2009","journal-title":"J Struct Funct Genomics"},{"key":"2026101621370453000_vbag290-B77","author":"National Academies of Sciences, Engineering, and Medicine","year":"2019"},{"key":"2026101621370453000_vbag290-B78","author":"National Research Council (US) Committee on Undergraduate Biology Education to Prepare Research Scientists for the 21st Century","year":"2003"},{"key":"2026101621370453000_vbag290-B79","volume-title":"Human Problem Solving","author":"Newell","year":"1972"},{"key":"2026101621370453000_vbag290-B80","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1002\/(SICI)1097-0134(199602)24:2<165::AID-PROT4>3.0.CO;2-I","article-title":"Defining a similarity threshold for a functional protein sequence pattern: the signal peptide cleavage site","volume":"24","author":"Nielsen","year":"1996","journal-title":"Proteins"},{"key":"2026101621370453000_vbag290-B81","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1038\/msb.2011.70","article-title":"The self-assessment trap: can we all be better than average?","volume":"7","author":"Norel","year":"2011","journal-title":"Mol Syst Biol"},{"key":"2026101621370453000_vbag290-B82","doi-asserted-by":"publisher","first-page":"1519","DOI":"10.1016\/j.str.2007.11.004","article-title":"Update on the protein structure initiative","volume":"15","author":"Norvell","year":"2007","journal-title":"Structure"},{"key":"2026101621370453000_vbag290-B84","doi-asserted-by":"publisher","first-page":"e1006494","DOI":"10.1371\/journal.pcbi.1006494","article-title":"Putting benchmarks in their rightful place: the heart of computational biology","volume":"14","author":"Peters","year":"2018","journal-title":"PLoS Comput Biol"},{"key":"2026101621370453000_vbag290-B85","doi-asserted-by":"publisher","first-page":"vbae043","DOI":"10.1093\/bioadv\/vbae043","article-title":"CAFA-evaluator: a python tool for benchmarking ontological classification methods","volume":"4","author":"Piovesan","year":"2024","journal-title":"Bioinform Adv"},{"key":"2026101621370453000_vbag290-B86","doi-asserted-by":"publisher","first-page":"vbaf044","DOI":"10.1093\/bioadv\/vbaf044","article-title":"Biological databases in the age of generative artificial intelligence","volume":"5","author":"Pop","year":"2025","journal-title":"Bioinform Adv"},{"key":"2026101621370453000_vbag290-B87","doi-asserted-by":"publisher","first-page":"796","DOI":"10.1038\/s41592-026-03028-7","article-title":"Quantifying uncertainty in protein representations across models and tasks","volume":"23","author":"Prabakaran","year":"2026","journal-title":"Nat Methods"},{"key":"2026101621370453000_vbag290-B88","doi-asserted-by":"publisher","first-page":"107702","DOI":"10.1016\/j.jsb.2021.107702","article-title":"3D variability analysis: resolving continuous flexibility and discrete heterogeneity from single particle cryo-EM","volume":"213","author":"Punjani","year":"2021","journal-title":"J Struct Biol"},{"key":"2026101621370453000_vbag290-B89","first-page":"5389","author":"Recht","year":"2019"},{"key":"2026101621370453000_vbag290-B90","author":"Roberts","year":"1963"},{"key":"2026101621370453000_vbag290-B91","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1186\/s13059-019-1846-5","article-title":"Benchmarking comes of age","volume":"20","author":"Robinson","year":"2019","journal-title":"Genome Biol"},{"key":"2026101621370453000_vbag290-B92","doi-asserted-by":"publisher","first-page":"7558","DOI":"10.1073\/pnas.90.16.7558","article-title":"Improved prediction of protein secondary structure by use of sequence profiles and neural networks","volume":"90","author":"Rost","year":"1993","journal-title":"Proc Natl Acad Sci USA"},{"key":"2026101621370453000_vbag290-B93","doi-asserted-by":"publisher","first-page":"470","DOI":"10.1038\/nrg.2016.69","article-title":"Crowdsourcing biomedical research: leveraging communities as innovation engines","volume":"17","author":"Saez-Rodriguez","year":"2016","journal-title":"Nat Rev Genet"},{"key":"2026101621370453000_vbag290-B94","doi-asserted-by":"publisher","first-page":"1724","DOI":"10.1038\/s41592-026-03182-y","article-title":"Benchmarking biomedical foundation models","volume":"23","author":"Saez-Rodriguez","year":"2026","journal-title":"Nat Methods"},{"key":"2026101621370453000_vbag290-B95","author":"Sage Bionetworks. First DREAM Target 2035 Drug Discovery Challenge"},{"key":"2026101621370453000_vbag290-B96","doi-asserted-by":"publisher","first-page":"248","DOI":"10.1038\/369248a0","article-title":"How does a protein fold?","volume":"369","author":"Sali","year":"1994","journal-title":"Nature"},{"key":"2026101621370453000_vbag290-B97","author":"Sch\u00e4fer","year":"2026"},{"key":"2026101621370453000_vbag290-B98","first-page":"1031","author":"Schummer","year":"2009"},{"key":"2026101621370453000_vbag290-B99","doi-asserted-by":"publisher","first-page":"1063","DOI":"10.1038\/nmeth.4458","article-title":"Critical assessment of metagenome interpretation\u2014a benchmark of metagenomics software","volume":"14","author":"Sczyrba","year":"2017","journal-title":"Nat Methods"},{"key":"2026101621370453000_vbag290-B100","author":"Senoner","year":"2025"},{"key":"2026101621370453000_vbag290-B101","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1038\/nsb0295-91","article-title":"Protein fold recognition","volume":"2","author":"Shortle","year":"1995","journal-title":"Nat Struct Biol"},{"key":"2026101621370453000_vbag290-B102","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbaf724","article-title":"A comprehensive survey of genome language models in bioinformatics","volume":"27","author":"Shu","year":"2026","journal-title":"Brief Bioinform"},{"key":"2026101621370453000_vbag290-B103","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1002\/(SICI)1097-0134(1999)37:3+<171::AID-PROT21>3.0.CO;2-Z","article-title":"Ab initio protein structure prediction of CASP III targets using ROSETTA","volume":"37","author":"Simons","year":"1999","journal-title":"Proteins"},{"key":"2026101621370453000_vbag290-B104","doi-asserted-by":"publisher","first-page":"2","DOI":"10.5642\/jhummath.201101.03","article-title":"Aesthetic considerations in mathematics","volume":"1","author":"Sinclair","year":"2011","journal-title":"J. Humanist. Math"},{"key":"2026101621370453000_vbag290-B106","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1759-4499-2-1","article-title":"Towards robot scientists for autonomous scientific discovery","volume":"2","author":"Sparkes","year":"2010","journal-title":"Autom Exp"},{"key":"2026101621370453000_vbag290-B107","doi-asserted-by":"publisher","first-page":"vbae193","DOI":"10.1093\/bioadv\/vbae193","article-title":"Optimizing design of genomics studies for clonal evolution analysis","volume":"4","author":"Srivatsa","year":"2024","journal-title":"Bioinform Adv"},{"key":"2026101621370453000_vbag290-B108","author":"Stapenhurst","year":"2017"},{"key":"2026101621370453000_vbag290-B109","doi-asserted-by":"publisher","first-page":"1078","DOI":"10.1107\/s0907444998009378","article-title":"Protein data bank (PDB): database of three-dimensional structural information of biological macromolecules","volume":"54","author":"Sussman","year":"1998","journal-title":"Acta Crystallogr D Biol Crystallogr"},{"key":"2026101621370453000_vbag290-B110","doi-asserted-by":"publisher","first-page":"3092","DOI":"10.1038\/s41467-020-16937-8","article-title":"Harmonization of quality metrics and power calculation in multi-omic studies","volume":"11","author":"Tarazona","year":"2020","journal-title":"Nat Commun"},{"key":"2026101621370453000_vbag290-B111","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1002\/qub2.68","article-title":"Perspectives on benchmarking foundation models for network biology","volume":"12","author":"Theodoris","year":"2024","journal-title":"Quant Biol"},{"key":"2026101621370453000_vbag290-B112","doi-asserted-by":"publisher","first-page":"616","DOI":"10.1038\/s41586-023-06139-9","article-title":"Transfer learning enables predictions in network biology","volume":"618","author":"Theodoris","year":"2023","journal-title":"Nature"},{"key":"2026101621370453000_vbag290-B113","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1093\/bioinformatics\/16.1.34","article-title":"History of the enzyme nomenclature system","volume":"16","author":"Tipton","year":"2000","journal-title":"Bioinformatics"},{"key":"2026101621370453000_vbag290-B114","doi-asserted-by":"publisher","first-page":"1304","DOI":"10.1126\/science.1058040","article-title":"The sequence of the human genome","volume":"291","author":"Venter","year":"2001","journal-title":"Science"},{"key":"2026101621370453000_vbag290-B116","doi-asserted-by":"publisher","first-page":"401","DOI":"10.1101\/gr.7.5.401","article-title":"Human whole-genome shotgun sequencing","volume":"7","author":"Weber","year":"1997","journal-title":"Genome Res"},{"key":"2026101621370453000_vbag290-B117","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1186\/s13059-019-1738-8","article-title":"Essential guidelines for computational method benchmarking","volume":"20","author":"Weber","year":"2019","journal-title":"Genome Biol"},{"key":"2026101621370453000_vbag290-B118","doi-asserted-by":"publisher","first-page":"e1003496","DOI":"10.1371\/journal.pcbi.1003496","article-title":"Bioinformatics curriculum guidelines: toward a definition of core competencies","volume":"10","author":"Welch","year":"2014","journal-title":"PLoS Comput Biol"},{"key":"2026101621370453000_vbag290-B119","doi-asserted-by":"publisher","first-page":"e0224288","DOI":"10.1371\/journal.pone.0224288","article-title":"Barriers to integration of bioinformatics into undergraduate life sciences education: a national study of US life sciences faculty uncover significant barriers to integrating bioinformatics into undergraduate instruction","volume":"14","author":"Williams","year":"2019","journal-title":"PLoS One"},{"key":"2026101621370453000_vbag290-B120","doi-asserted-by":"publisher","first-page":"S1","DOI":"10.1002\/gepi.20274","article-title":"Introduction to genetic analysis workshop 15 summaries","volume":"31 Suppl 1","author":"Witte","year":"2007","journal-title":"Genet Epidemiol"},{"key":"2026101621370453000_vbag290-B121","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1146\/annurev-biophys-102622-084607","article-title":"Critical assessment of methods for predicting the 3D structure of proteins and protein complexes","volume":"52","author":"Wodak","year":"2023","journal-title":"Annu Rev Biophys"},{"key":"2026101621370453000_vbag290-B122","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbae095","article-title":"Benchmarking multi-omics integration algorithms across single-cell RNA and ATAC data","volume":"25","author":"Xiao","year":"2024","journal-title":"Brief Bioinform"},{"key":"2026101621370453000_vbag290-B123","doi-asserted-by":"publisher","first-page":"12673","DOI":"10.1093\/nar\/gkab1159","article-title":"Decoding the effects of synonymous variants","volume":"49","author":"Zeng","year":"2021","journal-title":"Nucleic Acids Res"},{"key":"2026101621370453000_vbag290-B124","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1002\/prot.70068","article-title":"Assessment of protein complex predictions in CASP16: are We making progress?","volume":"94","author":"Zhang","year":"2026","journal-title":"Proteins"},{"key":"2026101621370453000_vbag290-B125","doi-asserted-by":"publisher","first-page":"244","DOI":"10.1186\/s13059-019-1835-8","article-title":"The CAFA challenge reports improved protein function prediction and new functional annotations for hundreds of genes through experimental screens","volume":"20","author":"Zhou","year":"2019","journal-title":"Genome Biol"}],"container-title":["Bioinformatics Advances"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformaticsadvances\/article-pdf\/6\/1\/vbag290\/72052670\/vbag290.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformaticsadvances\/article-pdf\/6\/1\/vbag290\/72052670\/vbag290.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T01:37:07Z","timestamp":1792201027000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformaticsadvances\/article\/doi\/10.1093\/bioadv\/vbag290\/8931828"}},"subtitle":[],"editor":[{"given":"Thomas","family":"Lengauer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026]]},"references-count":122,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1,7]]}},"URL":"https:\/\/doi.org\/10.1093\/bioadv\/vbag290","relation":{},"ISSN":["2635-0041"],"issn-type":[{"value":"2635-0041","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026]]},"published":{"date-parts":[[2026]]},"article-number":"vbag290"}}