{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T17:53:06Z","timestamp":1771523586307,"version":"3.50.1"},"reference-count":21,"publisher":"Oxford University Press (OUP)","issue":"24","license":[{"start":{"date-parts":[[2021,1,25]],"date-time":"2021-01-25T00:00:00Z","timestamp":1611532800000},"content-version":"vor","delay-in-days":41,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Ministry for Education, University and Research","award":["D15D18000410001"],"award-info":[{"award-number":["D15D18000410001"]}]},{"DOI":"10.13039\/501100003407","name":"Italian Ministry of Education","doi-asserted-by":"crossref","award":["MIUR-PRIN-201744NR8S"],"award-info":[{"award-number":["MIUR-PRIN-201744NR8S"]}],"id":[{"id":"10.13039\/501100003407","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,4,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Summary<\/jats:title>\n                  <jats:p>Identifying pathogenic variants and annotating them is a major challenge in human genetics, especially for the non-coding ones. Several tools have been developed and used to predict the functional effect of genetic variants. However, the calibration assessment of the predictions has received little attention. Calibration refers to the idea that if a model predicts a group of variants to be pathogenic with a probability P, it is expected that the same fraction P of true positive is found in the observed set. For instance, a well-calibrated classifier should label the variants such that among the ones to which it gave a probability value close to 0.7, approximately 70% actually belong to the pathogenic class. Poorly calibrated algorithms can be misleading and potentially harmful for clinical decision making.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Avaliability and implementation<\/jats:title>\n                  <jats:p>The dataset used for testing the methods is available through the DOI:10.5281\/zenodo.4448197.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btaa943","type":"journal-article","created":{"date-parts":[[2020,10,28]],"date-time":"2020-10-28T12:11:52Z","timestamp":1603887112000},"page":"5709-5711","source":"Crossref","is-referenced-by-count":13,"title":["Calibrating variant-scoring methods for clinical decision making"],"prefix":"10.1093","volume":"36","author":[{"given":"Silvia","family":"Benevenuta","sequence":"first","affiliation":[{"name":"Department of Medical Sciences, University of Torino , 10126 Torino, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2323-0963","authenticated-orcid":false,"given":"Emidio","family":"Capriotti","sequence":"additional","affiliation":[{"name":"BioFolD Unit, Department of Pharmacy and Biotechnology (FaBiT), University of Bologna , 40126 Bologna, Italy"}]},{"given":"Piero","family":"Fariselli","sequence":"additional","affiliation":[{"name":"Department of Medical Sciences, University of Torino , 10126 Torino, Italy"}]}],"member":"286","published-online":{"date-parts":[[2021,1,25]]},"reference":[{"key":"2023062408142288600_btaa943-B1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1175\/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2","article-title":"Verification of forecasts expressed in terms of probability","volume":"78","author":"Brier","year":"1950","journal-title":"Mon. Weather Rev"},{"key":"2023062408142288600_btaa943-B2","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1093\/bib\/bbr070","article-title":"Bioinformatics for personal genome interpretation","volume":"13","author":"Capriotti","year":"2012","journal-title":"Brief. Bioinform"},{"key":"2023062408142288600_btaa943-B3","doi-asserted-by":"crossref","first-page":"e1443","DOI":"10.1002\/wsbm.1443","article-title":"Integrating molecular networks with genetic variant interpretation for precision medicine","volume":"11","author":"Capriotti","year":"2019","journal-title":"Wiley Interdiscip. Rev. Syst. Biol. Med"},{"key":"2023062408142288600_btaa943-B4","doi-asserted-by":"crossref","first-page":"W247","DOI":"10.1093\/nar\/gkx369","article-title":"PhD-SNPg: a webserver and lightweight tool for scoring single nucleotide variants","volume":"45","author":"Capriotti","year":"2017","journal-title":"Nucleic Acids Res"},{"key":"2023062408142288600_btaa943-B5","doi-asserted-by":"crossref","first-page":"970","DOI":"10.1093\/bib\/bbz047","article-title":"Comparison and integration of computational methods for deleterious synonymous mutation prediction","volume":"21","author":"Cheng","year":"2020","journal-title":"Briefings in Bioinformatics"},{"key":"2023062408142288600_btaa943-B6","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.jclinepi.2019.02.004","article-title":"A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models","volume":"110","author":"Christodoulou","year":"2019","journal-title":"J. Clin. Epidemiol"},{"key":"2023062408142288600_btaa943-B7","doi-asserted-by":"crossref","first-page":"1142","DOI":"10.1016\/j.eururo.2014.11.025","article-title":"Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement","volume":"67","author":"Collins","year":"2015","journal-title":"Eur. Urol"},{"key":"2023062408142288600_btaa943-B8","doi-asserted-by":"crossref","first-page":"1635","DOI":"10.1093\/bioinformatics\/bty008","article-title":"A benchmark study of scoring methods for non-coding mutations","volume":"34","author":"Drubay","year":"2018","journal-title":"Bioinformatics"},{"key":"2023062408142288600_btaa943-B9","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1038\/ng.3477","article-title":"A spectral approach integrating functional genomic annotations for coding and noncoding variants","volume":"48","author":"Ionita-Laza","year":"2016","journal-title":"Nat. Genet"},{"key":"2023062408142288600_btaa943-B10","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1038\/ng.2892","article-title":"A general framework for estimating the relative pathogenicity of human genetic variants","volume":"46","author":"Kircher","year":"2014","journal-title":"Nat. Genet"},{"key":"2023062408142288600_btaa943-B11","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.cell.2019.02.032","article-title":"Genomic analysis in the age of human genome sequencing","volume":"177","author":"Lappalainen","year":"2019","journal-title":"Cell"},{"key":"2023062408142288600_btaa943-B12","first-page":"625","author":"Niculescu-Mizil","year":"2005"},{"key":"2023062408142288600_btaa943-B13","first-page":"2825","article-title":"Scikit-learn: machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"JMLR"},{"key":"2023062408142288600_btaa943-B14","doi-asserted-by":"crossref","first-page":"761","DOI":"10.1093\/bioinformatics\/btu703","article-title":"DANN: a deep learning approach for annotating the pathogenicity of genetic variants","volume":"31","author":"Quang","year":"2015","journal-title":"Bioinformatics"},{"key":"2023062408142288600_btaa943-B15","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1001\/jama.2018.5602","article-title":"Big data and predictive analytics: recalibrating expectations","volume":"320","author":"Shah","year":"2018","journal-title":"JAMA"},{"key":"2023062408142288600_btaa943-B16","doi-asserted-by":"crossref","first-page":"1536","DOI":"10.1093\/bioinformatics\/btv009","article-title":"An integrative approach to predicting the functional effects of non-coding and coding sequence variation","volume":"31","author":"Shihab","year":"2015","journal-title":"Bioinformatics"},{"key":"2023062408142288600_btaa943-B011590","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1177\/0272989X14547233","article-title":"Calibration of risk prediction models: impact on decision-analytic performance","volume":"35","author":"Van Calster","year":"2015","journal-title":"Medical Decision Making: An International Journal of the Society for Medical Decision Making"},{"key":"2023062408142288600_btaa943-B2179587","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1016\/j.jclinepi.2015.12.005","article-title":"A calibration hierarchy for risk models was defined: from utopia to empirical data","volume":"74","author":"Van Calster","year":"2016","journal-title":"Journal of Clinical Epidemiology"},{"key":"2023062408142288600_btaa943-B17","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1186\/s12916-019-1466-7","article-title":"Calibration: the Achilles heel of predictive analytics","volume":"17","author":"Van Calster","year":"2019","journal-title":"BMC Med"},{"key":"2023062408142288600_btaa943-B19","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1148\/radiol.2019182716","article-title":"A deep learning mammography-based model for improved breast cancer risk prediction","volume":"292","author":"Yala","year":"2019","journal-title":"Radiology"},{"key":"2023062408142288600_btaa943-B20","doi-asserted-by":"crossref","first-page":"931","DOI":"10.1038\/nmeth.3547","article-title":"Predicting effects of noncoding variants with deep learning-based sequence model","volume":"12","author":"Zhou","year":"2015","journal-title":"Nat. Methods"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btaa943\/36171277\/btaa943.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/36\/24\/5709\/50693032\/btaa943.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/36\/24\/5709\/50693032\/btaa943.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,24]],"date-time":"2023-06-24T23:44:11Z","timestamp":1687650251000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/36\/24\/5709\/6119340"}},"subtitle":[],"editor":[{"given":"Alfonso","family":"Valencia","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2020,12,15]]},"references-count":21,"journal-issue":{"issue":"24","published-print":{"date-parts":[[2021,4,5]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btaa943","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,12,15]]},"published":{"date-parts":[[2020,12,15]]}}}