{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T03:42:09Z","timestamp":1740109329034,"version":"3.37.3"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2022,5,8]],"date-time":"2022-05-08T00:00:00Z","timestamp":1651968000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,5,8]],"date-time":"2022-05-08T00:00:00Z","timestamp":1651968000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003086","name":"Eusko Jaurlaritza","doi-asserted-by":"publisher","award":["IT1244-19","Elkartek BID3A"],"award-info":[{"award-number":["IT1244-19","Elkartek BID3A"]}],"id":[{"id":"10.13039\/501100003086","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003086","name":"Eusko Jaurlaritza","doi-asserted-by":"publisher","award":["Elkartek project 3KIA","KK2020\/00049"],"award-info":[{"award-number":["Elkartek project 3KIA","KK2020\/00049"]}],"id":[{"id":"10.13039\/501100003086","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003086","name":"Eusko Jaurlaritza","doi-asserted-by":"publisher","award":["University-Society Project 15\/19"],"award-info":[{"award-number":["University-Society Project 15\/19"]}],"id":[{"id":"10.13039\/501100003086","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003329","name":"Ministerio de Econom\u00eda y Competitividad","doi-asserted-by":"publisher","award":["PID2019-104966GB-I00"],"award-info":[{"award-number":["PID2019-104966GB-I00"]}],"id":[{"id":"10.13039\/501100003329","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003086","name":"Eusko Jaurlaritza","doi-asserted-by":"publisher","award":["PRE_2013_1_1313. PRE 2014_2_87","PRE_2015_2_0280"],"award-info":[{"award-number":["PRE_2013_1_1313. PRE 2014_2_87","PRE_2015_2_0280"]}],"id":[{"id":"10.13039\/501100003086","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003086","name":"Eusko Jaurlaritza","doi-asserted-by":"publisher","award":["PRE_2016_2_0314","BERC 2022-2025"],"award-info":[{"award-number":["PRE_2016_2_0314","BERC 2022-2025"]}],"id":[{"id":"10.13039\/501100003086","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003086","name":"Eusko Jaurlaritza","doi-asserted-by":"publisher","award":["ELKARTEK programs"],"award-info":[{"award-number":["ELKARTEK programs"]}],"id":[{"id":"10.13039\/501100003086","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100014440","name":"Ministerio de Ciencia, Innovaci\u00f3n y Universidades","doi-asserted-by":"publisher","award":["BCAM Severo Ochoa accreditation SEV-2017-0718"],"award-info":[{"award-number":["BCAM Severo Ochoa accreditation SEV-2017-0718"]}],"id":[{"id":"10.13039\/100014440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003451","name":"Euskal Herriko Unibertsitatea","doi-asserted-by":"publisher","award":["IT1244-19 project","the ELKARTEK program"],"award-info":[{"award-number":["IT1244-19 project","the ELKARTEK program"]}],"id":[{"id":"10.13039\/501100003451","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003329","name":"Ministerio de Econom\u00eda y Competitividad","doi-asserted-by":"publisher","award":["Project PID2019-104966GB-I00"],"award-info":[{"award-number":["Project PID2019-104966GB-I00"]}],"id":[{"id":"10.13039\/501100003329","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003451","name":"Euskal Herriko Unibertsitatea","doi-asserted-by":"publisher","award":["University-Society Project 15\/19"],"award-info":[{"award-number":["University-Society Project 15\/19"]}],"id":[{"id":"10.13039\/501100003451","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Knowl Inf Syst"],"published-print":{"date-parts":[[2022,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Statistical tests are a powerful set of tools when applied correctly, but unfortunately the extended misuse of them has caused great concern. Among many other applications, they are used in the detection of biomarkers so as to use the resulting <jats:italic>p<\/jats:italic>-values as a reference with which the candidate biomarkers are ranked. Although statistical tests can be used to rank, they have not been designed for that use. Moreover, there is no need to compute any <jats:italic>p<\/jats:italic>-value to build a ranking of candidate biomarkers. Those two facts raise the question of whether or not alternative methods which are not based on the computation of statistical tests that match or improve their performances can be proposed. In this paper, we propose two alternative methods to statistical tests. In addition, we propose an evaluation framework to assess both statistical tests and alternative methods in terms of both the performance and the reproducibility. The results indicate that there are alternative methods that can match or surpass methods based on statistical tests in terms of the reproducibility when processing real data, while maintaining a similar performance when dealing with synthetic data. The main conclusion is that there is room for the proposal of such alternative methods.\n<\/jats:p>","DOI":"10.1007\/s10115-022-01677-6","type":"journal-article","created":{"date-parts":[[2022,5,8]],"date-time":"2022-05-08T03:41:33Z","timestamp":1651981293000},"page":"1549-1570","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Are the statistical tests the best way to deal with the biomarker selection problem?"],"prefix":"10.1007","volume":"64","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8597-3260","authenticated-orcid":false,"given":"Ari","family":"Urkullu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8128-1099","authenticated-orcid":false,"given":"Aritz","family":"P\u00e9rez","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9969-9664","authenticated-orcid":false,"given":"Borja","family":"Calvo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,5,8]]},"reference":[{"key":"1677_CR1","unstructured":"Alzubaidi AHA (2019) Evolutionary and deep mining models for effective biomarker discovery. Nottingham Trent University (United Kingdom)"},{"key":"1677_CR2","doi-asserted-by":"publisher","first-page":"e3544","DOI":"10.7717\/peerj.3544","volume":"5","author":"V Amrhein","year":"2017","unstructured":"Amrhein V, Korner-Nievergelt F, Roth T (2017) The earth is flat (p 0.05): significance thresholds and the crisis of unreplicable research. PeerJ 5:e3544","journal-title":"PeerJ"},{"issue":"5","key":"1677_CR3","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1093\/bib\/bbp016","volume":"10","author":"S Baek","year":"2009","unstructured":"Baek S, Tsai CA, Chen JJ (2009) Development of biomarker classifiers from high-dimensional data. Brief Bioinform 10(5):537\u2013546","journal-title":"Brief Bioinform"},{"issue":"7604","key":"1677_CR4","doi-asserted-by":"publisher","first-page":"452","DOI":"10.1038\/533452a","volume":"533","author":"M Baker","year":"2016","unstructured":"Baker M (2016) 1,500 scientists lift the lid on reproducibility. Nature News 533(7604):452","journal-title":"Nature News"},{"issue":"1","key":"1677_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1755-8794-3-33","volume":"3","author":"CG Bell","year":"2010","unstructured":"Bell CG, Teschendorff AE, Rakyan VK, Maxwell AP, Beck S, Savage DA (2010) Genome-wide dna methylation analysis for diabetic nephropathy in type 1 diabetes mellitus. BMC Med Genomics 3(1):1\u201311","journal-title":"BMC Med Genomics"},{"issue":"1","key":"1677_CR6","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1002\/gepi.21774","volume":"38","author":"Y Chen","year":"2014","unstructured":"Chen Y, Ning Y, Hong C, Wang S (2014) Semiparametric tests for identifying differentially methylated loci with case-control designs using illumina arrays. Genet Epidemiol 38(1):42\u201350","journal-title":"Genet Epidemiol"},{"key":"1677_CR7","doi-asserted-by":"crossref","unstructured":"Cohen J (1995) The earth is round (p 05): Rejoinder","DOI":"10.1037\/0003-066X.50.12.1103"},{"issue":"12","key":"1677_CR8","doi-asserted-by":"publisher","first-page":"171085","DOI":"10.1098\/rsos.171085","volume":"4","author":"D Colquhoun","year":"2017","unstructured":"Colquhoun D (2017) The reproducibility of research and the misinterpretation of p-values. R Soc Open Sci 4(12):171085","journal-title":"R Soc Open Sci"},{"issue":"1","key":"1677_CR9","doi-asserted-by":"publisher","first-page":"587","DOI":"10.1186\/1471-2105-11-587","volume":"11","author":"P Du","year":"2010","unstructured":"Du P, Zhang X, Huang CC, Jafari N, Kibbe WA, Hou L, Lin SM (2010) Comparison of beta-value and m-value methods for quantifying methylation levels by microarray analysis. BMC Bioinform 11(1):587","journal-title":"BMC Bioinform"},{"issue":"8","key":"1677_CR10","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","volume":"27","author":"T Fawcett","year":"2006","unstructured":"Fawcett T (2006) An introduction to roc analysis. Pattern Recognit Lett 27(8):861\u2013874","journal-title":"Pattern Recognit Lett"},{"key":"1677_CR11","volume-title":"Statistical methods for research workers","author":"RA Fisher","year":"1925","unstructured":"Fisher RA (1925) Statistical methods for research workers. Genesis Publishing Pvt Ltd, Edinburgh, London"},{"issue":"3","key":"1677_CR12","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1053\/j.seminhematol.2008.04.003","volume":"45","author":"S Goodman","year":"2008","unstructured":"Goodman S (2008) A dirty dozen: twelve p-value misconceptions. Semin Hematol 45(3):135\u2013140","journal-title":"Semin Hematol"},{"issue":"4","key":"1677_CR13","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1007\/s10654-016-0149-3","volume":"31","author":"S Greenland","year":"2016","unstructured":"Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG (2016) Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol 31(4):337\u2013350","journal-title":"Eur J Epidemiol"},{"key":"1677_CR14","volume-title":"Bioconductor Case Studies","author":"F Hahne","year":"2010","unstructured":"Hahne F, Huber W, Gentleman R, Falcon S (2010) Bioconductor Case Studies. Springer Science & Business Media, New York city, New York"},{"issue":"4","key":"1677_CR15","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1016\/j.compbiolchem.2010.07.002","volume":"34","author":"Z He","year":"2010","unstructured":"He Z, Yu W (2010) Stable feature selection for biomarker discovery. Comput Biol Chem 34(4):215\u2013225","journal-title":"Comput Biol Chem"},{"issue":"3","key":"1677_CR16","doi-asserted-by":"publisher","first-page":"e1002106","DOI":"10.1371\/journal.pbio.1002106","volume":"13","author":"ML Head","year":"2015","unstructured":"Head ML, Holman L, Lanfear R, Kahn AT, Jennions MD (2015) The extent and consequences of p-hacking in science. PLoS Biol 13(3):e1002106","journal-title":"PLoS Biol"},{"issue":"1","key":"1677_CR17","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1007\/s10994-013-5328-9","volume":"93","author":"J Hern\u00e1ndez-Orallo","year":"2013","unstructured":"Hern\u00e1ndez-Orallo J, Flach P, Ferri C (2013) Roc curves in cost space. Mach Learn 93(1):71\u201391","journal-title":"Mach Learn"},{"key":"1677_CR18","unstructured":"Kuncheva LI (2007) A stability index for feature selection. In: Artificial intelligence and applications, pp 421\u2013427"},{"issue":"11","key":"1677_CR19","doi-asserted-by":"publisher","first-page":"1046","DOI":"10.1016\/j.trac.2006.10.007","volume":"25","author":"JO Lay Jr","year":"2006","unstructured":"Lay JO Jr, Liyanage R, Borgmann S, Wilkins CL (2006) Problems with the \u201comics\u2019\u2019. TrAC Trends Anal Chem 25(11):1046\u20131056","journal-title":"TrAC Trends Anal Chem"},{"issue":"1\u20132","key":"1677_CR20","first-page":"175","volume":"20A","author":"J Neyman","year":"1928","unstructured":"Neyman J, Pearson ES (1928) On the use and interpretation of certain test criteria for purposes of statistical inference: part I. Biometrika 20A(1\u20132):175\u2013240","journal-title":"Biometrika"},{"key":"1677_CR21","doi-asserted-by":"crossref","unstructured":"Neyman J, Pearson ES (1933) The testing of statistical hypotheses in relation to probabilities a priori. In: Mathematical Proceedings of the Cambridge Philosophical Society, 29, pp 492\u2013510. Cambridge University Press","DOI":"10.1017\/S030500410001152X"},{"issue":"7487","key":"1677_CR22","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1038\/506150a","volume":"506","author":"R Nuzzo","year":"2014","unstructured":"Nuzzo R (2014) Statistical errors. Nature 506(7487):150\u2013152","journal-title":"Nature"},{"key":"1677_CR23","first-page":"223","volume":"6","author":"JD Perezgonzalez","year":"2015","unstructured":"Perezgonzalez JD (2015) Fisher, neyman-pearson or nhst? a tutorial for teaching data testing. Front Psychol 6:223","journal-title":"Front Psychol"},{"issue":"7534","key":"1677_CR24","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1038\/nature14192","volume":"517","author":"D Sch\u00fcbeler","year":"2015","unstructured":"Sch\u00fcbeler D (2015) Function and information content of dna methylation. Nature 517(7534):321\u2013326","journal-title":"Nature"},{"issue":"4","key":"1677_CR25","doi-asserted-by":"publisher","first-page":"440","DOI":"10.1101\/gr.103606.109","volume":"20","author":"AE Teschendorff","year":"2010","unstructured":"Teschendorff AE, Menon U, Gentry-Maharaj A, Ramus SJ, Weisenberger DJ, Shen H, Campan M, Noushmehr H, Bell CG, Maxwell AP et al (2010) Age-dependent dna methylation of genes that are suppressed in stem cells is a hallmark of cancer. Genome Res 20(4):440\u2013446","journal-title":"Genome Res"},{"issue":"1","key":"1677_CR26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/01973533.2015.1012991","volume":"37","author":"D Trafimow","year":"2015","unstructured":"Trafimow D, Marks M (2015) Editorial. Basic Appl Soc Psychol 37(1):1\u20132","journal-title":"Basic Appl Soc Psychol"},{"issue":"2","key":"1677_CR27","doi-asserted-by":"publisher","first-page":"379","DOI":"10.1007\/s10115-020-01519-3","volume":"63","author":"A Urkullu","year":"2021","unstructured":"Urkullu A, P\u00e9rez A, Calvo B (2021) Statistical model for reproducibility in ranking-based feature selection. Knowl Inf Syst 63(2):379\u2013410","journal-title":"Knowl Inf Syst"},{"issue":"7","key":"1677_CR28","doi-asserted-by":"publisher","first-page":"686","DOI":"10.1002\/gepi.20619","volume":"35","author":"S Wang","year":"2011","unstructured":"Wang S (2011) Method to detect differentially methylated loci with case-control designs using illumina arrays. Genet Epidemiol 35(7):686\u2013694","journal-title":"Genet Epidemiol"},{"issue":"2","key":"1677_CR29","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1080\/00031305.2016.1154108","volume":"70","author":"RL Wasserstein","year":"2016","unstructured":"Wasserstein RL, Lazar NA (2016) The ASA\u2019s statement on p-values: context, process, and purpose. Am Stat 70(2):129\u2013133","journal-title":"Am Stat"},{"issue":"7541","key":"1677_CR30","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1038\/519009f","volume":"519","author":"C Woolston","year":"2015","unstructured":"Woolston C (2015) Psychology journal bans P values. Nature 519(7541):9","journal-title":"Nature"}],"container-title":["Knowledge and Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10115-022-01677-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10115-022-01677-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10115-022-01677-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,4]],"date-time":"2022-06-04T09:10:44Z","timestamp":1654333844000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10115-022-01677-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,8]]},"references-count":30,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,6]]}},"alternative-id":["1677"],"URL":"https:\/\/doi.org\/10.1007\/s10115-022-01677-6","relation":{},"ISSN":["0219-1377","0219-3116"],"issn-type":[{"type":"print","value":"0219-1377"},{"type":"electronic","value":"0219-3116"}],"subject":[],"published":{"date-parts":[[2022,5,8]]},"assertion":[{"value":"18 June 2018","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 March 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 March 2022","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 May 2022","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}