{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,2]],"date-time":"2026-01-02T07:28:51Z","timestamp":1767338931732,"version":"3.37.3"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,1,28]],"date-time":"2020-01-28T00:00:00Z","timestamp":1580169600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,1,28]],"date-time":"2020-01-28T00:00:00Z","timestamp":1580169600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["Project RA 870\/7-1,  and Collaborative Research Center SFB 876, A3"],"award-info":[{"award-number":["Project RA 870\/7-1,  and Collaborative Research Center SFB 876, A3"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007601","name":"Horizon 2020","doi-asserted-by":"publisher","award":["Marie Sk\u0142odowska-Curie Grant No. 721746"],"award-info":[{"award-number":["Marie Sk\u0142odowska-Curie Grant No. 721746"]}],"id":[{"id":"10.13039\/501100007601","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n<jats:title>Background<\/jats:title>\n<jats:p>With modern methods in biotechnology, the search for biomarkers has advanced to a challenging statistical task exploring high dimensional data sets. Feature selection is a widely researched preprocessing step to handle huge numbers of biomarker candidates and has special importance for the analysis of biomedical data. Such data sets often include many input features not related to the diagnostic or therapeutic target variable. A less researched, but also relevant aspect for medical applications are costs of different biomarker candidates. These costs are often financial costs, but can also refer to other aspects, for example the decision between a painful biopsy marker and a simple urine test. In this paper, we propose extensions to two feature selection methods to control the total amount of such costs: greedy forward selection and genetic algorithms. In comprehensive simulation studies of binary classification tasks, we compare the predictive performance, the run-time and the detection rate of relevant features for the new proposed methods and five baseline alternatives to handle budget constraints.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Results<\/jats:title>\n<jats:p>In simulations with a predefined budget constraint, our proposed methods outperform the baseline alternatives, with just minor differences between them. Only in the scenario without an actual budget constraint, our adapted greedy forward selection approach showed a clear drop in performance compared to the other methods. However, introducing a hyperparameter to adapt the benefit-cost trade-off in this method could overcome this weakness.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Conclusions<\/jats:title>\n<jats:p>In feature cost scenarios, where a total budget has to be met, common feature selection algorithms are often not suitable to identify well performing subsets for a modelling task. Adaptations of these algorithms such as the ones proposed in this paper can help to tackle this problem.<\/jats:p>\n<\/jats:sec>","DOI":"10.1186\/s12859-020-3361-9","type":"journal-article","created":{"date-parts":[[2020,1,28]],"date-time":"2020-01-28T09:03:41Z","timestamp":1580202221000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Cost-Constrained feature selection in binary classification: adaptations for greedy forward selection and genetic algorithms"],"prefix":"10.1186","volume":"21","author":[{"given":"Rudolf","family":"Jagdhuber","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michel","family":"Lang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Arnulf","family":"Stenzl","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jochen","family":"Neuhaus","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8947-440X","authenticated-orcid":false,"given":"J\u00f6rg","family":"Rahnenf\u00fchrer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2020,1,28]]},"reference":[{"issue":"1","key":"3361_CR1","first-page":"7","volume":"13","author":"M Tan","year":"1993","unstructured":"Tan M. Cost-sensitive learning of classification knowledge and its applications in robotics. Mach Learn. 1993; 13(1):7\u201333.","journal-title":"Mach Learn"},{"key":"3361_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.knosys.2015.11.010","volume":"95","author":"Q Zhou","year":"2016","unstructured":"Zhou Q, Zhou H, Li T. Cost-sensitive feature selection using random forest: Selecting low-cost subsets of informative features. Knowl-Based Syst. 2016; 95:1\u201311.","journal-title":"Knowl-Based Syst"},{"issue":"7","key":"3361_CR3","doi-asserted-by":"publisher","first-page":"2481","DOI":"10.1016\/j.patcog.2014.01.008","volume":"47","author":"V Bol\u00f3n-Canedo","year":"2014","unstructured":"Bol\u00f3n-Canedo V, Porto-D\u00edaz I, S\u00e1nchez-Maro\u00f1o N, Alonso-Betanzos A. A framework for cost-based feature selection. Pattern Recogn. 2014; 47(7):2481\u20139.","journal-title":"Pattern Recogn"},{"issue":"1","key":"3361_CR4","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1016\/j.ijar.2013.04.003","volume":"55","author":"F Min","year":"2014","unstructured":"Min F, Hu Q, Zhu W. Feature selection with test cost constraint. Int J Approx Reason. 2014; 55(1):167\u201379.","journal-title":"Int J Approx Reason"},{"issue":"3","key":"3361_CR5","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1007\/s41066-016-0017-2","volume":"1","author":"F Min","year":"2016","unstructured":"Min F, Xu J. Semi-greedy heuristics for feature selection with test cost constraints. Granul Comput. 2016; 1(3):199\u2013211.","journal-title":"Granul Comput"},{"key":"3361_CR6","unstructured":"Liu J, Min F, Liao S, Zhu W. A genetic algorithm to attribute reduction with test cost constraint. In: 2011 6th International Conference on Computer Sciences and Convergence Information Technology (ICCIT). IEEE: 2011. p. 751\u20134."},{"key":"3361_CR7","doi-asserted-by":"publisher","unstructured":"Leskovec J, Krause A, Guestrin C, Faloutsos C, Faloutsos C, VanBriesen J, Glance N. Cost-effective outbreak detection in networks. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM: 2007. p. 420\u2013429. https:\/\/doi.org\/10.1145\/1281192.1281239.","DOI":"10.1145\/1281192.1281239"},{"issue":"2","key":"3361_CR8","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1137\/0202009","volume":"2","author":"JH Holland","year":"1973","unstructured":"Holland JH. Genetic algorithms and the optimal allocation of trials. SIAM J Comput. 1973; 2(2):88\u2013105.","journal-title":"SIAM J Comput"},{"issue":"Mar","key":"3361_CR9","first-page":"1157","volume":"3","author":"I Guyon","year":"2003","unstructured":"Guyon I, Elisseeff A. An introduction to variable and feature selection. J Mach Learn Res. 2003; 3(Mar):1157\u201382.","journal-title":"J Mach Learn Res"},{"key":"3361_CR10","doi-asserted-by":"publisher","first-page":"106839","DOI":"10.1016\/j.csda.2019.106839","volume":"143","author":"Andrea Bommert","year":"2020","unstructured":"Bommert A, Xudong S, Bischl B, Rahnenf\u00fchrer J, Lang M. Benchmark for filter methods for feature selection in high-dimensional data. Comput Stat Data Anal. 2019. https:\/\/doi.org\/10.1016\/j.csda.2019.106839.","journal-title":"Computational Statistics & Data Analysis"},{"issue":"6","key":"3361_CR11","doi-asserted-by":"publisher","first-page":"716","DOI":"10.1109\/TAC.1974.1100705","volume":"19","author":"H Akaike","year":"1974","unstructured":"Akaike H. A new look at the statistical model identification. IEEE Trans Autom Control. 1974; 19(6):716\u201323.","journal-title":"IEEE Trans Autom Control"},{"issue":"4","key":"3361_CR12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v053.i04","volume":"53","author":"L Scrucca","year":"2013","unstructured":"Scrucca L. GA: A package for genetic algorithms in R. J Stat Softw. 2013; 53(4):1\u201337.","journal-title":"J Stat Softw"},{"key":"3361_CR13","doi-asserted-by":"crossref","unstructured":"Scrucca L. On some extensions to GA package: hybrid optimisation, parallelisation and islands evolution. Submitted R J. 2016. Pre-print available at arXiv.","DOI":"10.32614\/RJ-2017-008"},{"issue":"Jan","key":"3361_CR14","first-page":"27","volume":"13","author":"G Brown","year":"2012","unstructured":"Brown G, Pocock A, Zhao M-J, Luj\u00e1n M. Conditional likelihood maximisation: a unifying framework for information theoretic feature selection. J Mach Learn Res. 2012; 13(Jan):27\u201366.","journal-title":"J Mach Learn Res"},{"key":"3361_CR15","series-title":"Springer Texts in Statistics","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-78189-1","volume-title":"Modern Multivariate Statistical Techniques","author":"Alan J. Izenman","year":"2008","unstructured":"Izenman AJ. Modern multivariate statistical techniques. Regression Classif Manifold Learn. 2008. https:\/\/doi.org\/10.1007\/978-0-387-78189-1."},{"issue":"170","key":"3361_CR16","first-page":"1","volume":"17","author":"B Bischl","year":"2016","unstructured":"Bischl B, Lang M, Kotthoff L, Schiffner J, Richter J, Studerus E, Casalicchio G, Jones ZM. mlr: Machine learning in R. J Mach Learn Res. 2016; 17(170):1\u20135.","journal-title":"J Mach Learn Res"},{"key":"3361_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2017\/7691937","volume":"2017","author":"Anne-Laure Boulesteix","year":"2017","unstructured":"Boulesteix A-L, De Bin R, Jiang X, Fuchs M. Ipf-lasso: Integrative-penalized regression with penalty factors for prediction based on multi-omics data. Comput Math Methods Med. 2017; 2017. https:\/\/doi.org\/10.1155\/2017\/7691937.","journal-title":"Computational and Mathematical Methods in Medicine"},{"issue":"10","key":"3361_CR18","doi-asserted-by":"publisher","first-page":"3783","DOI":"10.1021\/ac7025964","volume":"80","author":"T De Meyer","year":"2008","unstructured":"De Meyer T, Sinnaeve D, Van Gasse B, Tsiporkova E, Rietzschel ER, De Buyzere ML, Gillebert TC, Bekaert S, Martins JC, Van Criekinge W. Nmr-based characterization of metabolic alterations in hypertension using an adaptive, intelligent binning algorithm. Anal Chem. 2008; 80(10):3783\u201390.","journal-title":"Anal Chem"},{"key":"3361_CR19","doi-asserted-by":"publisher","unstructured":"de Torrente L, Zimmerman S, Suzuki M, Christopeit M, Greally JM, Mar J. The shape of gene expression distributions matter: how incorporating distribution shape improves the interpretation of cancer transcriptomic data. bioRxiv. 2019:572693. https:\/\/doi.org\/10.1101\/572693.","DOI":"10.1101\/572693"},{"issue":"22","key":"3361_CR20","doi-asserted-by":"publisher","first-page":"2059","DOI":"10.1056\/NEJMoa1301689","volume":"368","author":"CGAR Network","year":"2013","unstructured":"Network CGAR. Genomic and epigenomic landscapes of adult de novo acute myeloid leukemia. N Engl J Med. 2013; 368(22):2059\u201374.","journal-title":"N Engl J Med"},{"issue":"7353","key":"3361_CR21","doi-asserted-by":"publisher","first-page":"609","DOI":"10.1038\/nature10166","volume":"474","author":"CGAR Network","year":"2011","unstructured":"Network CGAR, et al.Integrated genomic analyses of ovarian carcinoma. Nature. 2011; 474(7353):609.","journal-title":"Nature"},{"issue":"7216","key":"3361_CR22","doi-asserted-by":"publisher","first-page":"1061","DOI":"10.1038\/nature07385","volume":"455","author":"CGAR Network","year":"2008","unstructured":"Network CGAR, et al.Comprehensive genomic characterization defines human glioblastoma genes and core pathways. Nature. 2008; 455(7216):1061.","journal-title":"Nature"},{"issue":"3","key":"3361_CR23","doi-asserted-by":"publisher","first-page":"225","DOI":"10.17713\/ajs.v32i3.458","volume":"32","author":"J Rahnenf\u00fchrer","year":"2003","unstructured":"Rahnenf\u00fchrer J, Futschik A. Cost-effective screening for differentially expressed genes in microarray experiments based on normal mixtures. Austrian J Stat. 2003; 32(3):225\u201338.","journal-title":"Austrian J Stat"},{"issue":"1","key":"3361_CR24","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1148\/radiology.143.1.7063747","volume":"143","author":"JA Hanley","year":"1982","unstructured":"Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (roc) curve. Radiology. 1982; 143(1):29\u201336.","journal-title":"Radiology"},{"issue":"5","key":"3361_CR25","doi-asserted-by":"publisher","first-page":"1755","DOI":"10.1016\/j.csda.2008.02.032","volume":"53","author":"LK Vaughan","year":"2009","unstructured":"Vaughan LK, Divers J, Padilla MA, Redden DT, Tiwari HK, Pomp D, Allison DB. The use of plasmodes as a supplement to simulations: a simple example evaluating individual admixture estimation methodologies. Comput Stat Data Anal. 2009; 53(5):1755\u201366.","journal-title":"Comput Stat Data Anal"},{"key":"3361_CR26","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1016\/j.csda.2013.10.018","volume":"72","author":"JM Franklin","year":"2014","unstructured":"Franklin JM, Schneeweiss S, Polinski JM, Rassen JA. Plasmode simulation for the evaluation of pharmacoepidemiologic methods in complex healthcare databases. Comput Stat Data Anal. 2014; 72:219\u201326.","journal-title":"Comput Stat Data Anal"},{"issue":"9","key":"3361_CR27","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1007\/s11306-018-1419-8","volume":"14","author":"M Banas","year":"2018","unstructured":"Banas M, Neumann S, Eiglsperger J, Schiffer E, Putz FJ, Reichelt-Wurm S, Kr\u00e4mer BK, Pagel P, Banas B. Identification of a urine metabolite constellation characteristic for kidney allograft rejection. Metabolomics. 2018; 14(9):116.","journal-title":"Metabolomics"},{"issue":"S1","key":"3361_CR28","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1002\/mrc.2461","volume":"47","author":"R Powers","year":"2009","unstructured":"Powers R. Nmr metabolomics and drug discovery. Magn Reson Chem. 2009; 47(S1):2\u201311.","journal-title":"Magn Reson Chem"},{"issue":"2","key":"3361_CR29","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1145\/2641190.2641198","volume":"15","author":"J Vanschoren","year":"2013","unstructured":"Vanschoren J, van Rijn JN, Bischl B, Torgo L. Openml: Networked science in machine learning. SIGKDD Explor. 2013; 15(2):49\u201360. https:\/\/doi.org\/10.1145\/2641190.2641198.","journal-title":"SIGKDD Explor"},{"key":"3361_CR30","unstructured":"Vanschoren J. OpenML Bioresponse. https:\/\/www.openml.org\/d\/4134. Accessed 25 Nov 2019."}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-020-3361-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s12859-020-3361-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-020-3361-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,1,27]],"date-time":"2021-01-27T00:04:16Z","timestamp":1611705856000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-020-3361-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,28]]},"references-count":30,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["3361"],"URL":"https:\/\/doi.org\/10.1186\/s12859-020-3361-9","relation":{},"ISSN":["1471-2105"],"issn-type":[{"type":"electronic","value":"1471-2105"}],"subject":[],"published":{"date-parts":[[2020,1,28]]},"assertion":[{"value":"16 September 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 January 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 January 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Sampling of the analyzed plasmode data set was ethically approved by the Universities of Leipzig and T\u00fcbingen (No. 205-15-01062015 and 379-2010BO2, respectively)","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests. RJ received personal fees from numares AG, outside the submitted work.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"26"}}