{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,7]],"date-time":"2026-02-07T15:06:14Z","timestamp":1770476774572,"version":"3.49.0"},"reference-count":31,"publisher":"Oxford University Press (OUP)","issue":"19","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2006,10,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Motivation: Gene selection algorithms for cancer classification, based on the expression of a small number of biomarker genes, have been the subject of considerable research in recent years. Shevade and Keerthi propose a gene selection algorithm based on sparse logistic regression (SLogReg) incorporating a Laplace prior to promote sparsity in the model parameters, and provide a simple but efficient training procedure. The degree of sparsity obtained is determined by the value of a regularization parameter, which must be carefully tuned in order to optimize performance. This normally involves a model selection stage, based on a computationally intensive search for the minimizer of the cross-validation error. In this paper, we demonstrate that a simple Bayesian approach can be taken to eliminate this regularization parameter entirely, by integrating it out analytically using an uninformative Jeffrey's prior. The improved algorithm (BLogReg) is then typically two or three orders of magnitude faster than the original algorithm, as there is no longer a need for a model selection step. The BLogReg algorithm is also free from selection bias in performance estimation, a common pitfall in the application of machine learning algorithms in cancer classification.<\/jats:p><jats:p>Results: The SLogReg, BLogReg and Relevance Vector Machine (RVM) gene selection algorithms are evaluated over the well-studied colon cancer and leukaemia benchmark datasets. The leave-one-out estimates of the probability of test error and cross-entropy of the BLogReg and SLogReg algorithms are very similar, however the BlogReg algorithm is found to be considerably faster than the original SLogReg algorithm. Using nested cross-validation to avoid selection bias, performance estimation for SLogReg on the leukaemia dataset takes almost 48 h, whereas the corresponding result for BLogReg is obtained in only 1 min 24 s, making BLogReg by far the more practical algorithm. BLogReg also demonstrates better estimates of conditional probability than the RVM, which are of great importance in medical applications, with similar computational expense.<\/jats:p><jats:p>Availability: A MATLAB implementation of the sparse logistic regression algorithm with Bayesian regularization (BLogReg) is available from<\/jats:p><jats:p>Contact: \u00a0gcc@cmp.uea.ac.uk<\/jats:p>","DOI":"10.1093\/bioinformatics\/btl386","type":"journal-article","created":{"date-parts":[[2006,7,15]],"date-time":"2006-07-15T01:51:27Z","timestamp":1152928287000},"page":"2348-2355","source":"Crossref","is-referenced-by-count":186,"title":["Gene selection in cancer classification using sparse logistic regression with Bayesian regularization"],"prefix":"10.1093","volume":"22","author":[{"given":"Gavin C.","family":"Cawley","sequence":"first","affiliation":[{"name":"School of Computing Sciences, University of East Anglia \u00a0 Norwich NR4 7TJ, UK"}]},{"given":"Nicola L. C.","family":"Talbot","sequence":"additional","affiliation":[{"name":"School of Computing Sciences, University of East Anglia \u00a0 Norwich NR4 7TJ, UK"}]}],"member":"286","published-online":{"date-parts":[[2006,7,14]]},"reference":[{"key":"2023012409232823600_b1","doi-asserted-by":"crossref","first-page":"6745","DOI":"10.1073\/pnas.96.12.6745","article-title":"Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays","volume":"96","author":"Alon","year":"1999","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023012409232823600_b2","doi-asserted-by":"crossref","first-page":"6562","DOI":"10.1073\/pnas.102102699","article-title":"Selection bias in gene extraction on the basis of microarray gene-expression data","volume":"99","author":"Ambroise","year":"2002","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023012409232823600_b3","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-4286-2","volume-title":"Statistical Decision Theory and Bayesian Analysis","author":"Berger","year":"1985","edition":"2nd"},{"key":"2023012409232823600_b4","first-page":"603","article-title":"Bayesian back-propagation","volume":"5","author":"Buntine","year":"1991","journal-title":"Complex Syst."},{"key":"2023012409232823600_b5","article-title":"Discriminative detection of regulatory elements","volume-title":"Bioinformatics","author":"Cawley","year":"2006"},{"key":"2023012409232823600_b6","article-title":"Sparse multinomial logistic regression via Bayesian regularisation using a Laplace prior","volume-title":"Neural Information Processing Systems","author":"Cawley","year":"2006"},{"key":"2023012409232823600_b7","doi-asserted-by":"crossref","first-page":"3385","DOI":"10.1093\/bioinformatics\/bti526","article-title":"Biomarker discovery in microarray gene expression data with Gaussian processes","volume":"21","author":"Chu","year":"2005","journal-title":"Bioinformatics"},{"key":"2023012409232823600_b8","first-page":"383","article-title":"Analysis of sparse Bayesian learning","volume-title":"Advances in Neural Information Processing Systems","author":"Faul","year":"2002"},{"key":"2023012409232823600_b9","article-title":"Fast marginal likelihood maximisation for sparse Bayesian models","volume-title":"Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics","author":"Faul","year":"2003"},{"key":"2023012409232823600_b10","doi-asserted-by":"crossref","first-page":"1150","DOI":"10.1109\/TPAMI.2003.1227989","article-title":"Adaptive sparseness for supervised learning","volume":"25","author":"Figueiredo","year":"2003","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"2023012409232823600_b11","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1126\/science.286.5439.531","article-title":"Molecular classification of cancer: class discovery and class prediction by gene expression monitoring","volume":"286","author":"Golub","year":"1999","journal-title":"Science"},{"key":"2023012409232823600_b12","volume-title":"Table of Integrals, Series and Products","author":"Gradshteyn","year":"1994","edition":"5th"},{"key":"2023012409232823600_b13","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1023\/A:1012487302797","article-title":"Gene selection for cancer classification using support vector machines","volume":"46","author":"Guyon","year":"2002","journal-title":"Mach. Learn."},{"key":"2023012409232823600_b14","volume-title":"Theory of Probability. Oxford Classic Texts in the Physical Sciences","author":"Jeffreys","year":"1961","edition":"3rd"},{"key":"2023012409232823600_b15","first-page":"1137","article-title":"A study of cross-validation and bootstrap for accuracy estimation and model selection","author":"Kohavi","year":"1995"},{"key":"2023012409232823600_b16","doi-asserted-by":"crossref","first-page":"532","DOI":"10.1093\/bioinformatics\/bti804","article-title":"A regularized discriminative model for the prediction of peptide-peptide interactions","volume":"22","author":"Lehrach","year":"2006","journal-title":"Bioinformatics"},{"key":"2023012409232823600_b17","doi-asserted-by":"crossref","first-page":"1332","DOI":"10.1093\/bioinformatics\/18.10.1332","article-title":"Bayesian automatic relevance determination algorithms for classifying gene expression data","volume":"18","author":"Li","year":"2002","journal-title":"Bioinformatics"},{"key":"2023012409232823600_b18","article-title":"On estimation of characters obtained in statistical procedure of reognition (in Russian)","volume":"3","author":"Luntz","year":"1969","journal-title":"Technicheskaya Kibernetica"},{"key":"2023012409232823600_b19","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1162\/neco.1992.4.3.415","article-title":"Bayesian interpolation","volume":"4","author":"MacKay","year":"1992","journal-title":"Neural Comput."},{"key":"2023012409232823600_b20","doi-asserted-by":"crossref","first-page":"720","DOI":"10.1162\/neco.1992.4.5.720","article-title":"The evidence framework applied to classification networks","volume":"4","author":"MacKay","year":"1992","journal-title":"Neural Comput."},{"key":"2023012409232823600_b21","doi-asserted-by":"crossref","first-page":"448","DOI":"10.1162\/neco.1992.4.3.448","article-title":"A practical Bayesian framework for backprop networks","volume":"4","author":"MacKay","year":"1992","journal-title":"Neural Comput."},{"key":"2023012409232823600_b22","article-title":"Hyperparameters: optimise or integrate out?","volume-title":"Maximum Entropy and Bayesian Methods","author":"MacKay","year":"1994"},{"key":"2023012409232823600_b23","first-page":"1035","article-title":"Comparison of approximate methods for handling hyperparameters","volume":"11","author":"MacKay","year":"1999","journal-title":"Neural Netw."},{"key":"2023012409232823600_b24","first-page":"85","article-title":"Predictive automatic relevance determination by expectation propagation","author":"Qi","year":"2004"},{"key":"2023012409232823600_b25","doi-asserted-by":"crossref","first-page":"2246","DOI":"10.1093\/bioinformatics\/btg308","article-title":"A simple and efficient algorithm for gene selection using sparse logistic regression","volume":"19","author":"Shevade","year":"2003","journal-title":"Bioinformatics"},{"key":"2023012409232823600_b26","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1111\/j.2517-6161.1974.tb00994.x","article-title":"Cross-validatory choice and assessment of statistical predictions","volume":"36","author":"Stone","year":"1974","journal-title":"J. R. Stat. Soc. B"},{"key":"2023012409232823600_b27","doi-asserted-by":"crossref","first-page":"3896","DOI":"10.1093\/bioinformatics\/bti631","article-title":"Simple decision rules for classifying human cancers from gene expression profiles","volume":"21","author":"Tan","year":"2005","journal-title":"Bioinformatics"},{"key":"2023012409232823600_b28","volume-title":"Solutions of Ill-Posed Problems","author":"Tikhonov","year":"1977"},{"key":"2023012409232823600_b29","first-page":"211","article-title":"Sparse Bayesian learning and the relevance vector machine","volume":"1","author":"Tipping","year":"2001","journal-title":"J. Mach. Learn. Res."},{"key":"2023012409232823600_b30","first-page":"514","article-title":"Gaussian processes for regression","volume-title":"Neural Information Processing Systems 8","author":"Williams","year":"1996"},{"key":"2023012409232823600_b31","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1162\/neco.1995.7.1.117","article-title":"Bayesian regularization and pruning using a Laplace prior","volume":"7","author":"Williams","year":"1995","journal-title":"Neural Comput."}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/22\/19\/2348\/48840826\/bioinformatics_22_19_2348.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/22\/19\/2348\/48840826\/bioinformatics_22_19_2348.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,9]],"date-time":"2025-01-09T22:34:35Z","timestamp":1736462075000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/22\/19\/2348\/240686"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2006,7,14]]},"references-count":31,"journal-issue":{"issue":"19","published-print":{"date-parts":[[2006,10,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btl386","relation":{},"ISSN":["1367-4811","1367-4803"],"issn-type":[{"value":"1367-4811","type":"electronic"},{"value":"1367-4803","type":"print"}],"subject":[],"published-other":{"date-parts":[[2006,10,1]]},"published":{"date-parts":[[2006,7,14]]}}}