{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,18]],"date-time":"2025-10-18T14:59:25Z","timestamp":1760799565891,"version":"build-2065373602"},"reference-count":38,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2012,2,17]],"date-time":"2012-02-17T00:00:00Z","timestamp":1329436800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>This paper presents a supervised variable selection method applied to regression problems. This method selects the variables applying a hierarchical clustering strategy based on information measures. The proposed technique can be applied to single-output regression datasets, and it is extendable to multi-output datasets. For single-output datasets, the method is compared against three other variable selection methods for regression on four datasets. In the multi-output case, it is compared against other state-of-the-art method and tested using two regression datasets. Two different figures of merit are used (for the single and multi-output cases) in order to analyze and compare the performance of the proposed method.<\/jats:p>","DOI":"10.3390\/e14020323","type":"journal-article","created":{"date-parts":[[2012,2,17]],"date-time":"2012-02-17T11:01:05Z","timestamp":1329476465000},"page":"323-343","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Filter-Type Variable Selection Based on Information Measures for Regression Tasks"],"prefix":"10.3390","volume":"14","author":[{"given":"Pedro","family":"Latorre Carmona","sequence":"first","affiliation":[{"name":"Institute of New Imaging Technologies, Universidad Jaume I, Campus del Riu Sec, s\/n, 12071 Castell\u00f3n de la Plana, Spain"}]},{"given":"Jos\u00e9 Mart\u00ednez","family":"Sotoca","sequence":"additional","affiliation":[{"name":"Institute of New Imaging Technologies, Universidad Jaume I, Campus del Riu Sec, s\/n, 12071 Castell\u00f3n de la Plana, Spain"}]},{"given":"Filiberto","family":"Pla","sequence":"additional","affiliation":[{"name":"Institute of New Imaging Technologies, Universidad Jaume I, Campus del Riu Sec, s\/n, 12071 Castell\u00f3n de la Plana, Spain"}]}],"member":"1968","published-online":{"date-parts":[[2012,2,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"131","DOI":"10.3233\/IDA-1997-1302","article-title":"Feature selection for classification","volume":"1","author":"Dash","year":"1997","journal-title":"Intell. Data Anal."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1007\/978-3-642-01805-3_4","article-title":"Advances in feature selection with mutual information","volume":"5400\/2009","author":"Verleysen","year":"2009","journal-title":"Similarity Based Clust."},{"key":"ref_3","unstructured":"Karagiannopoulos, M., Anyfantis, D., Kotsiantis, S.B., and Pintelas, P.E. (2007, January 20\u201322). Feature selection for regression problems. Proceedings of the 8th Hellenic European Research on Computer Mathematics & its Applications, Athens, Greece."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1155","DOI":"10.1016\/j.infsof.2010.05.009","article-title":"GA-based method for feature selection and parameters optimization for machine learning regression applied to software effort estimation","volume":"52","author":"Oliveira","year":"2010","journal-title":"Inf. Softw. Technol."},{"key":"ref_5","unstructured":"Eirola, E., Liiti\u00e4inen, E., and Lendasse, A. (2008, January 23\u201325). Using the delta test for variable selection. Proceedings of the European Symposium on Artificial Neural Networks\u2014Advances in Computational Intelligence and Learning, Bruges, Belgium."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1007\/s10255-008-8815-1","article-title":"Approximating Conditional density functions using dimension reduction","volume":"25","author":"Fan","year":"2009","journal-title":"Acta Math. Appl. Sin."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.chemolab.2005.06.010","article-title":"Mutual information for the selection of relevant variables in spectrometric nonlinear modelling","volume":"80","author":"Rossi","year":"2006","journal-title":"Chemom. Intell. Lab. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/34.824819","article-title":"Statistical pattern recognition: A review","volume":"22","author":"Jain","year":"2000","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_9","first-page":"279","article-title":"Floating search methods for feature selection with nonmonotonic criterion functions","volume":"2","author":"Pudil","year":"1994","journal-title":"Pattern Recogn."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1023\/A:1007379606734","article-title":"Multitask learning","volume":"28","author":"Caruana","year":"1997","journal-title":"Mach. Learn."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2068","DOI":"10.1016\/j.patcog.2009.12.013","article-title":"Supervised feature selection by clustering using conditional mutual information-based distances","volume":"43","author":"Sotoca","year":"2010","journal-title":"Pattern Recogn."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Latorre Carmona, P., Sotoca, J.M., Pla, F., Phoa, F.K.H., and Bioucas Dias, J. (2011, January 8\u201310). Feature selection in regression tasks using conditional mutual information. Proceedings of the Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA \u201911), Las Palmas de Gran Canaria, Spain.","DOI":"10.1007\/978-3-642-21257-4_28"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"5930","DOI":"10.1109\/TIT.2010.2080891","article-title":"On the interplay between conditional entropy and the error probability","volume":"56","author":"Ho","year":"2010","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_14","unstructured":"Cover, T.M., and Thomas, J.A. (1991). Elements of Information Theory, John Wiley & Sons Inc."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1226","DOI":"10.1109\/TPAMI.2005.159","article-title":"Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundance","volume":"27","author":"Peng","year":"2005","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1109\/72.977291","article-title":"Input feature selection for classification problems","volume":"13","author":"Kwak","year":"2002","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1080\/01621459.1963.10500845","article-title":"Hierarchical grouping to optimize an objective function","volume":"58","author":"Ward","year":"1963","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Yeung, R.W. (2002). A First Course in Information Theory, Springer.","DOI":"10.1007\/978-1-4419-8608-5"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ney, H. (2003, January 4\u20136). On the relationship between classification error bounds and training criteria in statistical pattern recognition. Proceedings of the Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA \u201903), Puerto de Andratx, Mallorca, Spain.","DOI":"10.1007\/978-3-540-44871-6_74"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1093\/biomet\/83.1.189","article-title":"Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems","volume":"83","author":"Fan","year":"1996","journal-title":"Biometrika"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1080\/10618600.1996.10474715","article-title":"Estimating and visualizing conditional densities","volume":"5","author":"Hyndman","year":"1996","journal-title":"J. Comput. Graph. Stat."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1707","DOI":"10.1016\/j.csda.2010.01.011","article-title":"Fast kernel conditional density estimation: A dual-tree Monte Carlo approach","volume":"54","author":"Holmes","year":"2010","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"832","DOI":"10.1214\/aoms\/1177728190","article-title":"Remarks on some nonparametric estimates of a density function","volume":"27","author":"Rosenblatt","year":"1956","journal-title":"Ann. Math. Stat."},{"key":"ref_24","unstructured":"Nocedal, J., and Wright, S.J. (2006). Numerical Optimization, Springer. [2nd ed.]."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1038","DOI":"10.1093\/ietisy\/e90-d.7.1038","article-title":"Particle swarms for feature extraction of hyperspectral data","volume":"E90D","author":"Monteiro","year":"2007","journal-title":"IEICE Trans. Inf. Syst."},{"key":"ref_26","unstructured":"Kennedy, J., and Eberhart, R. (December, January 27). Particle swarm optimization. Proceedings of the IEEE International Conference on Neural Networks, Perth, WA, Australia."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1111\/j.1467-9868.2005.00503.x","article-title":"Regularization and variable selection via the elastic net","volume":"67","author":"Zou","year":"2005","journal-title":"J. R. Stat. Soc. B"},{"key":"ref_28","unstructured":"Kolar, M., and Xing, E.P. (2010, January 13\u201315). Ultra-high dimensional multiple output learning with simultaneous orthogonal matching pursuit: Screening approach. Proceedings of the International Conference on Artificial Intelligence and Statistics, Sardinia, Italy."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1007\/s11222-008-9111-x","article-title":"Joint covariate selection and joint subspace selection for multiple classification problems","volume":"20","author":"Obozinski","year":"2010","journal-title":"Stat. Comput."},{"key":"ref_30","unstructured":"Moreno, J.F. (2005). SEN2FLEX Data Acquisition Report, Universidad de Valencia. Technical Report."},{"key":"ref_31","unstructured":"DELVE data repository. Available online: http:\/\/www.cs.toronto.edu\/\u223cdelve\/."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/0095-0696(78)90006-2","article-title":"Hedonic prices and the demand for clean air","volume":"5","author":"Harrison","year":"1978","journal-title":"J. Environ. Econ. Manag."},{"key":"ref_33","unstructured":"UCI machine learning repository. Available online: http:\/\/archive.ics.uci.edu\/ml\/."},{"key":"ref_34","unstructured":"Drucker, H., Burges, C., Kaufman, L., Kaufman, L., Smola, A., and Vapnik, V. (1997). Neural Information Processing Systems, MIT Press."},{"key":"ref_35","first-page":"2298","article-title":"SVM multiregression for non-linear channel estimation in multiple-input multiple-output systems","volume":"58","year":"2004","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1667","DOI":"10.1109\/TPAMI.2002.1114861","article-title":"Input feature selection by mutual information based on parzen window","volume":"24","author":"Kwak","year":"2002","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2044","DOI":"10.1016\/j.ins.2009.12.010","article-title":"Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power","volume":"180","author":"Luengo","year":"2010","journal-title":"Inf. Sci."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Johnson, R.A., and Wichern, D.W. (2007). Applied Multivariate Statistical Analysis, Prentice Hall.","DOI":"10.1002\/9780470061572.eqr239"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/14\/2\/323\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:48:57Z","timestamp":1760219337000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/14\/2\/323"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,2,17]]},"references-count":38,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2012,2]]}},"alternative-id":["e14020323"],"URL":"https:\/\/doi.org\/10.3390\/e14020323","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2012,2,17]]}}}