{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,27]],"date-time":"2026-01-27T11:33:55Z","timestamp":1769513635113,"version":"3.49.0"},"reference-count":31,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2019,8,31]],"date-time":"2019-08-31T00:00:00Z","timestamp":1567209600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004895","name":"European Social Fund","doi-asserted-by":"publisher","award":["-"],"award-info":[{"award-number":["-"]}],"id":[{"id":"10.13039\/501100004895","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>A key challenge in information theoretic feature selection is to estimate mutual information expressions that capture three desirable terms\u2014the relevancy of a feature with the output, the redundancy and the complementarity between groups of features. The challenge becomes more pronounced in multi-target problems, where the output space is multi-dimensional. Our work presents an algorithm that captures these three desirable terms and is suitable for the well-known multi-target prediction settings of multi-label\/dimensional classification and multivariate regression. We achieve this by combining two ideas\u2014deriving low-order information theoretic approximations for the input space and using quantization algorithms for deriving low-dimensional approximations of the output space. Under the above framework we derive a novel criterion, Group-JMI-Rand, which captures various high-order target interactions. In an extensive experimental study we showed that our suggested criterion achieves competing performance against various other information theoretic feature selection criteria suggested in the literature.<\/jats:p>","DOI":"10.3390\/e21090855","type":"journal-article","created":{"date-parts":[[2019,9,2]],"date-time":"2019-09-02T03:16:12Z","timestamp":1567394172000},"page":"855","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Information Theoretic Multi-Target Feature Selection via Output Space Quantization"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6582-7453","authenticated-orcid":false,"given":"Konstantinos","family":"Sechidis","sequence":"first","affiliation":[{"name":"Department of Computer Science, Aristotle University, 54124 Thessaloniki, Greece"},{"name":"School of Computer Science, University of Manchester, Manchester M13 9PL, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eleftherios","family":"Spyromitros-Xioufis","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Aristotle University, 54124 Thessaloniki, Greece"},{"name":"Expedia, 1207 Geneva, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ioannis","family":"Vlahavas","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Aristotle University, 54124 Thessaloniki, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,8,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Guyon, I.M., Gunn, S.R., Nikravesh, M., and Zadeh, L. (2006). Feature Extraction: Foundations and Applications, Springer. [1st ed.].","DOI":"10.1007\/978-3-540-35488-8"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1007\/s00521-013-1368-0","article-title":"A review of feature selection methods based on mutual information","volume":"24","author":"Vergara","year":"2014","journal-title":"Neural Comput. Appl."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Waegeman, W., Dembczynski, K., and Huellermeier, E. (2018). Multi-target prediction: A unifying view on problems and methods. arXiv.","DOI":"10.1007\/s10618-018-0595-5"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1757","DOI":"10.1016\/j.patcog.2004.03.009","article-title":"Learning multi-label scene classification","volume":"37","author":"Boutell","year":"2004","journal-title":"Pattern Recognit."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Elisseeff, A., and Weston, J. (2001). A Kernel Method for Multi-Labelled Classification. Advances in Neural Information Processing Systems (NIPS) 14, MIT Press.","DOI":"10.7551\/mitpress\/1120.003.0092"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1159","DOI":"10.1016\/j.ecolmodel.2009.01.037","article-title":"Using single-and multi-target regression trees and ensembles to model a compound index of vegetation condition","volume":"220","author":"Kocev","year":"2009","journal-title":"Ecol. Model."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Sechidis, K., Nikolaou, N., and Brown, G. (2014). Information Theoretic Feature Selection in Multi-label Data through Composite Likelihood. S+SSPR 2014, Springer.","DOI":"10.1007\/978-3-662-44415-3_15"},{"key":"ref_8","first-page":"27","article-title":"Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection","volume":"13","author":"Brown","year":"2012","journal-title":"J. Mach. Learn. Res. (JMLR)"},{"key":"ref_9","unstructured":"Sechidis, K., Spyromitros-Xioufis, E., and Vlahavas, I. (2019, January 24\u201326). Multi-target feature selection through output space clustering. Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Bruges, Belgium."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2507","DOI":"10.1093\/bioinformatics\/btm344","article-title":"A review of feature selection techniques in bioinformatics","volume":"23","author":"Saeys","year":"2007","journal-title":"Bioinformatics"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Tsoumakas, G., Katakis, I., and Vlahavas, I. (2009). Mining multi-label data. Data Mining and Knowledge Discovery Handbook, Springer.","DOI":"10.1007\/978-0-387-09823-4_34"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.neucom.2015.07.118","article-title":"A systematic review of multi-label feature selection and a new method based on label construction","volume":"180","author":"Monard","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_13","unstructured":"Yang, Y., and Pedersen, J.O. (1997, January 8\u201312). A Comparative Study on Feature Selection in Text Categorization. Proceedings of the 14th International Conference on Machine Learning (ICML), Nashville, TN, USA."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"929","DOI":"10.1049\/el.2012.1600","article-title":"Approximating mutual information for multi-label feature selection","volume":"48","author":"Lee","year":"2012","journal-title":"Electron. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Chen, W., Yan, J., Zhang, B., Chen, Z., and Yang, Q. (2007, January 28\u201331). Document transformation for multi-label feature selection in text categorization. Proceedings of the Seventh IEEE International Conference on Data Mining (ICDM 2007), Omaha, NE, USA.","DOI":"10.1109\/ICDM.2007.18"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/j.ijar.2017.04.002","article-title":"Dealing with under-reported variables: An information theoretic solution","volume":"85","author":"Sechidis","year":"2017","journal-title":"Int. J. Approx. Reason."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1016\/j.patrec.2012.10.005","article-title":"Feature selection for multi-label classification using multivariate mutual information","volume":"34","author":"Lee","year":"2013","journal-title":"Pattern Recognit. Lett."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2761","DOI":"10.1016\/j.patcog.2015.04.009","article-title":"Fast multi-label feature selection based on information-theoretic feature ranking","volume":"48","author":"Lee","year":"2015","journal-title":"Pattern Recognit."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1016\/j.patcog.2017.01.014","article-title":"SCLS: Multi-label Feature Selection based on Scalable Criterion for Large Label Set","volume":"66","author":"Lee","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1079","DOI":"10.1109\/TKDE.2010.164","article-title":"Random k-labelsets for multilabel classification","volume":"23","author":"Tsoumakas","year":"2011","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer. [2nd ed.].","DOI":"10.1007\/978-0-387-84858-7"},{"key":"ref_22","first-page":"163","article-title":"Some data analyses using mutual information","volume":"18","author":"Brillinger","year":"2004","journal-title":"Braz. J. Probab. Stat."},{"key":"ref_23","first-page":"2411","article-title":"Mulan: A Java Library for Multi-Label Learning","volume":"12","author":"Tsoumakas","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_24","unstructured":"Zhang, M.L., and Zhou, Z.H. (2005, January 25\u201327). A k-nearest neighbor based algorithm for multi-label classification. Proceedings of the IEEE International Conference on Granular Computing, Beijing, China."},{"key":"ref_25","first-page":"1","article-title":"Statistical comparisons of classifiers over multiple data sets","volume":"7","year":"2006","journal-title":"J. Mach. Learn. Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2044","DOI":"10.1016\/j.ins.2009.12.010","article-title":"Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power","volume":"180","author":"Luengo","year":"2010","journal-title":"Inf. Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"e2721","DOI":"10.7717\/peerj.2721","article-title":"A methodology for the design of experiments in computational intelligence with multiple regression models","volume":"4","author":"Gestal","year":"2016","journal-title":"PeerJ"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1007\/s10994-016-5546-z","article-title":"Multi-target regression via input space expansion: Treating targets as inputs","volume":"104","author":"Tsoumakas","year":"2016","journal-title":"Mach. Learn."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1023\/A:1025667309714","article-title":"Theoretical and empirical analysis of ReliefF and RReliefF","volume":"53","author":"Kononenko","year":"2003","journal-title":"Mach. Learn."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1007\/s10994-017-5648-2","article-title":"Simple strategies for semi-supervised feature selection","volume":"107","author":"Sechidis","year":"2018","journal-title":"Mach. Learn."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3365","DOI":"10.1093\/bioinformatics\/bty357","article-title":"Distinguishing prognostic and predictive biomarkers: An information theoretic approach","volume":"34","author":"Sechidis","year":"2018","journal-title":"Bioinformatics"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/9\/855\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:15:39Z","timestamp":1760188539000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/9\/855"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,8,31]]},"references-count":31,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2019,9]]}},"alternative-id":["e21090855"],"URL":"https:\/\/doi.org\/10.3390\/e21090855","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,8,31]]}}}