{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,9]],"date-time":"2026-02-09T17:04:36Z","timestamp":1770656676865,"version":"3.49.0"},"reference-count":34,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2018,1,30]],"date-time":"2018-01-30T00:00:00Z","timestamp":1517270400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001807","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Pesquisa do Estado de S\u00e3o Paulo","doi-asserted-by":"publisher","award":["2013\/07467-1"],"award-info":[{"award-number":["2013\/07467-1"]}],"id":[{"id":"10.13039\/501100001807","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001807","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Pesquisa do Estado de S\u00e3o Paulo","doi-asserted-by":"publisher","award":["2015\/01587-0"],"award-info":[{"award-number":["2015\/01587-0"]}],"id":[{"id":"10.13039\/501100001807","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>This paper uses a classical approach to feature selection: minimization of a cost function applied on estimated joint distributions. However, in this new formulation, the optimization search space is extended. The original search space is the Boolean lattice of features sets (BLFS), while the extended one is a collection of Boolean lattices of ordered pairs (CBLOP), that is (features, associated value), indexed by the elements of the BLFS. In this approach, we may not only select the features that are most related to a variable Y, but also select the values of the features that most influence the variable or that are most prone to have a specific value of Y. A local formulation of Shannon\u2019s mutual information, which generalizes Shannon\u2019s original definition, is applied on a CBLOP to generate a multiple resolution scale for characterizing variable dependence, the Local Lift Dependence Scale (LLDS). The main contribution of this paper is to define and apply the LLDS to analyse local properties of joint distributions that are neglected by the classical Shannon\u2019s global measure in order to select features. This approach is applied to select features based on the dependence between: i\u2014the performance of students on university entrance exams and on courses of their first semester in the university; ii\u2014the congress representative party and his vote on different matters; iii\u2014the cover type of terrains and several terrain properties.<\/jats:p>","DOI":"10.3390\/e20020097","type":"journal-article","created":{"date-parts":[[2018,1,30]],"date-time":"2018-01-30T12:13:37Z","timestamp":1517314417000},"page":"97","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Feature Selection based on the Local Lift Dependence Scale"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6087-4821","authenticated-orcid":false,"given":"Diego","family":"Marcondes","sequence":"first","affiliation":[{"name":"Institute of Mathematics and Statistics, University of S\u00e3o Paulo, S\u00e3o Paulo 05508-090, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adilson","family":"Simonis","sequence":"additional","affiliation":[{"name":"Institute of Mathematics and Statistics, University of S\u00e3o Paulo, S\u00e3o Paulo 05508-090, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Junior","family":"Barrera","sequence":"additional","affiliation":[{"name":"Institute of Mathematics and Statistics, University of S\u00e3o Paulo, S\u00e3o Paulo 05508-090, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,1,30]]},"reference":[{"key":"ref_1","first-page":"1157","article-title":"An introduction to variable and feature selection","volume":"3","author":"Guyon","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1016\/S0004-3702(97)00043-X","article-title":"Wrappers for feature subset selection","volume":"97","author":"Kohavi","year":"1997","journal-title":"Artif. Intell."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1007\/978-3-540-35488-8_6","article-title":"Embedded methods","volume":"Volume 207","author":"Guyon","year":"2006","journal-title":"Feature Extraction. Studies in Fuzziness and Soft Computing"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"John, G.H., Kohavi, R., and Pfleger, K. (1994, January 10\u201313). Irrelevant features and the subset selection problem. Proceedings of the Eleventh International Conference on International Conference on Machine Learning, New Brunswick, NJ, USA.","DOI":"10.1016\/B978-1-55860-335-6.50023-4"},{"key":"ref_5","unstructured":"Hall, M.A. (29\u20132, January 29). Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning. Proceedings of the Seventeenth International Conference on Machine Learning, Stanford, CA, USA."},{"key":"ref_6","unstructured":"Das, S. (July, January 28). Filters, Wrappers and a Boosting-Based Hybrid for Feature Selection. Proceedings of the Eighteenth International Conference on Machine Learning, Williamstown, MA, USA."},{"key":"ref_7","unstructured":"Yu, L., and Liu, H. (2003, January 21\u201324). Feature selection for high-dimensional data: A fast correlation-based filter solution. Proceedings of the 20th International Conference on Machine Learning (ICML-03), Washington, DC, USA."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1007\/s10044-006-0031-0","article-title":"W-operator window design by minimization of mean conditional entropy","volume":"9","author":"Martins","year":"2006","journal-title":"Pattern Anal. Appl."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1016\/j.patcog.2009.08.018","article-title":"U-curve: A branch-and-bound optimization algorithm for U-shaped cost functions on Boolean lattices applied to the feature selection problem","volume":"43","author":"Ris","year":"2010","journal-title":"Pattern Recognit."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1016\/j.patcog.2017.08.013","article-title":"A fast Branch-and-Bound algorithm for U-curve feature selection","volume":"73","author":"Reis","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1109\/34.990133","article-title":"Unsupervised feature selection using feature similarity","volume":"24","author":"Mitra","year":"2002","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_12","unstructured":"Lichman, M. (2018, January 26). UCI Machine Learning Repository, 2013. Available online: http:\/\/archive.ics.uci.edu\/ml\/index.php."},{"key":"ref_13","unstructured":"Coppock, D.S. (2018, January 26). Why Lift? Data Modelling and Mining. Available online: https:\/\/www.information-management.com\/news\/why-lift."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tuff\u00e9ry, S., and Riesco, R. (2011). Data Mining and Statistics for Decision Making, Wiley.","DOI":"10.1002\/9780470979174"},{"key":"ref_15","unstructured":"Shannon, C.E., and Weaver, W. (1949). The Mathematical Theory of Communication, University of Illinois Press."},{"key":"ref_16","unstructured":"Bouma, G. (2018, January 26). Normalized (Pointwise) Mutual Information in Collocation Extraction. Available online: https:\/\/svn.spraakdata.gu.se\/repos\/gerlof\/pub\/www\/Docs\/npmi-pfd.pdf."},{"key":"ref_17","unstructured":"Role, F., and Nadif, M. (2011, January 26\u201329). Handling the Impact of Low Frequency Events on Co-occurrence based Measures of Word Similarity. Proceedings of the International Conference on Knowledge Discovery and Information Retrieval, Paris, France."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On information and sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Deng, L.Y. (2006). The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning, Taylor & Francis.","DOI":"10.1198\/tech.2006.s353"},{"key":"ref_20","unstructured":"Neto, U.M.B., and Dougherty, E.R. (2015). Error Estimation for Pattern Recognition, Wiley."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1016\/S0304-3975(97)00115-1","article-title":"On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems","volume":"209","author":"Amaldi","year":"1998","journal-title":"Theor. Comput. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1109\/TIT.1963.1057810","article-title":"On the effectiveness of receptors in recognition systems","volume":"9","author":"Marill","year":"1963","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_23","first-page":"1100","article-title":"A direct method of nonparametric measurement selection","volume":"100","author":"Whitney","year":"1971","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1119","DOI":"10.1016\/0167-8655(94)90127-9","article-title":"Floating search methods in feature selection","volume":"15","author":"Pudil","year":"1994","journal-title":"Pattern Recognit. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1157","DOI":"10.1016\/S0167-8655(99)00083-5","article-title":"Adaptive floating search methods in feature selection","volume":"20","author":"Somol","year":"1999","journal-title":"Pattern Recognit. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Somol, P., Novovi\u010dov\u00e1, J., and Pudil, P. (2006). Flexible-hybrid sequential floating search in statistical feature selection. Structural, Syntactic, and Statistical Pattern Recognition, Springer.","DOI":"10.1007\/11815921_69"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1932","DOI":"10.1016\/j.patcog.2008.11.018","article-title":"An improvement on floating search algorithms for feature subset selection","volume":"42","author":"Nakariyakul","year":"2009","journal-title":"Pattern Recognit."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"917","DOI":"10.1109\/TC.1977.1674939","article-title":"A branch and bound algorithm for feature subset selection","volume":"9","author":"Narendra","year":"1977","journal-title":"IEEE Trans. Comput."},{"key":"ref_29","unstructured":"Ris, M.S. (2012). Minimization of Decomposable in U-Shaped Curves Functions Defined on Poset Chains\u2014Algorithms and Applications. [Ph.D. Thesis, Institute of Mathematics and Statistics, University of Sao Paulo]. (In Portuguese)."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1226","DOI":"10.1109\/TPAMI.2005.159","article-title":"Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy","volume":"27","author":"Peng","year":"2005","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"\u015amieja, M., and Warszycki, D. (2016). Average information content maximization\u2014A new approach for fingerprint hybridization and reduction. PLoS ONE, 11.","DOI":"10.1371\/journal.pone.0146666"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1667","DOI":"10.1109\/TPAMI.2002.1114861","article-title":"Input feature selection by mutual information based on Parzen window","volume":"24","author":"Kwak","year":"2002","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_33","first-page":"49","article-title":"On the generalized distance in statistics","volume":"2","author":"Mahalanobis","year":"1936","journal-title":"Proc. Natl. Inst. Sci. (Calcutta)"},{"key":"ref_34","unstructured":"R Core Team (2016). R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/2\/97\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T14:53:08Z","timestamp":1760194388000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/2\/97"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,1,30]]},"references-count":34,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2018,2]]}},"alternative-id":["e20020097"],"URL":"https:\/\/doi.org\/10.3390\/e20020097","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,1,30]]}}}