{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T11:02:34Z","timestamp":1770289354356,"version":"3.49.0"},"reference-count":53,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2024,11,26]],"date-time":"2024-11-26T00:00:00Z","timestamp":1732579200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The primary objective of our study is to analyze how the nature of explanatory variables influences the values and behavior of impurity measures, including the Shannon, R\u00e9nyi, Tsallis, Sharma\u2013Mittal, Sharma\u2013Taneja, and Kapur entropies. Our analysis aims to use these measures in the interactive learning of decision trees, particularly in the tie-breaking situations where an expert needs to make a decision. We simulate the values of explanatory variables from various probability distributions in order to consider a wide range of variability and properties. These probability distributions include the normal, Cauchy, uniform, exponential, and two beta distributions. This research assumes that the values of the binary responses are generated from the logistic regression model. All of the six mentioned probability distributions of the explanatory variables are presented in the same graphical format. The first two graphs depict histograms of the explanatory variables values and their corresponding probabilities generated by a particular model. The remaining graphs present distinct impurity measures with different parameters. In order to examine and discuss the behavior of the obtained results, we conduct a sensitivity analysis of the algorithms with regard to the entropy parameter values. We also demonstrate how certain explanatory variables affect the process of interactive tree learning.<\/jats:p>","DOI":"10.3390\/e26121020","type":"journal-article","created":{"date-parts":[[2024,11,26]],"date-time":"2024-11-26T07:40:32Z","timestamp":1732606832000},"page":"1020","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Influence of Explanatory Variable Distributions on the Behavior of the Impurity Measures Used in Classification Tree Learning"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6953-8907","authenticated-orcid":false,"given":"Krzysztof","family":"Gajowniczek","sequence":"first","affiliation":[{"name":"Institute of Information Technology, Warsaw University of Life Sciences-SGGW, 02-787 Warszawa, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4242-8411","authenticated-orcid":false,"given":"Marcin","family":"Dudzi\u0144ski","sequence":"additional","affiliation":[{"name":"Institute of Information Technology, Warsaw University of Life Sciences-SGGW, 02-787 Warszawa, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Clopper, C.J., and Pearson, E.S. (1934). The Use of Confidence or Fiducial Limits Illustrated in the Case of the Binomial. Biometrika, 26.","DOI":"10.2307\/2331986"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Breiman, L., Friedman, J.H., Olshen, R.A., and Stone, C.J. (2017). Classification and Regression Trees, Routledge.","DOI":"10.1201\/9781315139470"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1007\/BF00116251","article-title":"Induction of decision trees","volume":"1","author":"Quinlan","year":"1986","journal-title":"Mach. Learn."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Quinlan, J.R. (1993). Pruning Decision Trees. C4.5, Elsevier.","DOI":"10.1016\/B978-0-08-050058-4.50009-7"},{"key":"ref_5","unstructured":"Quinlan, J.R. (2024, October 10). Data Mining Tools See5 and C5.0. Available online: http:\/\/www.rulequest.com\/see5-info.html."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1006\/inco.1996.0040","article-title":"Lower Bounds on Learning Decision Lists and Trees","volume":"126","author":"Hancock","year":"1996","journal-title":"Inf. Comput."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/S0020-7373(87)80053-6","article-title":"Simplifying decision trees","volume":"27","author":"Quinlan","year":"1987","journal-title":"Int. J. Man-Mach. Stud."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1007\/BF00116837","article-title":"An empirical comparison of selection measures for decision-tree induction","volume":"3","author":"Mingers","year":"1989","journal-title":"Mach. Learn."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"476","DOI":"10.1109\/34.589207","article-title":"A comparative analysis of methods for pruning decision trees","volume":"19","author":"Esposito","year":"1997","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ankerst, M., Ester, M., and Kriegel, H.P. (2000, January 20\u201323). Towards an effective cooperation of the user and the computer for classification. Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Boston, MA, USA.","DOI":"10.1145\/347090.347124"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1016\/j.ijhcs.2006.07.005","article-title":"Design and evaluation of visualization support to facilitate decision trees classification","volume":"65","author":"Liu","year":"2007","journal-title":"Int. J. Hum.-Comput. Stud."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"van den Elzen, S., and van Wijk, J.J. (2011, January 23\u201328). BaobabView: Interactive construction and analysis of decision trees. Proceedings of the 2011 IEEE Conference on Visual Analytics Science and Technology (VAST), Providence, RI, USA.","DOI":"10.1109\/VAST.2011.6102453"},{"key":"ref_13","unstructured":"Pauwels, S., Moens, S., and Goethals, B. (2014). Interactive and manual construction of classification trees. BeneLearn, 81."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Poulet, F., and Do, T.N. (2008). Interactive Decision Tree Construction for Interval and Taxonomical Data. Visual Data Mining, Springer.","DOI":"10.1007\/978-3-540-71080-6_9"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Gajowniczek, K., and Z\u0105bkowski, T. (2021). ImbTreeEntropy and ImbTreeAUC: Novel R Packages for Decision Tree Learning on the Imbalanced Datasets. Electronics, 10.","DOI":"10.3390\/electronics10060657"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Gajowniczek, K., and Z\u0105bkowski, T. (2021). Interactive Decision Tree Learning and Decision Rule Extraction Based on the ImbTreeEntropy and ImbTreeAUC Packages. Processes, 9.","DOI":"10.3390\/pr9071107"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"100841","DOI":"10.1016\/j.softx.2021.100841","article-title":"ImbTreeEntropy: An R package for building entropy-based classification trees on imbalanced datasets","volume":"16","author":"Gajowniczek","year":"2021","journal-title":"SoftwareX"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Rokach, L., and Maimon, O. (2005). Decision Trees. Data Mining and Knowledge Discovery Handbook, Springer.","DOI":"10.1007\/0-387-25465-X_9"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kearns, M., and Mansour, Y. (1996, January 22\u201324). On the boosting ability of top-down decision tree learning algorithms. Proceedings of the Twenty-Eighth Annual ACM Symposium on Theory of Computing\u2014STOC \u201996, Philadelphia, PA, USA.","DOI":"10.1145\/237814.237994"},{"key":"ref_20","unstructured":"Dietterich, T., Kearns, M., and Mansour, Y. (July, January 28). Applying the weak learning framework to understand and improve C4.5. Proceedings of the ICML, Garda, Italy."},{"key":"ref_21","unstructured":"Fayyad, U.M., and Irani, K.B. (1992, January 12\u201316). The attribute selection problem in decision tree generation. Proceedings of the Tenth National Conference on Artificial Intelligence, San Jose, CA, USA."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"(1977). Friedman. A Recursive Partitioning Decision Rule for Nonparametric Classification. IEEE Trans. Comput., C-26, 404\u2013408.","DOI":"10.1109\/TC.1977.1674849"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1016\/0031-3203(80)90029-1","article-title":"A combined nonparametric approach to feature selection and binary decision tree design","volume":"12","author":"Rounds","year":"1980","journal-title":"Pattern Recognit."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Lopes, A.M., and Tenreiro Machado, J.A. (2019). Fractional-order modeling of electro-impedance spectroscopy information. Applications in Engineering, Life and Social Sciences, Part A, De Gruyter.","DOI":"10.1515\/9783110571905-002"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"998","DOI":"10.1177\/1077546312473323","article-title":"Fractional order models of leaves","volume":"20","author":"Lopes","year":"2013","journal-title":"J. Vib. Control"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Lopes, A.M., and Machado, J.A.T. (2020). A Review of Fractional Order Entropies. Entropy, 22.","DOI":"10.3390\/e22121374"},{"key":"ref_27","first-page":"460","article-title":"Proposition of a new class of entropy","volume":"38","author":"Akimoto","year":"2001","journal-title":"J. Korean Phys. Soc."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2516","DOI":"10.1016\/j.physleta.2009.05.026","article-title":"Entropies based on fractional calculus","volume":"373","author":"Ubriaco","year":"2009","journal-title":"Phys. Lett. A"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2014\/460364","article-title":"A Fractional Entropy in Fractal Phase Space: Properties and Characterization","volume":"2014","author":"Radhakrishnan","year":"2014","journal-title":"Int. J. Stat. Mech."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"9172","DOI":"10.1016\/j.ijleo.2016.06.119","article-title":"Fractional order entropy: New perspectives","volume":"127","author":"Karci","year":"2016","journal-title":"Optik"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Suthaharan, S. (2016). Decision Tree Learning. Machine Learning Models and Algorithms for Big Data Classification, Springer.","DOI":"10.1007\/978-1-4899-7641-3"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"De la Cruz-Garc\u00eda, J.S., Bory-Reyes, J., and Ramirez-Arellano, A. (2022). A Two-Parameter Fractional Tsallis Decision Tree. Entropy, 24.","DOI":"10.3390\/e24050572"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1111\/j.2517-6161.1968.tb00722.x","article-title":"A Generalization of Bayesian Inference","volume":"30","author":"Dempster","year":"1968","journal-title":"J. R. Stat. Soc. Ser. B Stat. Methodol."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Shafer, G. (1976). A Mathematical Theory of Evidence, Princeton University Press.","DOI":"10.1515\/9780691214696"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Fang, L., Yi, C., and Chong, W. (2009, January 14\u201317). An Evidence Theory Decision Tree Algorithm for Uncertain Data. Proceedings of the 2009 Third International Conference on Genetic and Evolutionary Computing, Guilin, China.","DOI":"10.1109\/WGEC.2009.90"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"113262","DOI":"10.1016\/j.eswa.2020.113262","article-title":"Applying Dempster\u2013Shafer theory for developing a flexible, accurate and interpretable classifier","volume":"148","author":"Baloian","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_38","unstructured":"Weaver, W. (1963). The Mathematical Theory of Communication, University of Illinois Press."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Maszczyk, T., and Duch, W. (2008). Comparison of Shannon, R\u00e9nyi and Tsallis Entropy Used in Decision Trees. Artificial Intelligence and Soft Computing\u2014ICAISC 2008, Proceedings of the 9th International Conference, Zakopane, Poland, 22\u201326 June 2008, Springer.","DOI":"10.1007\/978-3-540-69731-2_62"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Downarowicz, T. (2011). Entropy in Dynamical Systems, Cambridge University Press.","DOI":"10.1017\/CBO9780511976155"},{"key":"ref_41","first-page":"547","article-title":"On measures of entropy and information","volume":"Volume 4","year":"1961","journal-title":"Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1007\/BF01016429","article-title":"Possible generalization of Boltzmann-Gibbs statistics","volume":"52","author":"Tsallis","year":"1988","journal-title":"J. Stat. Phys."},{"key":"ref_43","unstructured":"Hasell, J. (2024, October 10). Measuring Inequality: What Is the Gini Coefficient? Our World in Data. Available online: https:\/\/ourworldindata.org\/what-is-the-gini-coefficient."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"119","DOI":"10.2307\/2986296","article-title":"An Exploratory Technique for Investigating Large Quantities of Categorical Data","volume":"29","author":"Kass","year":"1980","journal-title":"Appl. Stat."},{"key":"ref_45","unstructured":"R Core Team (2024). R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Gajowniczek, K., Or\u0142owski, A., and Z\u0105bkowski, T. (2018). Simulation Study on the Application of the Generalized Entropy Concept in Artificial Neural Networks. Entropy, 20.","DOI":"10.3390\/e20040249"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"A-38","DOI":"10.12693\/APhysPolA.127.A-38","article-title":"Q-Entropy Approach to Selecting High Income Households","volume":"127","author":"Gajowniczek","year":"2015","journal-title":"Acta Phys. Pol. A"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"103473","DOI":"10.1016\/j.artint.2021.103473","article-title":"What do we want from Explainable Artificial Intelligence (XAI)?\u2014A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research","volume":"296","author":"Langer","year":"2021","journal-title":"Artif. Intell."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"100230","DOI":"10.1016\/j.dajour.2023.100230","article-title":"A systematic review of Explainable Artificial Intelligence models and applications: Recent developments and future trends","volume":"7","author":"Saranya","year":"2023","journal-title":"Decis. Anal. J."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"577","DOI":"10.1007\/s11192-013-1032-6","article-title":"On development of fractional calculus during the last fifty years","volume":"98","author":"Machado","year":"2013","journal-title":"Scientometrics"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"119027","DOI":"10.1016\/j.ins.2023.119027","article-title":"A numerical comparative study of uncertainty measures in the Dempster\u2013Shafer evidence theory","volume":"639","author":"Urbani","year":"2023","journal-title":"Inf. Sci."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"127214","DOI":"10.1016\/j.physa.2022.127214","article-title":"A unified formulation of entropy and its application","volume":"596","author":"Balakrishnan","year":"2022","journal-title":"Phys. A Stat. Mech. Its Appl."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Mageed, I.A., and Zhang, Q. (2022, January 1\u20133). An Introductory Survey of Entropy Applications to Information Theory, Queuing Theory, Engineering, Computer Science, and Statistical Mechanics. Proceedings of the 2022 27th International Conference on Automation and Computing (ICAC), Bristol, UK.","DOI":"10.1109\/ICAC55051.2022.9911077"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/12\/1020\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:39:46Z","timestamp":1760114386000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/12\/1020"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,26]]},"references-count":53,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["e26121020"],"URL":"https:\/\/doi.org\/10.3390\/e26121020","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,26]]}}}