{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,14]],"date-time":"2025-10-14T00:34:02Z","timestamp":1760402042478,"version":"build-2065373602"},"reference-count":29,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,1,14]],"date-time":"2020-01-14T00:00:00Z","timestamp":1578960000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We describe a classifier made of an ensemble of decision trees, designed using information theory concepts. In contrast to algorithms C4.5 or ID3, the tree is built from the leaves instead of the root. Each tree is made of nodes trained independently of the others, to minimize a local cost function (information bottleneck). The trained tree outputs the estimated probabilities of the classes given the input datum, and the outputs of many trees are combined to decide the class. We show that the system is able to provide results comparable to those of the tree classifier in terms of accuracy, while it shows many advantages in terms of modularity, reduced complexity, and memory requirements.<\/jats:p>","DOI":"10.3390\/e22010100","type":"journal-article","created":{"date-parts":[[2020,1,15]],"date-time":"2020-01-15T10:30:27Z","timestamp":1579084227000},"page":"100","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Probabilistic Ensemble of Deep Information Networks"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4244-2053","authenticated-orcid":false,"given":"Giulio","family":"Franzese","sequence":"first","affiliation":[{"name":"Electronic and Telecommunications, Politecnico di Torino, 10100 Torino, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2152-7314","authenticated-orcid":false,"given":"Monica","family":"Visintin","sequence":"additional","affiliation":[{"name":"Electronic and Telecommunications, Politecnico di Torino, 10100 Torino, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,1,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R., and Friedman, J. (2001). The Elements Of Statistical Learning, Springer.","DOI":"10.1007\/978-0-387-21606-5"},{"key":"ref_2","unstructured":"Murphy, K. (2012). Machine Learning: A Probabilistic Perspective, The MIT Press."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Bergman, M.K. (2018). A Knowledge Representation Practionary, Springer.","DOI":"10.1007\/978-3-319-98092-8"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Rokach, L., and Maimon, O.Z. (2008). Data Mining with Decision Trees: Theory and Applications, World Scientific.","DOI":"10.1142\/9789812771728"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1007\/BF00116251","article-title":"Induction of decision trees","volume":"1","author":"Quinlan","year":"1986","journal-title":"Mach. Learn."},{"key":"ref_6","unstructured":"Quinlan, J. (1993). C4.5: Programs for Machine Learning, Morgan Kaufmann."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1613\/jair.279","article-title":"Improved Use of Continuous Attributes in C4.5","volume":"4","author":"Quinlan","year":"1996","journal-title":"J. Artif. Intell. Res."},{"key":"ref_8","unstructured":"Pearl, J. (2014). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Elsevie."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Barber, D. (2012). Bayesian Reasoning and Machine Learning, Cambridge University Press.","DOI":"10.1017\/CBO9780511804779"},{"key":"ref_10","unstructured":"Jensen, F.V. (1996). Introduction to Bayesian Networks, UCL Press."},{"key":"ref_11","unstructured":"Norouzi, M., Collins, M., Johnson, M.A., Fleet, D.J., and Kohli, P. (2015, January 7\u201312). Efficient Non-greedy Optimization of Decision Trees. Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/BF00058655","article-title":"Bagging Predictors","volume":"24","author":"Breiman","year":"1996","journal-title":"Mach. Learn."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tishby, N., and Zaslavsky, N. (2015). Deep Learning and the Information Bottleneck Principle. arXiv.","DOI":"10.1109\/ITW.2015.7133169"},{"key":"ref_15","unstructured":"Tishby, N., Pereira, F., and Bialek, W. (2000). The Information Bottleneck method. arXiv."},{"key":"ref_16","unstructured":"Franzese, G., and Visintin, M. (2018). Deep Information Networks. arXiv."},{"key":"ref_17","unstructured":"Slonim, N., and Tishby, N. (December, January 29). Agglomerative information bottleneck. Proceedings of the 12th International Conference on Neural Information Processing Systems, Denver, CO, USA."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"968","DOI":"10.3390\/e16020968","article-title":"Information bottleneck approach to predictive inference","volume":"16","author":"Still","year":"2014","journal-title":"Entropy"},{"key":"ref_19","unstructured":"Still, S. (2017). Thermodynamic cost and benefit of data representations. arXiv."},{"key":"ref_20","first-page":"165","article-title":"Information bottleneck for Gaussian variables","volume":"6","author":"Chechik","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"456","DOI":"10.3390\/e14030456","article-title":"The mathematical structure of information bottleneck methods","volume":"14","author":"Gedeon","year":"2012","journal-title":"Entropy"},{"key":"ref_22","first-page":"1612","article-title":"A short introduction to boosting","volume":"14","author":"Freund","year":"1999","journal-title":"Jpn. Soc. Artif. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). Xgboost: A scalable tree boosting system. Proceedings of the 22nd ACM Sigkdd International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1109\/TIT.1972.1054753","article-title":"An algorithm for computing the capacity of arbitrary discrete memoryless channels","volume":"18","author":"Arimoto","year":"1972","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"460","DOI":"10.1109\/TIT.1972.1054855","article-title":"Computation of channel capacity and rate-distortion functions","volume":"18","author":"Blahut","year":"1972","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_26","first-page":"385","article-title":"Idiot\u2019s Bayes\u2014Not so stupid after all?","volume":"69","author":"Hand","year":"2001","journal-title":"Int. Stat. Rev."},{"key":"ref_27","unstructured":"(2010, September 30). UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences. Available online: http:\/\/archive.ics.uci.edu\/ml."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Salekin, A., and Stankovic, J. (2016, January 4\u20137). Detection of chronic kidney disease and selecting important predictive attributes. Proceedings of the IEEE International Conference on Healthcare Informatics (ICHI), Chicago, IL, USA.","DOI":"10.1109\/ICHI.2016.36"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1023\/A:1009670302979","article-title":"Extraction of logical rules from neural networks","volume":"7","author":"Duch","year":"1998","journal-title":"Neural Process. Lett."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/1\/100\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:19:44Z","timestamp":1760361584000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/1\/100"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,14]]},"references-count":29,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,1]]}},"alternative-id":["e22010100"],"URL":"https:\/\/doi.org\/10.3390\/e22010100","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2020,1,14]]}}}