{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T21:24:20Z","timestamp":1773955460401,"version":"3.50.1"},"posted":{"date-parts":[[2026]]},"group-title":"SSRN","reference-count":46,"publisher":"Elsevier BV","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:p>Decision trees are a cornerstone of interpretable machine learning and are widely used for their simplicity and effectiveness in classification tasks. To address the growing need for models that can operate on continuous, unbounded data, decision trees have been reinvented for the data stream setting, where they must learn incrementally under constraints such as limited memory, evolving distributions, and delayed supervision. A critical component of these tree-based models, particularly those based on the Hoeffding Trees, is the split criterion, which determines how the input space is partitioned. This study introduces a new split criterion for stream-based Hoeffding trees, based on a unified five-parameter entropic formulation that generalizes several well-known measures, including Shannon, Gini, Tsallis, and R\u00e9nyi entropies. While such formulations have been explored in batch learning, their application to streaming scenarios has not been made. By incorporating this criterion into a variety of established streaming classifiers and evaluating performance on standard benchmark datasets, we demonstrate consistent and statistically significant improvements over existing methods, including those implemented in the River library. Notably, we report gains of up to 40% in immediate evaluation metrics, along with consistent wins and some draws on the prequential Macro-F1, with no observed losses against baseline criteria. The generality of the approach introduces additional computational overhead and also enables greater expressiveness and adaptability in handling uncertainty and nonstationary data. This work advances the integration of information-theoretic principles into online learning and highlights the importance of efficient hyperparameter tuning and adaptive entropy selection in streaming environments.<\/jats:p>","DOI":"10.2139\/ssrn.6442025","type":"posted-content","created":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T14:42:10Z","timestamp":1773931330000},"source":"Crossref","is-referenced-by-count":0,"title":["A Parametric Information-gain to Improve Online Tree-based Machine Learning Models"],"prefix":"10.2139","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3183-7914","authenticated-orcid":true,"given":"Vasco Vieira","family":"Costa","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4995-0539","authenticated-orcid":true,"given":"Diogo","family":"Costa","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7980-0972","authenticated-orcid":true,"given":"Bruno","family":"Veloso","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3628-6795","authenticated-orcid":true,"given":"Eug\u00e9nio  M.","family":"Rocha","sequence":"additional","affiliation":[]}],"member":"78","reference":[{"key":"ref1","doi-asserted-by":"crossref","DOI":"10.1201\/b17320","author":"C C Aggarwal","year":"2014","journal-title":"Data Classification -Algorithms and Applications"},{"key":"ref2","author":"E Akturk","year":"2007","journal-title":"Is Sharma-Mittal entropy really a step beyond Tsallis and Renyi entropies?"},{"key":"ref3","doi-asserted-by":"crossref","DOI":"10.1016\/j.mlwa.2021.100094","article-title":"A comparison among interpretative proposals for random forests","volume":"6","author":"M Aria","year":"2021","journal-title":"Machine Learning with Applications"},{"key":"ref4","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1145\/3605098.3635899","article-title":"Just change on change: Adaptive splitting time for decision trees in data stream classification","author":"D N Assis","year":"2024","journal-title":"Proceedings of the 39th ACM\/SIGAPP Symposium on Applied Computing"},{"key":"ref5","author":"R Ayll\ufffdn-Gavil\ufffdn","year":"2025","journal-title":"Splitting criteria for ordinal decision trees: an experimental study"},{"key":"ref6","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1145\/3297280.3297334","article-title":"Learning regularized hoe!ding trees from data streams","author":"J P Barddal","year":"2019","journal-title":"Proceedings of the 34th ACM\/SIGAPP Symposium on Applied Computing"},{"key":"ref7","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1016\/j.inffus.2019.03.006","article-title":"An overview and comprehensive comparison of ensembles for concept drift","volume":"52","author":"R S M D Barros","year":"2019","journal-title":"Information Fusion"},{"key":"ref8","first-page":"2653","article-title":"Time for a change: a tutorial for comparing multiple classifiers through bayesian analysis","volume":"18","author":"A Benavoli","year":"2017","journal-title":"J. Mach. Learn. Res"},{"key":"ref9","first-page":"249","article-title":"Adaptive learning from evolving data streams","author":"A Bifet","year":"2009","journal-title":"Advances in Intelligent Data Analysis VIII"},{"key":"ref10","author":"L Breiman","year":"1984","journal-title":"Classification and Regression Trees"},{"key":"ref11","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1007\/s10618-011-0222-1","article-title":"Hellinger distance decision trees are robust and skew-insensitive","volume":"24","author":"D A Cieslak","year":"2012","journal-title":"Data Mining and Knowledge Discovery"},{"key":"ref12","article-title":"A new parametric information-gain criterion for tree-based machine learning algorithms","author":"D Costa","year":"2025","journal-title":"PeerJ Computer Science"},{"key":"ref13","doi-asserted-by":"crossref","DOI":"10.1016\/j.jcmds.2024.100098","article-title":"Estimating data complexity and drift through a multiscale generalized impurity approach","volume":"12","author":"D Costa","year":"2024","journal-title":"Journal of Computational Mathematics and Data Science"},{"key":"ref14","doi-asserted-by":"crossref","first-page":"4765","DOI":"10.1007\/s10462-022-10275-5","article-title":"Recent advances in decision trees: an updated survey","volume":"56","author":"V G Costa","year":"2023","journal-title":"Artificial Intelligence Review"},{"key":"ref15","first-page":"1","article-title":"Splitting with confidence in decision trees with application to stream mining","author":"R De Rosa","year":"2015","journal-title":"2015 International Joint Conference on Neural Networks (IJCNN)"},{"key":"ref16","first-page":"71","article-title":"Mining high-speed data streams","author":"P Domingos","year":"2000","journal-title":"Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining"},{"key":"ref17","doi-asserted-by":"crossref","first-page":"39","DOI":"10.15439\/2015F121","article-title":"Comparison of decision trees with R\ufffdnyi and Tsallis entropy applied for imbalanced churn dataset","author":"K Gajowniczek","year":"2015","journal-title":"Proceedings of the 2015 Federated Conference on Computer Science and Information Systems, ACSIS"},{"key":"ref18","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1007\/s10994-012-5320-9","article-title":"On evaluating stream learning algorithms","volume":"90","author":"J Gama","year":"2013","journal-title":"Machine Learning"},{"key":"ref19","doi-asserted-by":"crossref","first-page":"23","DOI":"10.3233\/IDA-2006-10103","article-title":"Decision trees for mining data streams","volume":"10","author":"J A Gama","year":"2006","journal-title":"Intell. Data Anal"},{"key":"ref20","doi-asserted-by":"crossref","first-page":"1469","DOI":"10.1007\/s10994-017-5642-8","article-title":"Adaptive random forests for evolving data stream classification","volume":"106","author":"H M Gomes","year":"2017","journal-title":"Machine Learning"},{"key":"ref21","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1145\/3373464.3373470","article-title":"Machine learning for streaming data: state of the art, challenges, and opportunities","volume":"21","author":"H M Gomes","year":"2019","journal-title":"SIGKDD Explor. Newsl"},{"key":"ref22","first-page":"327","article-title":"8 -classification: Basic concepts","author":"J Han","year":"2012","journal-title":"Data Mining (Third Edition)"},{"key":"ref23","first-page":"2562","article-title":"Dynamic model tree for interpretable data stream learning","author":"J Haug","year":"2022","journal-title":"2022 IEEE 38th International Conference on Data Engineering (ICDE)"},{"key":"ref24","doi-asserted-by":"crossref","DOI":"10.21105\/joss.02173","article-title":"Autorank: A python package for automated ranking of classifiers","volume":"5","author":"S Herbold","year":"2020","journal-title":"Journal of Open Source Software"},{"key":"ref25","article-title":"A practical tutorial for decision tree induction: Evaluation measures for candidate splits and opportunities","volume":"54","author":"V A S Hern\ufffdndez","year":"2021","journal-title":"ACM Comput. Surv"},{"key":"ref26","doi-asserted-by":"crossref","first-page":"6725","DOI":"10.1007\/s40747-024-01524-x","article-title":"Scalable concept drift adaptation for stream data mining","volume":"10","author":"L Hu","year":"2024","journal-title":"Complex & Intelligent Systems"},{"key":"ref27","doi-asserted-by":"crossref","first-page":"62762","DOI":"10.1109\/ACCESS.2020.2985255","article-title":"A new splitting criterion for better interpretable trees","volume":"8","author":"S Hwang","year":"2020","journal-title":"IEEE Access"},{"key":"ref28","article-title":"Random forests with parametric entropy-based information gains for classification and regression problems","author":"V Ignatenko","year":"2024","journal-title":"PeerJ Computer Science"},{"key":"ref29","doi-asserted-by":"crossref","first-page":"2516","DOI":"10.1109\/TNNLS.2017.2698204","article-title":"New splitting criteria for decision trees in stationary data streams","volume":"29","author":"M Jaworski","year":"2018","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"ref30","author":"A Leroux","year":"2018","journal-title":"Information gain ratio correction: Improving prediction with more balanced decision tree splits"},{"key":"ref31","article-title":"Towards improving decision tree induction by combining split evaluation measures","volume":"277","author":"O Loyola-Gonz\ufffdlez","year":"2023","journal-title":"Knowledge-Based Systems"},{"key":"ref32","doi-asserted-by":"crossref","first-page":"1953","DOI":"10.1145\/3219819.3220005","article-title":"Extremely fast decision tree","author":"C Manapragada","year":"2018","journal-title":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining"},{"key":"ref33","first-page":"643","article-title":"Comparison of Shannon, R\ufffdnyi and Tsallis entropy used in decision trees","author":"T Maszczyk","year":"2008","journal-title":"Artificial Intelligence and Soft Computing -ICAISC"},{"key":"ref34","first-page":"298","article-title":"Correcting the usage of the hoe!ding inequality in stream mining","volume":"XII","author":"P Matuszyk","year":"2013","journal-title":"Advances in Intelligent Data Analysis"},{"key":"ref35","doi-asserted-by":"crossref","first-page":"86716","DOI":"10.1109\/ACCESS.2024.3416838","article-title":"A survey of decision trees: Concepts, algorithms, and applications","volume":"12","author":"I D Mienye","year":"2024","journal-title":"IEEE Access"},{"key":"ref36","article-title":"River: machine learning for streaming data in python","author":"J Montiel","year":"2021","journal-title":"Journal of Machine Learning Research"},{"key":"ref37","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1080\/01621459.1963.10500855","article-title":"Problems in the analysis of survey data, and a proposal","volume":"58","author":"J N Morgan","year":"1963","journal-title":"Journal of the American Statistical Association"},{"key":"ref38","article-title":"Behavioral insights of adaptive splitting decision trees in evolving data stream classification","author":"D Nowak Assis","year":"2025","journal-title":"Knowledge and Information Systems"},{"key":"ref39","first-page":"571","article-title":"Improved information gain estimates for decision tree induction","author":"S Nowozin","year":"2012","journal-title":"Proceedings of the 29th International Coference on International Conference on Machine Learning"},{"key":"ref40","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1023\/A:1022643204877","article-title":"Induction of decision trees","volume":"1","author":"J R Quinlan","year":"1986","journal-title":"Machine Learning"},{"key":"ref41","author":"J R Quinlan","year":"1993","journal-title":"C4.5: Programs for Machine Learning"},{"key":"ref42","first-page":"547","article-title":"On measures of entropy and information","author":"A R\ufffdnyi","year":"1960","journal-title":"Proceedings of the fourth Berkeley Symposium on Mathematics, Statistics and Probability"},{"key":"ref43","doi-asserted-by":"crossref","DOI":"10.1016\/j.swevo.2021.101006","article-title":"Induction of decision trees as classification models through metaheuristics","volume":"69","author":"R Rivera-Lopez","year":"2022","journal-title":"Swarm and Evolutionary Computation"},{"key":"ref44","first-page":"379","article-title":"A mathematical theory of communication. The Bell System Technical","volume":"27","author":"C E Shannon","year":"1948","journal-title":"Journal"},{"key":"ref45","doi-asserted-by":"crossref","first-page":"1805","DOI":"10.1007\/s10618-020-00698-5","article-title":"Challenges in benchmarking stream learning algorithms with real-world data","volume":"34","author":"V M A Souza","year":"2020","journal-title":"Data Mining and Knowledge Discovery"},{"key":"ref46","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1007\/BF01016429","article-title":"Possible generalization of Boltzmann-Gibbs statistics","volume":"52","author":"C Tsallis","year":"1988","journal-title":"Journal of Statistical Physics"}],"container-title":[],"original-title":[],"deposited":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T16:54:30Z","timestamp":1773939270000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ssrn.com\/abstract=6442025"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026]]},"references-count":46,"URL":"https:\/\/doi.org\/10.2139\/ssrn.6442025","relation":{},"subject":[],"published":{"date-parts":[[2026]]},"subtype":"preprint"}}