{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,22]],"date-time":"2025-12-22T08:14:17Z","timestamp":1766391257855,"version":"build-2065373602"},"reference-count":40,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2021,11,23]],"date-time":"2021-11-23T00:00:00Z","timestamp":1637625600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>In machine learning, ensembles of models based on Multi-Layer Perceptrons (MLPs) or decision trees are considered successful models. However, explaining their responses is a complex problem that requires the creation of new methods of interpretation. A natural way to explain the classifications of the models is to transform them into propositional rules. In this work, we focus on random forests and gradient-boosted trees. Specifically, these models are converted into an ensemble of interpretable MLPs from which propositional rules are produced. The rule extraction method presented here allows one to precisely locate the discriminating hyperplanes that constitute the antecedents of the rules. In experiments based on eight classification problems, we compared our rule extraction technique to \u201cSkope-Rules\u201d and other state-of-the-art techniques. Experiments were performed with ten-fold cross-validation trials, with propositional rules that were also generated from ensembles of interpretable MLPs. By evaluating the characteristics of the extracted rules in terms of complexity, fidelity, and accuracy, the results obtained showed that our rule extraction technique is competitive. To the best of our knowledge, this is one of the few works showing a rule extraction technique that has been applied to both ensembles of decision trees and neural networks.<\/jats:p>","DOI":"10.3390\/a14120339","type":"journal-article","created":{"date-parts":[[2021,11,24]],"date-time":"2021-11-24T02:46:08Z","timestamp":1637721968000},"page":"339","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["A Rule Extraction Technique Applied to Ensembles of Neural Networks, Random Forests, and Gradient-Boosted Trees"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6070-3459","authenticated-orcid":false,"given":"Guido","family":"Bologna","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Applied Sciences and Arts of Western Switzerland, Rue de la Prairie 4, 1202 Geneva, Switzerland"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1016\/0950-7051(96)81920-4","article-title":"Survey and critique of techniques for extracting rules from trained artificial neural networks","volume":"8","author":"Andrews","year":"1995","journal-title":"Knowl. Based Syst."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Diederich, J. (2008). Rule Extraction from Support Vector Machines, Springer Science & Business Media.","DOI":"10.1007\/978-3-540-75390-2"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/BF00058655","article-title":"Bagging predictors","volume":"24","author":"Breiman","year":"1996","journal-title":"Mach. Learn."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Freund, Y., and Schapire, R.E. (1995, January 13\u201315). A desicion-theoretic generalization of on-line learning and an application to boosting. Proceedings of the European Conference on Computational Learning Theory, Barcelona, Spain.","DOI":"10.1007\/3-540-59119-2_166"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.inffus.2004.04.004","article-title":"Diversity creation methods: A survey and categorisation","volume":"6","author":"Brown","year":"2005","journal-title":"Inf. Fusion"},{"key":"ref_6","unstructured":"Bologna, G. (1998). Symbolic rule extraction from the DIMLP neural network. International Workshop on Hybrid Neural Systems, Springer."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_8","unstructured":"Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer Science & Business Media."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI","volume":"58","author":"Arrieta","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Saito, K., and Nakano, R. (1988, January 24\u201327). Medical diagnostic expert system based on PDP model. Proceedings of the IEEE 1988 International Conference on Neural Networks, San Diego, CA, USA.","DOI":"10.1109\/ICNN.1988.23855"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1142\/S0129065701000680","article-title":"A study on rule extraction from several combined neural networks","volume":"11","author":"Bologna","year":"2001","journal-title":"Int. J. Neural Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1016\/j.jal.2004.03.004","article-title":"Is it worth generating rules from neural network ensembles?","volume":"2","author":"Bologna","year":"2004","journal-title":"J. Appl. Log."},{"key":"ref_13","first-page":"4084850","article-title":"A comparison study on rule extraction from neural network ensembles, boosted shallow trees, and SVMs","volume":"2018","author":"Bologna","year":"2018","journal-title":"Appl. Comput. Intell. Soft Comput."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Bologna, G. (2021, January 17\u201320). Transparent Ensembles for COVID-19 Prognosis. Proceedings of the International Cross-Domain Conference for Machine Learning and Knowledge Extraction, Online.","DOI":"10.1007\/978-3-030-84060-0_22"},{"key":"ref_15","first-page":"3","article-title":"Extracting symbolic rules from trained neural network ensembles","volume":"16","author":"Zhou","year":"2003","journal-title":"Artif. Intell. Commun."},{"key":"ref_16","unstructured":"Johansson, U. (2007). Obtaining Accurate and Comprehensible Data Mining Models: An Evolutionary Approach, Department of Computer and Information Science, Link\u00f6ping University."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Hara, A., and Hayashi, Y. (2012, January 10\u201315). Ensemble neural network rule extraction using Re-RX algorithm. Proceedings of the 2012 International Joint Conference on Neural Networks(IJCNN), Brisbane, QLD, Australia.","DOI":"10.1109\/IJCNN.2012.6252446"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Hayashi, Y., Sato, R., and Mitra, S. (2013, January 4\u20139). A new approach to three ensemble neural network rule extraction using recursive-rule extraction algorithm. Proceedings of the 2013 International Joint Conference on Neural Networks (IJCNN), Dallas, TX, USA.","DOI":"10.1109\/IJCNN.2013.6706823"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1016\/j.procs.2019.09.182","article-title":"A new transparent ensemble method based on deep learning","volume":"159","author":"Sendi","year":"2019","journal-title":"Procedia Comput. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"916","DOI":"10.1214\/07-AOAS148","article-title":"Predictive learning via rule ensembles","volume":"2","author":"Friedman","year":"2008","journal-title":"Ann. Appl. Stat."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2049","DOI":"10.1214\/10-AOAS367","article-title":"Node harvest","volume":"4","author":"Meinshausen","year":"2010","journal-title":"Ann. Appl. Stat."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1707","DOI":"10.1142\/S0219622017500055","article-title":"Rule Extraction from Decision Trees Ensembles: New Algorithms Based on Heuristic Search and Sparse Group Lasso Methods","volume":"16","author":"Mashayekhi","year":"2017","journal-title":"Int. J. Inf. Technol. Decis. Mak."},{"key":"ref_23","unstructured":"Friedman, J., Hastie, T., and Tibshirani, R. (2010). A note on the group Lasso and a sparse group Lasso. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.inffus.2020.03.013","article-title":"Explainable decision forest: Transforming a decision forest into an interpretable tree","volume":"61","author":"Sagi","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1007\/s41060-018-0144-8","article-title":"Interpreting tree ensembles with intrees","volume":"7","author":"Deng","year":"2019","journal-title":"Int. J. Data Sci. Anal."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1007\/BF00993309","article-title":"C4.5: Programs for Machine Learning by J. Ross Quinlan. Morgan Kaufmann Publishers, Inc., 1993","volume":"16","author":"Salzberg","year":"1994","journal-title":"Mach. Learn."},{"key":"ref_27","unstructured":"Breiman, L., Friedman, J., Stone, C.J., and Olshen, R.A. (1984). Classification and Regression Trees, CRC Press."},{"key":"ref_28","first-page":"1401","article-title":"A brief introduction to boosting","volume":"99","author":"Schapire","year":"1999","journal-title":"Ijcai"},{"key":"ref_29","unstructured":"Bologna, G., and Pellegrini, C. (1998, January 4\u20139). Constraining the MLP power of expression to facilitate symbolic rule extraction. Proceedings of the 1998 IEEE International Joint Conference on Neural Networks Proceedings, IEEE World Congress on Computational Intelligence (Cat. No.98CH36227), Anchorage, AK, USA."},{"key":"ref_30","unstructured":"Brayton, R., Hachtel, G., Hemachandra, L., Newton, A., and Sangiovanni-Vincentelli, A. (1982, January 10\u201312). A comparison of logic minimization strategies using ESPRESSO: An APL program package for partitioned logic minimization. Proceedings of the International Symposium on Circuits and Systems, Rome, Italy."},{"key":"ref_31","unstructured":"Lichman, M. (2013). UCI Machine Learning Repository, University of California, School of Information and Computer Sciences."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/S0020-7373(87)80053-6","article-title":"Simplifying decision trees","volume":"27","author":"Quinlan","year":"1987","journal-title":"Int. J. Man-Mach. Stud."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"9193","DOI":"10.1073\/pnas.87.23.9193","article-title":"Multisurface method of pattern separation for medical diagnosis applied to breast cytology","volume":"87","author":"Wolberg","year":"1990","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_34","first-page":"259","article-title":"Divorce prediction using correlation based feature selection and artificial neural networks","volume":"9","author":"Kemal","year":"2019","journal-title":"Nev\u015fehir Hac\u0131 Bekta\u015f Veli \u00dcniversitesi SBE Dergisi"},{"key":"ref_35","first-page":"262","article-title":"Classification of radar returns from the ionosphere using neural networks","volume":"10","author":"Sigillito","year":"1989","journal-title":"Johns Hopkins APL Tech. Dig."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"4164","DOI":"10.1118\/1.2786864","article-title":"The prediction of breast cancer biopsy outcomes using two CAD approaches that both emphasize an intelligible decision process","volume":"34","author":"Elter","year":"2007","journal-title":"Med. Phys."},{"key":"ref_37","unstructured":"Cortez, P., and Silva, A.M.G. (2008, January 9\u201311). Using data mining to predict secondary school student performance. Proceedings of the 5th Annual Future Business Technology Conference, Porto, Portugal."},{"key":"ref_38","unstructured":"Schlimmer, J.C. (1987). Concept Acquisition through Representational Adjustment. [Ph.D. Thesis, University of California]."},{"key":"ref_39","first-page":"2825","article-title":"Scikit-learn: Machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1016\/j.patcog.2017.05.025","article-title":"Handcrafted vs. non-handcrafted features for computer vision classification","volume":"71","author":"Nanni","year":"2017","journal-title":"Pattern Recognit."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/12\/339\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:34:29Z","timestamp":1760168069000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/12\/339"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,23]]},"references-count":40,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["a14120339"],"URL":"https:\/\/doi.org\/10.3390\/a14120339","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2021,11,23]]}}}