{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,16]],"date-time":"2026-02-16T09:19:27Z","timestamp":1771233567772,"version":"3.50.1"},"reference-count":126,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2023,1,18]],"date-time":"2023-01-18T00:00:00Z","timestamp":1674000000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Information Visualization"],"published-print":{"date-parts":[[2023,4]]},"abstract":"<jats:p> Bagging and boosting are two popular ensemble methods in machine learning (ML) that produce many individual decision trees. Due to the inherent ensemble characteristic of these methods, they typically outperform single decision trees or other ML models in predictive performance. However, numerous decision paths are generated for each decision tree, increasing the overall complexity of the model and hindering its use in domains that require trustworthy and explainable decisions, such as finance, social care, and health care. Thus, the interpretability of bagging and boosting algorithms\u2014such as random forest and adaptive boosting\u2014reduces as the number of decisions rises. In this paper, we propose a visual analytics tool that aims to assist users in extracting decisions from such ML models via a thorough visual inspection workflow that includes selecting a set of robust and diverse models (originating from different ensemble learning algorithms), choosing important features according to their global contribution, and deciding which decisions are essential for global explanation (or locally, for specific cases). The outcome is a final decision based on the class agreement of several models and the explored manual decisions exported by users. We evaluated the applicability and effectiveness of VisRuler via a use case, a usage scenario, and a user study. The evaluation revealed that most users managed to successfully use our system to explore decision rules visually, performing the proposed tasks and answering the given questions in a satisfying way. <\/jats:p>","DOI":"10.1177\/14738716221142005","type":"journal-article","created":{"date-parts":[[2023,1,18]],"date-time":"2023-01-18T11:44:40Z","timestamp":1674042280000},"page":"115-139","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":13,"title":["VisRuler: Visual analytics for extracting decision rules from bagged and boosted decision trees"],"prefix":"10.1177","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9079-2376","authenticated-orcid":false,"given":"Angelos","family":"Chatzimparmpas","sequence":"first","affiliation":[{"name":"Department of Computer Science and Media Technology, Linnaeus University, V\u00e4xj\u00f6, Sweden"}]},{"given":"Rafael M.","family":"Martins","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Media Technology, Linnaeus University, V\u00e4xj\u00f6, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0519-2537","authenticated-orcid":false,"given":"Andreas","family":"Kerren","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Media Technology, Linnaeus University, V\u00e4xj\u00f6, Sweden"},{"name":"Department of Science and Technology, Link\u00f6ping University, Norrk\u00f6ping, Sweden"}]}],"member":"179","published-online":{"date-parts":[[2023,1,18]]},"reference":[{"key":"bibr1-14738716221142005","volume-title":"Ensemble learning","author":"Zhou ZH.","year":"2009"},{"key":"bibr2-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1249"},{"key":"bibr3-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1007\/BF00117832"},{"key":"bibr4-14738716221142005","first-page":"148","volume-title":"Proceedings of the 13th International Conference on Machine Learning","author":"Freund Y","year":"1996"},{"key":"bibr5-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1007\/BF00116037"},{"key":"bibr6-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80023-1"},{"key":"bibr7-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1038\/nbt0908-1011"},{"key":"bibr8-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"issue":"5","key":"bibr9-14738716221142005","first-page":"771","volume":"14","author":"Freund Y","year":"1999","journal-title":"J Japan Soc Artif Intell"},{"key":"bibr10-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1613\/jair.614"},{"issue":"1","key":"bibr11-14738716221142005","first-page":"1558","volume":"18","author":"Wyner AJ","year":"2017","journal-title":"J Mach Learn Res"},{"issue":"1","key":"bibr12-14738716221142005","first-page":"3133","volume":"15","author":"Fernandez-Delgado M","year":"2014","journal-title":"J Mach Learn Res"},{"key":"bibr13-14738716221142005","unstructured":"Dua D, Graff C. UCI machine learning repository, http:\/\/archive.ics.uci.edu\/ml (2017, accessed 12 November 2022)."},{"key":"bibr14-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1214\/ss\/1009213726"},{"key":"bibr15-14738716221142005","first-page":"1721","volume-title":"Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining","author":"Caruana R"},{"key":"bibr16-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2016.2598829"},{"key":"bibr17-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-90403-0_1"},{"key":"bibr18-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"bibr19-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-21606-5"},{"key":"bibr20-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939874"},{"key":"bibr21-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2019.113100"},{"key":"bibr22-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007515423169"},{"key":"bibr23-14738716221142005","unstructured":"Kotsiantis SB, Pintelas PE. Combining bagging and boosting (2007, accessed 12 November 2022)."},{"key":"bibr24-14738716221142005","first-page":"108","volume-title":"Artificial Intelligence in medicine: knowledge representation and transparent and explainable systems","author":"Kopitar L","year":"2019"},{"key":"bibr25-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1145\/3236386.3241340"},{"key":"bibr26-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1145\/3359786"},{"key":"bibr27-14738716221142005","doi-asserted-by":"publisher","DOI":"10.3390\/electronics8080832"},{"key":"bibr28-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-28954-6_2"},{"key":"bibr29-14738716221142005","first-page":"1952","volume":"2","author":"Kim B","year":"2014","journal-title":"Adv Neural Inform Process Syst"},{"key":"bibr30-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3045560"},{"key":"bibr31-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2864475"},{"key":"bibr32-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030354"},{"key":"bibr33-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14452"},{"key":"bibr34-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2019.00037"},{"key":"bibr35-14738716221142005","author":"Neto MP","journal-title":"IEEE Trans Vis Comput Graph"},{"key":"bibr36-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2017.2744378"},{"key":"bibr37-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1007\/s12650-018-0514-2"},{"key":"bibr38-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/PacificVis52677.2021.00032"},{"key":"bibr39-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-88207-5_7"},{"key":"bibr40-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1013203451"},{"key":"bibr41-14738716221142005","first-page":"2579","volume":"9","author":"van der Maaten L","year":"2008","journal-title":"J Mach Learn Res"},{"key":"bibr42-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/VAST.2011.6102453"},{"key":"bibr43-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TAI.2000.889842"},{"key":"bibr44-14738716221142005","author":"Lee T","year":"2016","journal-title":"arXiv:1610.05463"},{"key":"bibr45-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2864477"},{"key":"bibr46-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/INFVIS.2001.963292"},{"key":"bibr47-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1017\/S1930297500006239"},{"key":"bibr48-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/VAST.2011.6102439"},{"key":"bibr49-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1057\/palgrave.ivs.9500065"},{"key":"bibr50-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882291"},{"key":"bibr51-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/VAST.2014.7042480"},{"key":"bibr52-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2017.2745158"},{"key":"bibr53-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.231"},{"key":"bibr54-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1016\/j.cag.2014.02.004"},{"key":"bibr55-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1145\/347090.347124"},{"key":"bibr56-14738716221142005","first-page":"178","volume-title":"Proceedings of the 3rd SIAM international conference on data mining","author":"Teoh ST"},{"key":"bibr57-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1145\/956750.956837"},{"key":"bibr58-14738716221142005","author":"Do TN.","year":"2007","journal-title":"Technical Report 06\u201301, College of Information Technology, Cantho University, Cantho, Vietnam"},{"key":"bibr59-14738716221142005","author":"Yuan J","year":"2022","journal-title":"ArXiv e-prints"},{"key":"bibr60-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/VIS47514.2020.00058"},{"key":"bibr61-14738716221142005","volume-title":"CEUR Workshop Proc","author":"Di Castro F","year":"2019"},{"key":"bibr62-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1145\/347090.347139"},{"key":"bibr63-14738716221142005","author":"Deng J","journal-title":"Proceedings of EuroVis 2021 short papers"},{"key":"bibr64-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1006\/ijhc.2001.0499"},{"key":"bibr65-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/VIS49827.2021.9623303"},{"key":"bibr66-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/SIBGRAPI.2014.14"},{"key":"bibr67-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1016\/j.bdr.2021.100239"},{"key":"bibr68-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2864812"},{"key":"bibr69-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1007\/s12650-019-00607-z"},{"key":"bibr70-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/MLUI54255.2021.00008"},{"key":"bibr71-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0012078"},{"key":"bibr72-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.0050120"},{"key":"bibr73-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1016\/j.visinf.2020.04.005"},{"key":"bibr74-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2014.02.040"},{"key":"bibr75-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1000225"},{"key":"bibr76-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/VISUAL.2019.8933596"},{"key":"bibr77-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0100736"},{"key":"bibr78-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/VIZSEC.2016.7739577"},{"key":"bibr79-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0078433"},{"key":"bibr80-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030352"},{"key":"bibr81-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14300"},{"key":"bibr82-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2864825"},{"key":"bibr83-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TBDATA.2018.2877350"},{"key":"bibr84-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518895"},{"key":"bibr85-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2864499"},{"key":"bibr86-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2016.2598828"},{"key":"bibr87-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13972"},{"key":"bibr88-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13970"},{"key":"bibr89-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030361"},{"key":"bibr90-14738716221142005","unstructured":"AutoML. Google cloud AutoML, https:\/\/cloud.google.com\/automl\/ (accessed 12 November 2022)."},{"key":"bibr91-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.125"},{"key":"bibr92-14738716221142005","first-page":"13","volume-title":"Proceedings of the EuroVis workshop on visual analytics","author":"Sehgal G","year":"2018"},{"key":"bibr93-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12389"},{"key":"bibr94-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2019.2922592"},{"key":"bibr95-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2010.03.014"},{"key":"bibr96-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-74827-4_62"},{"key":"bibr97-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-010-9192-8"},{"key":"bibr98-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1177\/1473871620904671"},{"key":"bibr99-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14034"},{"key":"bibr100-14738716221142005","first-page":"2238","volume":"26","author":"Zou JY","year":"2013","journal-title":"Adv Neural Inf Process Syst"},{"key":"bibr101-14738716221142005","unstructured":"Angelos Chatzimparmpas. VisRuler code, https:\/\/github.com\/angeloschatzimparmpas\/VisRuler (2022, accessed 12 November 2022)"},{"key":"bibr102-14738716221142005","first-page":"281","volume":"13","author":"Bergstra J","year":"2012","journal-title":"J Mach Learn Res"},{"key":"bibr103-14738716221142005","volume-title":"World happiness report 2019","author":"John FH","year":"2019"},{"key":"bibr104-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/SECon.2012.6196887"},{"key":"bibr105-14738716221142005","first-page":"173","volume-title":"Proceedings of the 2005 international conference on subspace, latent structure and feature selection","author":"Rogers J","year":"2005"},{"key":"bibr106-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1016\/j.phpro.2012.03.160"},{"key":"bibr107-14738716221142005","author":"McInnes L","year":"2018","journal-title":"arXiv:1802.03426"},{"key":"bibr108-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2944182"},{"key":"bibr109-14738716221142005","first-page":"226","volume-title":"Proceedings of the second international conference on knowledge discovery and data mining","author":"Ester M","year":"1996"},{"key":"bibr110-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102422"},{"key":"bibr111-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1111\/j.1469-1809.1936.tb02137.x"},{"key":"bibr112-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2865146"},{"key":"bibr113-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-006-6226-1"},{"key":"bibr114-14738716221142005","first-page":"3149","volume-title":"Proceedings of the 31st international conference on neural information processing systems","author":"Ke G","year":"2017"},{"key":"bibr115-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"bibr116-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/ICBDA.2016.7509789"},{"key":"bibr117-14738716221142005","volume-title":"Information visualization: perception for design","author":"Ware C.","year":"2019"},{"key":"bibr118-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1057\/ivs.2009.23"},{"key":"bibr119-14738716221142005","author":"Claesen M","journal-title":"Proceedings of the MIC"},{"key":"bibr120-14738716221142005","author":"Claesen M","year":"2014","journal-title":"arXiv:1412-1114"},{"key":"bibr121-14738716221142005","author":"Li T","year":"2018","journal-title":"Proceedings of the machine learning from user interaction for visualization and analytics workshop at IEEE VIS"},{"key":"bibr122-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.65"},{"key":"bibr123-14738716221142005","author":"Hilasaca GM","year":"2019","journal-title":"arXiv:1903-06262"},{"key":"bibr124-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/PACIFICVIS.2017.8031609"},{"key":"bibr125-14738716221142005","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346574"},{"key":"bibr126-14738716221142005","author":"Turkay C","year":"2018","journal-title":"arXiv:1812.08032"}],"container-title":["Information Visualization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/14738716221142005","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/14738716221142005","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/14738716221142005","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,2]],"date-time":"2025-03-02T00:05:20Z","timestamp":1740873920000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/14738716221142005"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,18]]},"references-count":126,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,4]]}},"alternative-id":["10.1177\/14738716221142005"],"URL":"https:\/\/doi.org\/10.1177\/14738716221142005","relation":{},"ISSN":["1473-8716","1473-8724"],"issn-type":[{"value":"1473-8716","type":"print"},{"value":"1473-8724","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,18]]}}}