{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T14:48:52Z","timestamp":1780325332286,"version":"3.54.1"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"9-10","license":[{"start":{"date-parts":[[2020,9,1]],"date-time":"2020-09-01T00:00:00Z","timestamp":1598918400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,9,4]],"date-time":"2020-09-04T00:00:00Z","timestamp":1599177600000},"content-version":"vor","delay-in-days":3,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002341","name":"Academy of Finland","doi-asserted-by":"crossref","award":["294238"],"award-info":[{"award-number":["294238"]}],"id":[{"id":"10.13039\/501100002341","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100002341","name":"Academy of Finland","doi-asserted-by":"crossref","award":["319264"],"award-info":[{"award-number":["319264"]}],"id":[{"id":"10.13039\/501100002341","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100002341","name":"Academy of Finland","doi-asserted-by":"crossref","award":["313195"],"award-info":[{"award-number":["313195"]}],"id":[{"id":"10.13039\/501100002341","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Vilho, Yrj\u00f6 and Kalle V\u00e4is\u00e4l\u00e4 Foundation of the Finnish Academy of Science and Letters","award":["2017"],"award-info":[{"award-number":["2017"]}]},{"name":"Foundation for Aalto University Science and Technology","award":["2018"],"award-info":[{"award-number":["2018"]}]},{"DOI":"10.13039\/501100005637","name":"Finnish Foundation for Technology Promotion","doi-asserted-by":"crossref","award":["2019"],"award-info":[{"award-number":["2019"]}],"id":[{"id":"10.13039\/501100005637","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2020,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>A salient approach to interpretable machine learning is to restrict modeling to simple models. In the Bayesian framework, this can be pursued by restricting the model structure and prior to favor interpretable models. Fundamentally, however, interpretability is about users\u2019 preferences, not the data generation mechanism; it is more natural to formulate interpretability as a utility function. In this work, we propose an interpretability utility, which explicates the trade-off between explanation fidelity and interpretability in the Bayesian framework. The method consists of two steps. First, a reference model, possibly a black-box Bayesian predictive model which does not compromise accuracy, is fitted to the training data. Second, a proxy model from an interpretable model family that best mimics the predictive behaviour of the reference model is found by optimizing the interpretability utility function. The approach is model agnostic\u2014neither the interpretable model nor the reference model are restricted to a certain class of models\u2014and the optimization problem can be solved using standard tools. Through experiments on real-word data sets, using decision trees as interpretable models and Bayesian additive regression models as reference models, we show that for the same level of interpretability, our approach generates more accurate models than the alternative of restricting the prior. We also propose a systematic way to measure stability of interpretabile models constructed by different interpretability approaches and show that our proposed approach generates more stable models.<\/jats:p>","DOI":"10.1007\/s10994-020-05901-8","type":"journal-article","created":{"date-parts":[[2020,9,4]],"date-time":"2020-09-04T16:03:41Z","timestamp":1599235421000},"page":"1855-1876","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["A decision-theoretic approach for model interpretability in Bayesian framework"],"prefix":"10.1007","volume":"109","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7639-1501","authenticated-orcid":false,"given":"Homayun","family":"Afrabandpey","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tomi","family":"Peltola","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juho","family":"Piironen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aki","family":"Vehtari","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Samuel","family":"Kaski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,9,4]]},"reference":[{"key":"5901_CR1","unstructured":"Bastani, H., Bastani, O., & Kim, C. (2018). Interpreting predictive models for human-in-the-loop analytics. arXiv preprint arXiv:1705.08504 (pp. 1\u201345)."},{"key":"5901_CR2","unstructured":"Breiman, L., Friedman, J., Stone, C. J., & Olshen, R. A. (1984). Classification and regression trees. CRC press."},{"key":"5901_CR3","unstructured":"Breiman, L., & Shang, N. (1996). Born again trees. Technical report, University of California, Berkeley, Berkeley, CA (Vol. 1, p. 2)."},{"issue":"4","key":"5901_CR4","doi-asserted-by":"crossref","first-page":"1208","DOI":"10.1016\/j.csda.2008.10.033","volume":"53","author":"B Briand","year":"2009","unstructured":"Briand, B., Ducharme, G. R., Parache, V., & Mercat-Rommens, C. (2009). A similarity measure to assess the stability of classification trees. Computational Statistics & Data Analysis, 53(4), 1208\u20131217.","journal-title":"Computational Statistics & Data Analysis"},{"issue":"443","key":"5901_CR5","doi-asserted-by":"crossref","first-page":"935","DOI":"10.1080\/01621459.1998.10473750","volume":"93","author":"HA Chipman","year":"1998","unstructured":"Chipman, H. A., George, E. I., & McCulloch, R. E. (1998). Bayesian CART model search. Journal of the American Statistical Association, 93(443), 935\u2013948.","journal-title":"Journal of the American Statistical Association"},{"issue":"1","key":"5901_CR6","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1214\/09-AOAS285","volume":"4","author":"HA Chipman","year":"2010","unstructured":"Chipman, H. A., George, E. I., & McCulloch, R. E. (2010). BART: Bayesian additive regression trees. The Annals of Applied Statistics, 4(1), 266\u2013298.","journal-title":"The Annals of Applied Statistics"},{"issue":"4","key":"5901_CR7","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1016\/j.dss.2009.05.016","volume":"47","author":"P Cortez","year":"2009","unstructured":"Cortez, P., Cerdeira, A., Almeida, F., Matos, T., & Reis, J. (2009). Modeling wine preferences by data mining from physicochemical properties. Decision Support Systems, 47(4), 547\u2013553.","journal-title":"Decision Support Systems"},{"key":"5901_CR8","unstructured":"Craven, M., & Shavlik, J. W. (1996). Extracting tree-structured representations of trained networks. In Advances in neural information processing systems (pp. 24\u201330)."},{"issue":"4","key":"5901_CR9","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1007\/s41060-018-0144-8","volume":"7","author":"H Deng","year":"2019","unstructured":"Deng, H. (2019). Interpreting tree ensembles with intrees. International Journal of Data Science and Analytics, 7(4), 277\u2013287.","journal-title":"International Journal of Data Science and Analytics"},{"issue":"2","key":"5901_CR10","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1093\/biomet\/85.2.363","volume":"85","author":"DGT Denison","year":"1998","unstructured":"Denison, D. G. T., Mallick, B. K., & Smith, A. F. M. (1998). A Bayesian CART algorithm. Biometrika, 85(2), 363\u2013377.","journal-title":"Biometrika"},{"key":"5901_CR11","unstructured":"Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608."},{"key":"5901_CR12","unstructured":"Du, M., Liu, N., & Hu, X. (2018). Techniques for interpretable machine learning. arXiv preprint arXiv:1808.00033."},{"issue":"2\u20133","key":"5901_CR13","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1007\/s13748-013-0040-3","volume":"2","author":"H Fanaee-T","year":"2014","unstructured":"Fanaee-T, H., & Gama, J. (2014). Event labeling combining ensemble detectors and background knowledge. Progress in Artificial Intelligence, 2(2\u20133), 113\u2013127.","journal-title":"Progress in Artificial Intelligence"},{"key":"5901_CR14","unstructured":"Gal, Y., & Ghahramani, Z. (2016a). Bayesian convolutional neural networks with Bernoulli approximate variational inference. In 4th international conference on learning representations (ICLR) workshop track."},{"key":"5901_CR15","unstructured":"Gal, Y., & Ghahramani, Z. (2016b). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33rd international conference on machine learning (pp. 1050\u20131059)."},{"issue":"19","key":"5901_CR16","doi-asserted-by":"crossref","first-page":"3039","DOI":"10.1002\/sim.7313","volume":"36","author":"J Guo","year":"2017","unstructured":"Guo, J., Riebler, A., & Rue, H. (2017). Bayesian bivariate meta-analysis of diagnostic test studies with interpretable priors. Statistics in Medicine, 36(19), 3039\u20133058.","journal-title":"Statistics in Medicine"},{"key":"5901_CR17","unstructured":"Hara, S., & Hayashi, K. (2018). Making tree ensembles interpretable: A Bayesian model selection approach. In International conference on artificial intelligence and statistics (pp. 77\u201385)."},{"issue":"1","key":"5901_CR18","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/0095-0696(78)90006-2","volume":"5","author":"D Harrison Jr","year":"1978","unstructured":"Harrison, D, Jr., & Rubinfeld, D. L. (1978). Hedonic housing prices and the demand for clean air. Journal of Environmental Economics and Management, 5(1), 81\u2013102.","journal-title":"Journal of Environmental Economics and Management"},{"issue":"4","key":"5901_CR19","doi-asserted-by":"crossref","first-page":"869","DOI":"10.1007\/s11222-017-9767-1","volume":"28","author":"B Hern\u00e1ndez","year":"2018","unstructured":"Hern\u00e1ndez, B., Raftery, A. E., Pennington, S. R., & Parnell, A. C. (2018). Bayesian additive regression trees using Bayesian model averaging. Statistics and Computing, 28(4), 869\u2013890.","journal-title":"Statistics and Computing"},{"issue":"3","key":"5901_CR20","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1080\/00031305.1995.10476165","volume":"49","author":"DC Hoaglin","year":"1995","unstructured":"Hoaglin, D. C., & Velleman, P. F. (1995). A critical look at some analyses of major league baseball salaries. The American Statistician, 49(3), 277\u2013285.","journal-title":"The American Statistician"},{"key":"5901_CR21","doi-asserted-by":"publisher","unstructured":"Johnson, R. W. (1996). Fitting percentage of body fat to simple body measurements. Journal of Statistics Education. https:\/\/doi.org\/10.1080\/10691898.1996.11910505.","DOI":"10.1080\/10691898.1996.11910505"},{"key":"5901_CR22","doi-asserted-by":"crossref","unstructured":"Jung, J., Concannon, C., Shroff, R., Goel, S., & Goldstein, D. G. (2017). Simple rules for complex decisions. arXiv preprint arXiv:1702.04690.","DOI":"10.2139\/ssrn.2919024"},{"issue":"2","key":"5901_CR23","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1111\/j.1467-8640.1989.tb00315.x","volume":"5","author":"D Kibler","year":"1989","unstructured":"Kibler, D., Aha, D. W., & Albert, M. K. (1989). Instance-based prediction of real-valued attributes. Computational Intelligence, 5(2), 51\u201357.","journal-title":"Computational Intelligence"},{"key":"5901_CR24","unstructured":"Kim, B., Glassman, E., Johnson, B., & Shah, J.. (2015). ibcm: Interactive Bayesian case model empowering humans via intuitive interaction. Technical report: MIT-CSAIL-TR."},{"issue":"2","key":"5901_CR25","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1037\/a0029146","volume":"142","author":"JK Kruschke","year":"2013","unstructured":"Kruschke, J. K. (2013). Bayesian estimation supersedes the t test. Journal of Experimental Psychology: General, 142(2), 573.","journal-title":"Journal of Experimental Psychology: General"},{"key":"5901_CR26","doi-asserted-by":"crossref","unstructured":"Kuttichira, D. P., Gupta, S., Li, C., Rana, S., & Venkatesh, S. (2019). Explaining black-box models using interpretable surrogates. In Pacific Rim international conference on artificial intelligence (pp. 3\u201315). Springer.","DOI":"10.1007\/978-3-030-29908-8_1"},{"key":"5901_CR27","unstructured":"Lage, I., Ross, A. S., Kim, B., Gershman, S. J, & Doshi-Velez, F. (2018). Human-in-the-loop interpretability prior. arXiv preprint arXiv:1805.11571."},{"key":"5901_CR28","doi-asserted-by":"crossref","unstructured":"Lakkaraju, H., Bach, S. H, & Leskovec, J. (2016). Interpretable decision sets: A joint framework for description and prediction. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, (pp. 1675\u20131684).","DOI":"10.1145\/2939672.2939874"},{"key":"5901_CR29","doi-asserted-by":"crossref","unstructured":"Lakkaraju, H., Kamar, E., Caruana, R., & Leskovec, J. (2019). Faithful and customizable explanations of black box models. In Proceedings of the 2019 AAAI\/ACM conference on AI, ethics, and society (pp. 131\u2013138).","DOI":"10.1145\/3306618.3314229"},{"issue":"11","key":"5901_CR30","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","volume":"86","author":"Y LeCun","year":"1998","unstructured":"LeCun, Y., Bottou, L., Bengio, Y., Haffner, P., et al. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278\u20132324.","journal-title":"Proceedings of the IEEE"},{"issue":"3","key":"5901_CR31","doi-asserted-by":"crossref","first-page":"1350","DOI":"10.1214\/15-AOAS848","volume":"9","author":"B Letham","year":"2015","unstructured":"Letham, B., Rudin, C., McCormick, T. H., Madigan, D., et al. (2015). Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model. The Annals of Applied Statistics, 9(3), 1350\u20131371.","journal-title":"The Annals of Applied Statistics"},{"issue":"10","key":"5901_CR32","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1145\/3233231","volume":"61","author":"ZC Lipton","year":"2018","unstructured":"Lipton, Z. C. (2018). The mythos of model interpretability. Communications of the ACM, 61(10), 36\u201343.","journal-title":"Communications of the ACM"},{"key":"5901_CR33","doi-asserted-by":"crossref","unstructured":"Lou, Y., Caruana, R., & Gehrke, J. (2012). Intelligible models for classification and regression. In Proceedings of the 18th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 150\u2013158).","DOI":"10.1145\/2339530.2339556"},{"issue":"4","key":"5901_CR34","doi-asserted-by":"crossref","first-page":"2049","DOI":"10.1214\/10-AOAS367","volume":"4","author":"N Meinshausen","year":"2010","unstructured":"Meinshausen, N. (2010). Node harvest. The Annals of Applied Statistics, 4(4), 2049\u20132072.","journal-title":"The Annals of Applied Statistics"},{"key":"5901_CR35","unstructured":"Peltola, T. (2018). Local interpretable model-agnostic explanations of Bayesian predictive models via Kullback\u2013Leibler projections. arXiv preprint arXiv:1810.02678."},{"key":"5901_CR36","unstructured":"Piironen, J., Paasiniemi, M., & Vehtari, A. (2018). Projective inference in high-dimensional problems: Prediction and feature selection. arXiv preprint arXiv:1810.02406."},{"key":"5901_CR37","unstructured":"Popkes, A.-L., Overweg, H., Ercole, A., Li, Y., Hern\u00e1ndez-Lobato, J. M., Zaykov, Y., & Zhang, C. (2019). Interpretable outcome prediction with sparse Bayesian neural networks in intensive care. arXiv preprint arXiv:1905.02599."},{"key":"5901_CR38","doi-asserted-by":"crossref","unstructured":"Quinlan, J. R. (1993). Combining instance-based and model-based learning. In Proceedings of the tenth international conference on machine learning (pp. 236\u2013243).","DOI":"10.1016\/B978-1-55860-307-3.50037-X"},{"key":"5901_CR39","doi-asserted-by":"crossref","unstructured":"Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining (pp. 1135\u20131144). ACM.","DOI":"10.1145\/2939672.2939778"},{"issue":"3","key":"5901_CR40","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1198\/106186004X2165","volume":"13","author":"X Su","year":"2004","unstructured":"Su, X., Wang, M., & Fan, J. (2004). Maximum likelihood regression trees. Journal of Computational and Graphical Statistics, 13(3), 586\u2013598.","journal-title":"Journal of Computational and Graphical Statistics"},{"issue":"13","key":"5901_CR41","doi-asserted-by":"crossref","first-page":"i395","DOI":"10.1093\/bioinformatics\/bty257","volume":"34","author":"I Sundin","year":"2018","unstructured":"Sundin, I., Peltola, T., Micallef, L., Afrabandpey, H., Soare, M., Majumder, M. M., et al. (2018). Improving genomics-based predictions for precision medicine through active elicitation of expert knowledge. Bioinformatics, 34(13), i395\u2013i403.","journal-title":"Bioinformatics"},{"issue":"3","key":"5901_CR42","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1007\/s10994-015-5528-6","volume":"102","author":"B Ustun","year":"2016","unstructured":"Ustun, B., & Rudin, C. (2016). Supersparse linear integer models for optimized medical scoring systems. Machine Learning, 102(3), 349\u2013391.","journal-title":"Machine Learning"},{"key":"5901_CR43","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1214\/12-SS102","volume":"6","author":"A Vehtari","year":"2012","unstructured":"Vehtari, A., & Ojanen, J. (2012). A survey of Bayesian predictive methods for model assessment, selection and comparison. Statistics Surveys, 6, 142\u2013228.","journal-title":"Statistics Surveys"},{"key":"5901_CR44","unstructured":"Wang, T. (2018). Multi-value rule sets for interpretable classification with feature-efficient representations. In Advances in neural information processing systems (pp. 10835\u201310845)."},{"issue":"1","key":"5901_CR45","first-page":"2357","volume":"18","author":"T Wang","year":"2017","unstructured":"Wang, T., Rudin, C., Doshi-Velez, F., Liu, Y., Klampfl, E., & MacNeille, P. (2017). A Bayesian framework for learning rule sets for interpretable classification. The Journal of Machine Learning Research, 18(1), 2357\u20132393.","journal-title":"The Journal of Machine Learning Research"},{"key":"5901_CR46","doi-asserted-by":"crossref","unstructured":"Wu, M., Hughes, M. C, Parbhoo, S., Zazzi, M., Roth, V., Doshi-Velez, F. (2018). Beyond sparsity: Tree regularization of deep models for interpretability. In Thirty-second AAAI conference on artificial intelligence.","DOI":"10.1609\/aaai.v32i1.11501"},{"key":"5901_CR47","unstructured":"Yang, H., Rudin, C., & Seltzer, M. (2017). Scalable Bayesian rule lists. In Proceedings of the 34th international conference on machine learning (Vol. 70, pp. 3921\u20133930). JMLR.org."},{"key":"5901_CR48","unstructured":"Zhou, Y., & Hooker, G. (2016). Interpreting models via single tree approximation. arXiv preprint arXiv:1610.09036."}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-020-05901-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-020-05901-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-020-05901-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,13]],"date-time":"2024-08-13T06:06:55Z","timestamp":1723529215000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-020-05901-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9]]},"references-count":48,"journal-issue":{"issue":"9-10","published-print":{"date-parts":[[2020,9]]}},"alternative-id":["5901"],"URL":"https:\/\/doi.org\/10.1007\/s10994-020-05901-8","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9]]},"assertion":[{"value":"10 January 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 July 2020","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 August 2020","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 September 2020","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}