{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T08:36:04Z","timestamp":1771576564474,"version":"3.50.1"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2022,6,18]],"date-time":"2022-06-18T00:00:00Z","timestamp":1655510400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,6,18]],"date-time":"2022-06-18T00:00:00Z","timestamp":1655510400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Swiss Federal Institute of Technology Zurich"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Data Min Knowl Disc"],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>A vast and growing literature on explaining deep learning models has emerged. This paper contributes to that literature by introducing a global gradient-based model-agnostic method, which we call Marginal Attribution by Conditioning on Quantiles (MACQ). Our approach is based on analyzing the marginal attribution of predictions (outputs) to individual features (inputs). Specifically, we consider variable importance by fixing (global) output levels, and explaining how features marginally contribute to these fixed global output levels. MACQ can be seen as a marginal attribution counterpart to approaches such as accumulated local effects, which study the sensitivities of outputs by perturbing inputs. Furthermore, MACQ allows us to separate marginal attribution of individual features from interaction effects and to visualize the 3-way relationship between marginal attribution, output level, and feature value.\n<\/jats:p>","DOI":"10.1007\/s10618-022-00841-4","type":"journal-article","created":{"date-parts":[[2022,6,18]],"date-time":"2022-06-18T05:02:43Z","timestamp":1655528563000},"page":"1335-1370","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Interpreting deep learning models with marginal attribution by conditioning on quantiles"],"prefix":"10.1007","volume":"36","author":[{"given":"Michael","family":"Merz","sequence":"first","affiliation":[]},{"given":"Ronald","family":"Richman","sequence":"additional","affiliation":[]},{"given":"Andreas","family":"Tsanakas","sequence":"additional","affiliation":[]},{"given":"Mario V.","family":"W\u00fcthrich","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,6,18]]},"reference":[{"key":"841_CR1","unstructured":"Abadi M et al (2015) TensorFlow: large-scale machine learning on heterogeneous systems. https:\/\/www.tensorflow.org\/"},{"key":"841_CR2","doi-asserted-by":"publisher","first-page":"1505","DOI":"10.1016\/S0378-4266(02)00281-9","volume":"7","author":"C Acerbi","year":"2002","unstructured":"Acerbi C (2002) Spectral measures of risk: a coherent representation of subjective risk aversion. J Bank Finance 7:1505\u20131518","journal-title":"J Bank Finance"},{"key":"841_CR3","doi-asserted-by":"crossref","unstructured":"Ancona M, Ceolini E, \u00d6ztireli C, Gross M (2019) Gradient-based attribution methods. In: Samek W, Montavon G, Vedaldi A, Hansen LK, M\u00fcller K-R (eds) Explainable AI: interpreting, explaining and visualizing deep learning, lecture notes in artificial intelligence 11700. Springer, pp 168\u2013191","DOI":"10.1007\/978-3-030-28954-6_9"},{"issue":"4","key":"841_CR4","doi-asserted-by":"publisher","first-page":"1059","DOI":"10.1111\/rssb.12377","volume":"82","author":"DW Apley","year":"2020","unstructured":"Apley DW, Zhu J (2020) Visualizing the effects of predictor variables in black box supervised learning models. J R Stat Soc Ser B 82(4):1059\u20131086","journal-title":"J R Stat Soc Ser B"},{"key":"841_CR5","first-page":"1137","volume":"3","author":"Y Bengio","year":"2003","unstructured":"Bengio Y, Ducharme R, Vincent P, Jauvin C (2003) A neural probabilistic language model. J Mach Learn Res 3:1137\u20131155","journal-title":"J Mach Learn Res"},{"key":"841_CR6","doi-asserted-by":"crossref","unstructured":"Binder A, Bach S, Montavon G, M\u00fcller K-R, Samek W (2016) Layer-wise relevance propagation for deep neural network architectures. In: Kim K, Joukov N (eds) Information science and applications (ICISA), lecture notes in electrical engineering 376. Springer","DOI":"10.1007\/978-981-10-0557-2_87"},{"issue":"1","key":"841_CR7","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L (2001) Random forests. Mach Learn 45(1):5\u201332","journal-title":"Mach Learn"},{"key":"841_CR8","unstructured":"Chollet F et al (2015) Keras. https:\/\/github.com\/fchollet\/keras"},{"key":"841_CR9","unstructured":"Dietterich TG (2000a) An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Mach Learn 40(2):139\u2013157"},{"key":"841_CR10","doi-asserted-by":"crossref","unstructured":"Dietterich TG (2000b) Ensemble methods in machine learning. In: Kittel J, Roli F (eds) Multiple classifier systems, lecture notes in computer science, 1857. Springer, pp 1\u201315","DOI":"10.1007\/3-540-45014-9_1"},{"issue":"S1","key":"841_CR11","doi-asserted-by":"publisher","first-page":"S28","DOI":"10.1111\/insr.12409","volume":"88","author":"B Efron","year":"2020","unstructured":"Efron B (2020) Prediction, estimation and attribution. Int Stat Rev 88(S1):S28\u2013S59","journal-title":"Int Stat Rev"},{"key":"841_CR12","doi-asserted-by":"publisher","first-page":"113","DOI":"10.1007\/s13748-013-0040-3","volume":"2","author":"H Fanaee-T","year":"2014","unstructured":"Fanaee-T H, Gama J (2014) Event labeling combining ensemble detectors and background knowledge. Prog Artif Intell 2:113\u2013127","journal-title":"Prog Artif Intell"},{"issue":"5","key":"841_CR13","doi-asserted-by":"publisher","first-page":"1189","DOI":"10.1214\/aos\/1013203451","volume":"29","author":"JH Friedman","year":"2001","unstructured":"Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat 29(5):1189\u20131232","journal-title":"Ann Stat"},{"issue":"3","key":"841_CR14","doi-asserted-by":"publisher","first-page":"916","DOI":"10.1214\/07-AOAS148","volume":"2","author":"JH Friedman","year":"2008","unstructured":"Friedman JH, Popescu BE (2008) Predictive learning via rule ensembles. Ann Appl Stat 2(3):916\u2013954","journal-title":"Ann Appl Stat"},{"issue":"1","key":"841_CR15","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1080\/10618600.2014.907095","volume":"24","author":"A Goldstein","year":"2015","unstructured":"Goldstein A, Kapelner A, Bleich J, Pitkin E (2015) Peeking inside the black box: visualizing statistical learning with plots of individual conditional expectation. J Comput Graph Stat 24(1):44\u201365","journal-title":"J Comput Graph Stat"},{"key":"841_CR16","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1016\/S0927-5398(00)00011-6","volume":"7","author":"C Gourieroux","year":"2000","unstructured":"Gourieroux C, Laurent JP, Scaillet O (2000) Sensitivity analysis of values at risk. J Empir Finance 7:225\u2013245","journal-title":"J Empir Finance"},{"key":"841_CR17","unstructured":"Guo C, Berkhahn F (2016) Entity embeddings of categorical variables. arXiv:1604.06737"},{"issue":"1","key":"841_CR18","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1287\/opre.1080.0531","volume":"57","author":"LJ Hong","year":"2009","unstructured":"Hong LJ (2009) Estimating quantile sensitivities. Oper Res 57(1):118\u2013130","journal-title":"Oper Res"},{"issue":"1","key":"841_CR19","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1017\/asb.2021.23","volume":"52","author":"M Lindholm","year":"2022","unstructured":"Lindholm M, Richman R, Tsanakas A, W\u00fcthrich MV (2022) Discrimination-free insurance pricing. ASTIN Bull 52(1):55\u201389","journal-title":"ASTIN Bull"},{"key":"841_CR20","unstructured":"Loader C, Sun J, Technologies Lucent, Liaw A (2020) locfit: local regression, likelihood and density estimation. R package version 1.5-9.4"},{"key":"841_CR21","first-page":"4765","volume-title":"Advances in neural information processing systems 30","author":"SM Lundberg","year":"2017","unstructured":"Lundberg SM, Lee S-I (2017) A unified approach to interpreting model predictions. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems 30. Curran Associates, Montreal, pp 4765\u201374"},{"key":"841_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.artint.2018.07.007","volume":"267","author":"T Miller","year":"2019","unstructured":"Miller T (2019) Explanation in artificial intelligence: insights form social sciences. Artif Intell 267:1\u201338","journal-title":"Artif Intell"},{"key":"841_CR23","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1016\/j.patcog.2016.11.008","volume":"65","author":"G Montavon","year":"2017","unstructured":"Montavon G, Lapuschkin S, Binder A, Samek W, M\u00fcller K-R (2017) Explaining nonlinear classification decisions with deep Taylor decomposition. Pattern Recognit 65:211\u2013222","journal-title":"Pattern Recognit"},{"key":"841_CR24","doi-asserted-by":"crossref","unstructured":"Ribeiro MT, Singh S, Guestrin C (2016) \u201cWhy should I trust you?\u201d: explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining (KDD\u201916). Association for Computing Machinery, New York, pp 1135\u20131144","DOI":"10.1145\/2939672.2939778"},{"key":"841_CR25","doi-asserted-by":"crossref","unstructured":"Richman R, W\u00fcthrich MV (2020) Nagging predictors. Risks 8\/3, article 83","DOI":"10.3390\/risks8030083"},{"key":"841_CR26","doi-asserted-by":"crossref","unstructured":"Samek W, M\u00fcller K-R (2019) Toward explainable artificial intelligence. In: Samek W, Montavon G, Vedaldi A, Hansen LK, M\u00fcller K-R (eds) Explainable AI: interpreting, explaining and visualizing deep learning, lecture notes in artificial intelligence 11700. Springer, pp 5\u201323","DOI":"10.1007\/978-3-030-28954-6_1"},{"key":"841_CR27","first-page":"307","volume-title":"Contributions to the theory of games (AM-28)","author":"LS Shapley","year":"1953","unstructured":"Shapley LS (1953) A value for n-Person games. In: Kuhn HW, Tucker AW (eds) Contributions to the theory of games (AM-28), vol II. Princeton University Press, Princeton, pp 307\u2013318"},{"key":"841_CR28","unstructured":"Shrikumar A, Greenside P, Shcherbina A, Kundaje A (2016) Not just a black box: learning important features through propagating activation differences. arXiv:1605.01713"},{"key":"841_CR29","unstructured":"Shrikumar A, Greenside P, Kundaje A (2017) Learning important features through propagating activation differences. In: Proceedings of the 34th international conference on machine learning, proceedings of machine learning research, PMLR, vol 70. International Convention Centre, Sydney, Australia, pp 3145\u20133153"},{"key":"841_CR30","unstructured":"Sundararajan M, Taly A, Yan Q (2017) Axiomatic attribution for deep networks. In: Proceedings of the 34th international conference on machine learning, proceedings of machine learning research (PMLR), vol 70. International Convention Centre, Sydney, Australia, pp 3319\u20133328"},{"issue":"1","key":"841_CR31","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1111\/risa.12434","volume":"36","author":"A Tsanakas","year":"2016","unstructured":"Tsanakas A, Millossovich P (2016) Sensitivity analysis using risk measures. Risk Anal 36(1):30\u201348","journal-title":"Risk Anal"},{"issue":"1","key":"841_CR32","doi-asserted-by":"publisher","first-page":"71","DOI":"10.2143\/AST.26.1.563234","volume":"26","author":"S Wang","year":"1996","unstructured":"Wang S (1996) Premium calculation by transforming the layer premium density. ASTIN Bull 26(1):71\u201392","journal-title":"ASTIN Bull"},{"key":"841_CR33","doi-asserted-by":"crossref","unstructured":"W\u00fcthrich MV, Merz M (2021) Statistical foundations of actuarial learning and its applications. SSRN Manuscript ID 3822407","DOI":"10.2139\/ssrn.3822407"},{"issue":"1","key":"841_CR34","doi-asserted-by":"publisher","first-page":"272","DOI":"10.1080\/07350015.2019.1624293","volume":"39","author":"Q Zhao","year":"2021","unstructured":"Zhao Q, Hastie T (2021) Causal interpretations of black-box models. J Bus Econ Stat 39(1):272\u2013281","journal-title":"J Bus Econ Stat"},{"key":"841_CR35","doi-asserted-by":"publisher","DOI":"10.1201\/b12207","volume-title":"Ensemble methods: foundations and algorithms","author":"Z-H Zhou","year":"2012","unstructured":"Zhou Z-H (2012) Ensemble methods: foundations and algorithms. Chapman & Hall\/CRC, London"},{"issue":"1\u20132","key":"841_CR36","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1016\/S0004-3702(02)00190-X","volume":"137","author":"Z-H Zhou","year":"2002","unstructured":"Zhou Z-H, Wu J, Tang W (2002) Ensembling neural networks: many could be better than all. Artif Intell 137(1\u20132):239\u2013263","journal-title":"Artif Intell"}],"container-title":["Data Mining and Knowledge Discovery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10618-022-00841-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10618-022-00841-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10618-022-00841-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,11]],"date-time":"2023-11-11T10:10:13Z","timestamp":1699697413000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10618-022-00841-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,18]]},"references-count":36,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["841"],"URL":"https:\/\/doi.org\/10.1007\/s10618-022-00841-4","relation":{},"ISSN":["1384-5810","1573-756X"],"issn-type":[{"value":"1384-5810","type":"print"},{"value":"1573-756X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,18]]},"assertion":[{"value":"22 March 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 April 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 June 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}