{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T13:45:46Z","timestamp":1782999946536,"version":"3.54.5"},"reference-count":56,"publisher":"Springer Science and Business Media LLC","issue":"11","license":[{"start":{"date-parts":[[2021,9,14]],"date-time":"2021-09-14T00:00:00Z","timestamp":1631577600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,9,14]],"date-time":"2021-09-14T00:00:00Z","timestamp":1631577600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Max Planck Institute for Informatics"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Comput Vis"],"published-print":{"date-parts":[[2021,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Today\u2019s deep learning systems deliver high performance based on end-to-end training but are notoriously hard to inspect. We argue that there are at least two reasons making inspectability challenging: (i) representations are distributed across hundreds of channels and (ii) a unifying metric quantifying inspectability is lacking. In this paper, we address both issues by proposing Semantic Bottlenecks (SB), which can be integrated into pretrained networks, to align channel outputs with individual visual concepts and introduce the model agnostic Area Under inspectability Curve (AUiC) metric to measure the alignment. We present a case study on semantic segmentation to demonstrate that SBs improve the AUiC up to six-fold over regular network outputs. We explore two types of SB-layers in this work. First, concept-supervised SB-layers (SSB), which offer inspectability w.r.t. predefined concepts that the model is demanded to rely on. And second, unsupervised SBs (USB), which offer equally strong AUiC improvements by restricting distributedness of representations across channels. Importantly, for both SB types, we can recover state of the art segmentation performance across two different models despite a drastic dimensionality reduction from 1000s of non aligned channels to 10s of semantics-aligned channels that all downstream results are based on.<\/jats:p>","DOI":"10.1007\/s11263-021-01498-0","type":"journal-article","created":{"date-parts":[[2021,9,14]],"date-time":"2021-09-14T02:03:26Z","timestamp":1631585006000},"page":"3136-3153","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Semantic Bottlenecks: Quantifying and Improving Inspectability of Deep Representations"],"prefix":"10.1007","volume":"129","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6525-4528","authenticated-orcid":false,"given":"Max","family":"Losch","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mario","family":"Fritz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bernt","family":"Schiele","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,9,14]]},"reference":[{"key":"1498_CR1","unstructured":"Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., Kim, B. (2018). Sanity checks for saliency maps. In NeurIPS"},{"key":"1498_CR2","unstructured":"Al-Shedivat, M., Dubey, A., Xing, E. P. (2020). Contextual explanation networks. Journal of Machine Learning Research, 21, 194\u20131"},{"key":"1498_CR3","doi-asserted-by":"crossref","unstructured":"Bach, S., Binder, A., Montavon, G., Klauschen, F., M\u00fcller, K. R., Samek, W. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7), e0130140","DOI":"10.1371\/journal.pone.0130140"},{"key":"1498_CR4","doi-asserted-by":"crossref","unstructured":"Bau, D., Zhou, B., Khosla, A., Oliva, A., Torralba, A. (2017). Network dissection: quantifying interpretability of deep visual representations. In CVPR","DOI":"10.1109\/CVPR.2017.354"},{"key":"1498_CR5","unstructured":"Bau, D., Zhu, J.Y., Strobelt, H., Zhou, B., Tenenbaum, J.B., Freeman, W.T., Torralba, A. (2019). Gan dissection: visualizing and understanding generative adversarial networks. In ICLR"},{"issue":"4","key":"1498_CR6","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1145\/2461912.2462002","volume":"32","author":"S Bell","year":"2013","unstructured":"Bell, S., Upchurch, P., Snavely, N., & Bala, K. (2013). Opensurfaces: a richly annotated catalog of surface appearance. ACM Transactions on Graphics (TOG), 32(4), 111.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"1498_CR7","doi-asserted-by":"crossref","unstructured":"Bucher, M., Herbin, S., Jurie, F. (2018). Semantic bottleneck for computer vision tasks. In ACCV (pp. 695\u2013712). Springer","DOI":"10.1007\/978-3-030-20890-5_44"},{"key":"1498_CR8","unstructured":"Burgess, C.P., Matthey, L., Watters, N., Kabra, R., Higgins, I., Botvinick, M., Lerchner, A. (2019). Monet: unsupervised scene decomposition and representation. arXiv preprint arXiv:1901.11390"},{"key":"1498_CR9","unstructured":"Chen, C., Li, O., Tao, D., Barnett, A., Rudin, C., Su, J.K. (2019). This looks like that: deep learning for interpretable image recognition. In NeurIPS (pp. 8930\u20138941 )"},{"issue":"4","key":"1498_CR10","doi-asserted-by":"publisher","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","volume":"40","author":"LC Chen","year":"2018","unstructured":"Chen, L. C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018). Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. TPAMI, 40(4), 834\u2013848.","journal-title":"TPAMI"},{"key":"1498_CR11","doi-asserted-by":"crossref","unstructured":"Chen, R., Chen, H., Ren, J., Huang, G., Zhang, Q. (2019). Explaining neural networks semantically and quantitatively. In ICCV (pp. 9187\u20139196)","DOI":"10.1109\/ICCV.2019.00928"},{"key":"1498_CR12","doi-asserted-by":"crossref","unstructured":"Chen, X., Mottaghi, R., Liu, X., Fidler, S., Urtasun, R., Yuille, A. (2014). Detect what you can: detecting and representing objects using holistic models and body parts. In CVPR","DOI":"10.1109\/CVPR.2014.254"},{"key":"1498_CR13","unstructured":"Chu, E., Roy, D., Andreas, J. (2020). Are visual explanations useful? A case study in model-in-the-loop prediction. arXiv preprint arXiv:2007.12248"},{"key":"1498_CR14","doi-asserted-by":"crossref","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B. (2016) The cityscapes dataset for semantic urban scene understanding. In CVPR","DOI":"10.1109\/CVPR.2016.350"},{"key":"1498_CR15","doi-asserted-by":"crossref","unstructured":"Esser, P., Rombach, R., Ommer, B. (2020). A disentangling invertible interpretation network for explaining latent representations. In CVPR (pp. 9223\u20139232)","DOI":"10.1109\/CVPR42600.2020.00924"},{"key":"1498_CR16","doi-asserted-by":"crossref","unstructured":"Fong, R., Patrick, M., Vedaldi, A. (2019). Understanding deep networks via extremal perturbations and smooth masks. In ICCV","DOI":"10.1109\/ICCV.2019.00304"},{"key":"1498_CR17","doi-asserted-by":"crossref","unstructured":"Fong, R., Vedaldi, A. (2018). Net2vec: quantifying and explaining how concepts are encoded by filters in deep neural networks. In CVPR (pp. 8730\u20138738)","DOI":"10.1109\/CVPR.2018.00910"},{"key":"1498_CR18","unstructured":"Greff, K., Kaufmann, R.L., Kabra, R., Watters, N., Burgess, C., Zoran, D., Matthey, L., Botvinick, M., Lerchner, A. (2019). Multi-object representation learning with iterative variational inference. arXiv preprint arXiv:1903.00450"},{"key":"1498_CR19","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep residual learning for image recognition. In CVPR","DOI":"10.1109\/CVPR.2016.90"},{"issue":"5","key":"1498_CR20","first-page":"6","volume":"2","author":"I Higgins","year":"2017","unstructured":"Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., et al. (2017). beta-vae: learning basic visual concepts with a constrained variational framework. ICLR, 2(5), 6.","journal-title":"ICLR"},{"key":"1498_CR21","unstructured":"Hooker, S., Erhan, D., Kindermans, P.J., Kim, B. (2019) A benchmark for interpretability methods in deep neural networks. In Advances in neural information processing systems (pp. 9737\u20139748)"},{"key":"1498_CR22","unstructured":"Jacobsen, J.H., Smeulders, A.W., Oyallon, E. (2018). i-revnet: deep invertible networks. In International conference on learning representations (ICLR)"},{"key":"1498_CR23","unstructured":"Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., et\u00a0al. (2018). Interpretability beyond feature attribution: quantitative testing with concept activation vectors (tcav). In ICML"},{"key":"1498_CR24","unstructured":"Kingma, D.P., Welling, M. (2013). Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114"},{"key":"1498_CR25","unstructured":"Koh, P.W., Nguyen, T., Tang, Y.S., Mussmann, S., Pierson, E., Kim, B., Liang, P. (2020). Concept bottleneck models. In ICML. PMLR"},{"key":"1498_CR26","unstructured":"Li, L.J., Su, H., Fei-Fei, L., Xing, E.P. (2010). Object bank: a high-level image representation for scene classification & semantic feature sparsification. In NeurIPS"},{"key":"1498_CR27","doi-asserted-by":"crossref","unstructured":"Li, O., Liu, H., Chen, C., Rudin, C. (2018). Deep learning for case-based reasoning through prototypes: a neural network that explains its predictions. In AAAI","DOI":"10.1609\/aaai.v32i1.11771"},{"key":"1498_CR28","doi-asserted-by":"crossref","unstructured":"Lin, D., Shen, X., Lu, C., Jia, J. (2015) Deep lac: deep localization, alignment and classification for fine-grained recognition. In CVPR (pp. 1666\u20131674)","DOI":"10.1109\/CVPR.2015.7298775"},{"issue":"3","key":"1498_CR29","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1145\/3236386.3241340","volume":"16","author":"ZC Lipton","year":"2018","unstructured":"Lipton, Z. C. (2018). The mythos of model interpretability. Queue, 16(3), 30.","journal-title":"Queue"},{"key":"1498_CR30","unstructured":"Liu, H., Simonyan, K., Yang, Y. (2019). DARTS: differentiable architecture search. In ICLR"},{"key":"1498_CR31","unstructured":"Lundberg, S.M., Lee, S.I. (2017). A unified approach to interpreting model predictions. In NeurIPS"},{"key":"1498_CR32","doi-asserted-by":"crossref","unstructured":"Mahendran, A., Vedaldi, A. (2015). Understanding deep image representations by inverting them. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)","DOI":"10.1109\/CVPR.2015.7299155"},{"key":"1498_CR33","doi-asserted-by":"crossref","unstructured":"Marcos, D., Lobry, S., Tuia, D. (2019). Semantically interpretable activation maps: what-where-how explanations within cnns. arXiv preprint arXiv:1909.08442","DOI":"10.1109\/ICCVW.2019.00518"},{"key":"1498_CR34","unstructured":"Melis, D.A., Jaakkola, T. (2018). Towards robust interpretability with self-explaining neural networks. In NeurIPS"},{"key":"1498_CR35","unstructured":"Mu, J., Andreas, J. (2020). Compositional explanations of neurons. In NeurIPS"},{"key":"1498_CR36","doi-asserted-by":"crossref","unstructured":"Neuhold, G., Ollmann, T., Rota\u00a0Bulo, S., Kontschieder, P. (2017). The mapillary vistas dataset for semantic understanding of street scenes. In ICCV (pp. 4990\u20134999)","DOI":"10.1109\/ICCV.2017.534"},{"key":"1498_CR37","unstructured":"Petsiuk, V., Das, A., Saenko, K. (2018). Rise: randomized input sampling for explanation of black-box models. In BMVC"},{"key":"1498_CR38","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., Guestrin, C. (2016). Why should i trust you?: explaining the predictions of any classifier. block In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining","DOI":"10.1145\/2939672.2939778"},{"key":"1498_CR39","doi-asserted-by":"crossref","unstructured":"Samek, W., Binder, A., Montavon, G., Lapuschkin, S., M\u00fcller, K.R. (2016). Evaluating the visualization of what a deep neural network has learned. In IEEE transactions on neural networks and learning systems","DOI":"10.1109\/TNNLS.2016.2599820"},{"key":"1498_CR40","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D., et\u00a0al. (2017). Grad-cam: visual explanations from deep networks via gradient-based localization. In ICCV (pp. 618\u2013626)","DOI":"10.1109\/ICCV.2017.74"},{"key":"1498_CR41","unstructured":"Simonyan, K., Vedaldi, A., Zisserman, A. (2014). Deep inside convolutional networks: visualising image classification models and saliency maps. In ICLR"},{"key":"1498_CR42","unstructured":"Srinivas, S., Fleuret, F. (2019). Full-gradient representation for neural network visualization. In Advances in neural information processing systems (pp. 4124\u20134133)"},{"key":"1498_CR43","unstructured":"Sundararajan, M., Taly, A., Yan, Q. (2017). Axiomatic attribution for deep networks. In ICML"},{"key":"1498_CR44","unstructured":"Tao, A., Sapra, K., Catanzaro, B. (2020). Hierarchical multi-scale attention for semantic segmentation. arXiv preprint arXiv:2005.10821"},{"key":"1498_CR45","unstructured":"Wada, K. (2016). labelme: image polygonal annotation with python. https:\/\/github.com\/wkentaro\/labelme"},{"key":"1498_CR46","doi-asserted-by":"crossref","unstructured":"Xiao, T., Liu, Y., Zhou, B., Jiang, Y., Sun, J. (2018). Unified perceptual parsing for scene understanding. In ECCV","DOI":"10.1007\/978-3-030-01228-1_26"},{"key":"1498_CR47","unstructured":"Xie, S., Zheng, H., Liu, C., Lin, L. (2019). SNAS: stochastic neural architecture search. In ICLR"},{"key":"1498_CR48","unstructured":"Yeh, C.K., Kim, B., Arik, S.O., Li, C.L., Ravikumar, P., Pfister, T. (2019). On concept-based explanations in deep neural networks. arXiv preprint arXiv:1910.07969"},{"key":"1498_CR49","unstructured":"Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., Lipson, H. (2015). Understanding neural networks through deep visualization. arXiv:1506.06579"},{"key":"1498_CR50","doi-asserted-by":"crossref","unstructured":"Yuan, Y., Chen, X., Wang, J. (2020). Object-contextual representations for semantic segmentation. In ECCV","DOI":"10.1007\/978-3-030-58539-6_11"},{"key":"1498_CR51","doi-asserted-by":"crossref","unstructured":"Zeiler, M.D., Fergus, R. (2014). Visualizing and understanding convolutional networks. In ECCV","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"1498_CR52","doi-asserted-by":"crossref","unstructured":"Zhang, Q., Nian\u00a0Wu, Y., Zhu, S.C. (2018). Interpretable convolutional neural networks. In CVPR (pp. 8827\u20138836)","DOI":"10.1109\/CVPR.2018.00920"},{"key":"1498_CR53","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J. (2017). Pyramid scene parsing network. In CVPR","DOI":"10.1109\/CVPR.2017.660"},{"key":"1498_CR54","doi-asserted-by":"crossref","unstructured":"Zhou, B., Bau, D., Oliva, A., Torralba, A. (2017). Interpreting deep visual representations via network dissection. arXiv e-prints arXiv:1711.05611","DOI":"10.1167\/18.10.1244"},{"key":"1498_CR55","doi-asserted-by":"crossref","unstructured":"Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A. (2017). Scene parsing through ade20k dataset. In CVPR","DOI":"10.1109\/CVPR.2017.544"},{"key":"1498_CR56","unstructured":"Zintgraf, L.M., Cohen, T.S., Adel, T., Welling, M. (2017). Visualizing deep neural network decisions: prediction difference analysis. arXiv:1702.04595"}],"container-title":["International Journal of Computer Vision"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-021-01498-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11263-021-01498-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-021-01498-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T03:43:11Z","timestamp":1673235791000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11263-021-01498-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,14]]},"references-count":56,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2021,11]]}},"alternative-id":["1498"],"URL":"https:\/\/doi.org\/10.1007\/s11263-021-01498-0","relation":{},"ISSN":["0920-5691","1573-1405"],"issn-type":[{"value":"0920-5691","type":"print"},{"value":"1573-1405","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,14]]},"assertion":[{"value":"24 January 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 July 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 September 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}