{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T22:22:36Z","timestamp":1778365356263,"version":"3.51.4"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,3,1]],"date-time":"2022-03-01T00:00:00Z","timestamp":1646092800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,10]],"date-time":"2022-03-10T00:00:00Z","timestamp":1646870400000},"content-version":"vor","delay-in-days":9,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["ZE-1062\/4-1"],"award-info":[{"award-number":["ZE-1062\/4-1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Minds &amp; Machines"],"published-print":{"date-parts":[[2022,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Models developed using machine learning are increasingly prevalent in scientific research. At the same time, these models are notoriously opaque. Explainable AI aims to mitigate the impact of opacity by rendering opaque models transparent. More than being just the solution to a problem, however, Explainable AI can also play an invaluable role in scientific exploration. This paper describes how post-hoc analytic techniques from Explainable AI can be used to refine target phenomena in medical science, to identify starting points for future investigations of (potentially) causal relationships, and to generate possible explanations of target phenomena in cognitive science. In this way, this paper describes how Explainable AI\u2014over and above machine learning itself\u2014contributes to the efficiency and scope of data-driven scientific research.<\/jats:p>","DOI":"10.1007\/s11023-021-09583-6","type":"journal-article","created":{"date-parts":[[2022,3,10]],"date-time":"2022-03-10T12:03:32Z","timestamp":1646913812000},"page":"219-239","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":62,"title":["Scientific Exploration and Explainable Artificial Intelligence"],"prefix":"10.1007","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9702-7706","authenticated-orcid":false,"given":"Carlos","family":"Zednik","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hannes","family":"Boelsen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,3,10]]},"reference":[{"key":"9583_CR1","doi-asserted-by":"crossref","unstructured":"Bau, D., Zhou, B., Khosla, A., Oliva, A., & Torralba, A. (2017). Network dissection: Quantifying interpretability of deep visual representations. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3319\u20133327.","DOI":"10.1109\/CVPR.2017.354"},{"key":"9583_CR2","doi-asserted-by":"publisher","DOI":"10.1007\/s11229-021-03305-2","author":"C Beisbart","year":"2021","unstructured":"Beisbart, C. (2021). Opacity thought through: On the intransparency of computer simulations. Synthese. https:\/\/doi.org\/10.1007\/s11229-021-03305-2","journal-title":"Synthese"},{"key":"9583_CR3","doi-asserted-by":"crossref","unstructured":"Boon, M. (2020). \u201cHow Scientists are Brought Back into Science: The Error of Empiricism\u201d. In: Bertolaso M., Sterpetti F. (eds.) A Critical Reflection on Automated Science. Human Perspectives in Health Sciences and Technology, vol 1. Springer, Cham.","DOI":"10.1007\/978-3-030-25001-0_4"},{"issue":"3","key":"9583_CR4","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1007\/s00186-012-0404-7","volume":"77","author":"P B\u00fchlmann","year":"2013","unstructured":"B\u00fchlmann, P. (2013). Causal statistical inference in high dimensions. Mathematical Methods in Operations Research, 77(3), 357\u2013370.","journal-title":"Mathematical Methods in Operations Research"},{"issue":"1","key":"9583_CR5","doi-asserted-by":"publisher","first-page":"205395171562251","DOI":"10.1177\/2053951715622512","volume":"3","author":"J Burrell","year":"2016","unstructured":"Burrell, J. (2016). How the machine \u2018thinks\u2019: Understanding opacity in machine learning algorithms. Big Data & Society, 3(1), 205395171562251.","journal-title":"Big Data & Society"},{"issue":"4","key":"9583_CR6","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1016\/j.tics.2019.01.009","volume":"23","author":"RM Cichy","year":"2019","unstructured":"Cichy, R. M., & Kaiser, D. (2019). Deep neural networks as scientific models. Trends in Cognitive Sciences, 23(4), 305\u2013317.","journal-title":"Trends in Cognitive Sciences"},{"key":"9583_CR7","doi-asserted-by":"publisher","first-page":"27755","DOI":"10.1038\/srep27755","volume":"6","author":"RM Cichy","year":"2016","unstructured":"Cichy, R. M., Khosla, A., Pantazis, D., Torralba, A., & Oliva, A. (2016). Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Scientific Reports, 6, 27755.","journal-title":"Scientific Reports"},{"key":"9583_CR8","doi-asserted-by":"crossref","unstructured":"Confalonieri, R., Coba, L., Wagner, B., & Besold, T. R. (2021). A historical perspective of explainable Artificial Intelligence. WIREs Data Mining and Knowledge Discovery, 11(1), e1391.","DOI":"10.1002\/widm.1391"},{"key":"9583_CR9","doi-asserted-by":"publisher","DOI":"10.7208\/chicago\/9780226039824.001.0001","volume-title":"In Search of Mechanisms: Discoveries Across the Life Sciences","author":"C Craver","year":"2013","unstructured":"Craver, C., & Darden, L. (2013). In Search of Mechanisms: Discoveries Across the Life Sciences. Chicago University Press."},{"key":"9583_CR10","doi-asserted-by":"crossref","unstructured":"Dattilo, A., Vanderburg, A., Shallue, C. J., Mayo, A. W., Berlind, P., Bieryla, A., Calkins, M. L., Esquerdo, G. A., Everett, M. E., Howell, S. B., Latham, D. W., Scott, N. J., & Yu, L. (2019). Identifying exoplanets with deep learning II: Two new super-earths uncovered by a neural network in K2 data. arXiv, 1903.10507.","DOI":"10.3847\/1538-3881\/ab0e12"},{"key":"9583_CR11","unstructured":"Doran, D., Schulz, S., & Besold, T. R. (2017). What does explainable AI really mean? A new conceptualization of perspectives. arXiv, 1710.00794"},{"key":"9583_CR12","doi-asserted-by":"publisher","first-page":"645","DOI":"10.1007\/s11023-018-9481-6","volume":"28","author":"JM Dur\u00e1n","year":"2018","unstructured":"Dur\u00e1n, J. M., & Formanek, N. (2018). Grounds for trust: Essential epistemic opacity and computational reliabilism. Minds & Machines, 28, 645\u2013666.","journal-title":"Minds & Machines"},{"key":"9583_CR13","doi-asserted-by":"crossref","unstructured":"Erasmus, A., Brunet, T. D. P., & Fisher, E. (2020). What is interpretability? Philosophy & Technology.","DOI":"10.1007\/s13347-020-00435-2"},{"key":"9583_CR14","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-27954-1","volume-title":"How to do science with models. A philosophical primer","author":"A Gelfert","year":"2016","unstructured":"Gelfert, A. (2016). How to do science with models. A philosophical primer. Dordrecht."},{"issue":"3","key":"9583_CR15","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1609\/aimag.v38i3.2741","volume":"38","author":"B Goodman","year":"2017","unstructured":"Goodman, B., & Flaxman, S. (2017). European Union regulations on algorithmic decision-making and a \u201cright to explanation.\u201d AI Magazine, 38(3), 50\u201357.","journal-title":"AI Magazine"},{"key":"9583_CR16","doi-asserted-by":"crossref","unstructured":"Hohman, F. M., Kahng, M., Pienta, R., & Chau, D. H. (2018). Visual analytics in deep learning: An interrogative survey for the next frontiers. IEEE Transactions on Visualization and Computer Graphics.","DOI":"10.1109\/TVCG.2018.2843369"},{"issue":"3","key":"9583_CR17","doi-asserted-by":"publisher","first-page":"615","DOI":"10.1007\/s11229-008-9435-2","volume":"169","author":"P Humphreys","year":"2009","unstructured":"Humphreys, P. (2009). The philosophical novelty of computer simulation methods. Synthese, 169(3), 615\u2013626.","journal-title":"Synthese"},{"issue":"4","key":"9583_CR18","first-page":"1","volume":"2","author":"N Kriegeskorte","year":"2008","unstructured":"Kriegeskorte, N., Mur, M., & Bandettini, P. (2008). Representational similarity analysis \u2013 connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2(4), 1\u201328.","journal-title":"Frontiers in Systems Neuroscience"},{"key":"9583_CR19","doi-asserted-by":"crossref","unstructured":"Lapuschkin, S., W\u00e4ldchen, S., Binder, A., Montavon, G., Samek, W., & M\u00fcller, K.-R. (2019). Unmasking Clever Hans predictors and assessing what machines really learn. Nature Communications, 10(1096).","DOI":"10.1038\/s41467-019-08987-4"},{"key":"9583_CR20","unstructured":"Li, X., Wu, J., Chen, E. Z., & Jiang, H. (2019). What evidence does deep learning model use to classify skin lesions? arXiv, 1811.01051v2."},{"key":"9583_CR21","unstructured":"Lipton, Z.C. (2016). The mythos of model interpretability. arXiv, 1606.03490v3."},{"key":"9583_CR22","unstructured":"Lundberg, S. M. & Lee, S. (2017). A unified approach to interpreting model predictions. arXiv, 1705.07874v2."},{"issue":"5","key":"9583_CR23","doi-asserted-by":"publisher","first-page":"1307","DOI":"10.1007\/s00425-018-2976-9","volume":"248","author":"W Ma","year":"2018","unstructured":"Ma, W., Qiu, Z., Song, J., Li, J., Cheng, Q., Zhai, J., & Ma, C. (2018). A deep convolutional neural network approach for predicting phenotypes from genotypes. Planta, 248(5), 1307\u20131318.","journal-title":"Planta"},{"key":"9583_CR24","volume-title":"Vision: A computational investigation into the human representation and processing of visual information","author":"D Marr","year":"1982","unstructured":"Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information. MIT Press."},{"issue":"5","key":"9583_CR25","doi-asserted-by":"publisher","first-page":"869","DOI":"10.1086\/705494","volume":"86","author":"M Massimi","year":"2019","unstructured":"Massimi, M. (2019). Two kinds of exploratory models. Philosophy of Science, 86(5), 869\u2013881.","journal-title":"Philosophy of Science"},{"issue":"1","key":"9583_CR26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/srep26094","volume":"6","author":"R Miotto","year":"2016","unstructured":"Miotto, R., Li, L., Kidd, B. A., & Dudley, J. T. (2016). Deep patient: An unsupervised representation to predict the future of patients from the electronic health records. Scientific Reports, 6(1), 1\u201310.","journal-title":"Scientific Reports"},{"key":"9583_CR27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.dsp.2017.10.011","volume":"73","author":"G Montavon","year":"2018","unstructured":"Montavon, G., Samek, W., & M\u00fcller, K.-R. (2018). Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 73, 1\u201315.","journal-title":"Digital Signal Processing"},{"key":"9583_CR28","volume-title":"Causality: Models, Reasoning, and Inference","author":"J Pearl","year":"2000","unstructured":"Pearl, J. (2000). Causality: Models, Reasoning, and Inference. Cambridge University Press."},{"key":"9583_CR29","doi-asserted-by":"publisher","first-page":"905","DOI":"10.1086\/683328","volume":"82","author":"W Pietsch","year":"2015","unstructured":"Pietsch, W. (2015). Aspects of Theory-Ladenness in Data-Intensive Science. Philosophy of Science, 82, 905\u2013916.","journal-title":"Philosophy of Science"},{"issue":"2","key":"9583_CR30","doi-asserted-by":"publisher","first-page":"198","DOI":"10.1086\/680332","volume":"82","author":"E Ratti","year":"2015","unstructured":"Ratti, E. (2015). Big Data Biology: Between Eliminative Inferences and Exploratory Experiments. Philosophy of Science, 82(2), 198\u2013218.","journal-title":"Philosophy of Science"},{"key":"9583_CR31","doi-asserted-by":"crossref","unstructured":"Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). \u201cWhy Should I Trust You?\u201d: Explaining the Predictions of Any Classifier. arXiv, 1602.04938v3.","DOI":"10.1145\/2939672.2939778"},{"key":"9583_CR32","doi-asserted-by":"crossref","unstructured":"Ritchie, J. B., Kaplan, D.M., & Klein, C. (2019). Decoding the brain: neural representation and the limits of multivariate pattern analysis in cognitive neuroscience. British Journal for the Philosophy of Science, 70(2, 581\u2013607.","DOI":"10.1093\/bjps\/axx023"},{"issue":"5","key":"9583_CR33","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","volume":"1","author":"C Rudin","year":"2019","unstructured":"Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206\u2013215.","journal-title":"Nature Machine Intelligence"},{"key":"9583_CR34","volume-title":"Four decades of scientific explanation","author":"WC Salmon","year":"1989","unstructured":"Salmon, W. C. (1989). Four decades of scientific explanation. University of Minnesota Press."},{"key":"9583_CR35","doi-asserted-by":"crossref","unstructured":"Schmidt, J., Marques, M. R. G., Botti, S., & Marques, M. A. L.(2019). Recent advances and applications of machine learning in solid-state materials science. npj Computational Materials, 5, 83.","DOI":"10.1038\/s41524-019-0221-0"},{"issue":"3","key":"9583_CR36","doi-asserted-by":"publisher","first-page":"393","DOI":"10.1007\/s11229-006-9099-8","volume":"153","author":"O Shagrir","year":"2006","unstructured":"Shagrir, O. (2006). Why we view the brain as a computer. Synthese, 153(3), 393\u2013416.","journal-title":"Synthese"},{"key":"9583_CR37","doi-asserted-by":"crossref","unstructured":"Steinle, F. (1997). Entering New Fields: Exploratory Uses of Experimentation. Philosophy of Science 64 (Proceedings): S64\u2013S74.","DOI":"10.1086\/392587"},{"key":"9583_CR38","doi-asserted-by":"crossref","unstructured":"Sullivan, E. (2019). Understanding from machine learning models. The British Journal for the Philosophy of Science, axz035.","DOI":"10.1093\/bjps\/axz035"},{"key":"9583_CR39","unstructured":"Tomsett, R., Braines, D., Harborne, D., Preece, A., & Chakraborty, S. (2018). Interpretable to whom? A role-based model for analyzing interpretable machine learning systems. arXiv, 1806.07552."},{"key":"9583_CR40","doi-asserted-by":"crossref","unstructured":"Wachter, S., Mittelstadt, B., & Russell, C. (2018). Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law & Technology, 31(2).","DOI":"10.2139\/ssrn.3063289"},{"key":"9583_CR41","doi-asserted-by":"crossref","unstructured":"Wu, M., Hughes, M. C., Parbhoo, S., Zazzi, M., Roth, V., & Doshi-Velez, F. (2018). Beyond sparsity: Tree regularization of deep models for interpretability. arXiv, 1711.06178v1.","DOI":"10.1609\/aaai.v32i1.11501"},{"issue":"11","key":"9583_CR42","doi-asserted-by":"publisher","first-page":"1185","DOI":"10.7150\/ijms.10001","volume":"11","author":"Y Wu","year":"2014","unstructured":"Wu, Y., Ding, Y., Tanaka, Y., & Zhang, W. (2014). Risk factors contributing to type 2 diabetes and recent advances in the treatment and prevention. International Journal of Medical Sciences, 11(11), 1185\u20131200.","journal-title":"International Journal of Medical Sciences"},{"key":"9583_CR43","doi-asserted-by":"crossref","unstructured":"Zednik, C. (2018). Will machine learning yield machine intelligence? In V. M\u00fcller (ed.), Philosophy and Theory of Artificial Intelligence 2017. PT-AI 2017. Studies in Applied Philosophy, Epistemology and Rational Ethics, 44. Springer: Cham.","DOI":"10.1007\/978-3-319-96448-5_23"},{"key":"9583_CR44","doi-asserted-by":"publisher","DOI":"10.1007\/s13347-019-00382-7","author":"C Zednik","year":"2019","unstructured":"Zednik, C. (2019). Solving the black box problem: A normative framework for explainable artificial intelligence. Philosophy & Technology. https:\/\/doi.org\/10.1007\/s13347-019-00382-7","journal-title":"Philosophy & Technology"},{"issue":"4","key":"9583_CR45","doi-asserted-by":"publisher","first-page":"661","DOI":"10.1007\/s13347-018-0330-6","volume":"32","author":"J Zerilli","year":"2018","unstructured":"Zerilli, J., Knott, A., Maclaurin, J., & Gavaghan, C. (2018). Transparency in algorithmic and human decision-making: Is there a double standard? Philosophy & Technology, 32(4), 661\u2013683.","journal-title":"Philosophy & Technology"},{"issue":"10","key":"9583_CR46","doi-asserted-by":"publisher","first-page":"4311","DOI":"10.1021\/acs.molpharmaceut.8b00930","volume":"15","author":"A Zhavoronkov","year":"2018","unstructured":"Zhavoronkov, A. (2018). Artificial intelligence for drug discovery, biomarker development, and generation of novel chemistry. Molecular Pharmaceutics, 15(10), 4311\u20134313.","journal-title":"Molecular Pharmaceutics"},{"key":"9583_CR47","doi-asserted-by":"crossref","unstructured":"Zilke, J. R., Mencia, E. L., & Janssen, F. (2016). DeepRED \u2013 Rule extraction from deep neural networks. In T. Calders, M. Ceci, & D. Malerba (eds.), Discovery Science 19th International Conference (pp. 457\u2013473).","DOI":"10.1007\/978-3-319-46307-0_29"},{"key":"9583_CR48","unstructured":"Zintgraf, L. M., Cohen, T. S., Adel, T., & Welling, M. (2017). Visualizing deep neural network decisions: Prediction difference analysis. arXiv, 1702.04595."}],"container-title":["Minds and Machines"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11023-021-09583-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11023-021-09583-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11023-021-09583-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,28]],"date-time":"2023-01-28T19:45:23Z","timestamp":1674935123000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11023-021-09583-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,3]]}},"alternative-id":["9583"],"URL":"https:\/\/doi.org\/10.1007\/s11023-021-09583-6","relation":{},"ISSN":["0924-6495","1572-8641"],"issn-type":[{"value":"0924-6495","type":"print"},{"value":"1572-8641","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3]]},"assertion":[{"value":"7 May 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 November 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 March 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no conflicts of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}