{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T11:40:59Z","timestamp":1777549259362,"version":"3.51.4"},"reference-count":51,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2022,1,21]],"date-time":"2022-01-21T00:00:00Z","timestamp":1642723200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,1,21]],"date-time":"2022-01-21T00:00:00Z","timestamp":1642723200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Minds &amp; Machines"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Users of sociotechnical systems often have no way to independently verify whether the system output which they use to make decisions is correct; they are epistemically dependent on the system. We argue that this leads to problems when the system is wrong, namely to bad decisions and violations of the norm of practical reasoning. To prevent this from occurring we suggest the implementation of defeaters: information that a system is unreliable in a specific case (undercutting defeat) or independent information that the output is wrong (rebutting defeat). Practically, we suggest to design defeaters based on the different ways in which a system might produce erroneous outputs, and analyse this suggestion with a case study of the risk classification algorithm used by the Dutch tax agency.<\/jats:p>","DOI":"10.1007\/s11023-022-09591-0","type":"journal-article","created":{"date-parts":[[2022,1,21]],"date-time":"2022-01-21T12:03:17Z","timestamp":1642766597000},"page":"541-562","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Spotting When Algorithms Are Wrong"],"prefix":"10.1007","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0004-0681","authenticated-orcid":false,"given":"Stefan","family":"Buijsman","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Herman","family":"Veluwenkamp","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,1,21]]},"reference":[{"key":"9591_CR1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2021.05.008","author":"M Abdar","year":"2021","unstructured":"Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P. W., Cao, X., Khosravi, A., Acharya, U. R., Makarenkov, V., & Nahavandi, S. (2021). A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information Fusion. https:\/\/doi.org\/10.1016\/j.inffus.2021.05.008","journal-title":"Information Fusion"},{"key":"9591_CR2","doi-asserted-by":"publisher","first-page":"2937","DOI":"10.1007\/s11229-017-1549-y","volume":"197","author":"J Adam Carter","year":"2017","unstructured":"Adam Carter, J. (2017). Intellectual autonomy, epistemic dependence and cognitive enhancement. Synthese, 197, 2937\u20132961.","journal-title":"Synthese"},{"key":"9591_CR3","volume-title":"Epistemic autonomy","author":"J Adam Carter","year":"2021","unstructured":"Adam Carter, J. (2021). Epistemic autonomy and externalism. In K. Lougheed & J. Matheson (Eds.), Epistemic autonomy. Routledge."},{"issue":"1","key":"9591_CR4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1177\/1529100619832930","volume":"20","author":"L Barrett","year":"2019","unstructured":"Barrett, L., Adoplhs, R., Marsella, S., Martinez, A., & Pollak, S. (2019). Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest, 20(1), 1\u201368.","journal-title":"Psychological Science in the Public Interest"},{"key":"9591_CR5","doi-asserted-by":"publisher","DOI":"10.1093\/0199275742.001.0001","volume-title":"Justification without awareness","author":"M Bergmann","year":"2006","unstructured":"Bergmann, M. (2006). Justification without awareness. Oxford University Press."},{"issue":"2","key":"9591_CR6","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1111\/j.1468-0068.2008.00677.x","volume":"42","author":"J Brown","year":"2008","unstructured":"Brown, J. (2008). Subject-sensitive invariantism and the knowledge norm for practical reasoning. Nous, 42(2), 167\u2013189.","journal-title":"Nous"},{"key":"9591_CR7","unstructured":"Chan, S. (2017). Stanislav Petrov, Soviet Officer who helped avert nuclear war, is dead at 77. The New York Times. Retrieved September 18, 2017, from https:\/\/www.nytimes.com\/2017\/09\/18\/world\/europe\/stanislav-petrov-nuclear-war-dead.html"},{"key":"9591_CR8","doi-asserted-by":"crossref","unstructured":"Chiang, C., & Yin, M. (2021). You'd better stop! Understanding human reliance on machine learning models under covariate shift. In The 13th ACM web science conference, June 2021.","DOI":"10.1145\/3447535.3462487"},{"key":"9591_CR9","unstructured":"Desiere, S., Langenbucher, K., & Struyven, L. (2019). Statistical profiling in public employment services: An international comparison. OECD Social, Employment and Migration Working Papers, 224. OECD."},{"key":"9591_CR10","doi-asserted-by":"crossref","unstructured":"Dodge, J., Liao, Q., Zhang, Y., Bellamy, R., & Dugan, C. (2019). Explaining models: an empirical study of how explanations impact fairness judgment. In Proceedings of the 24th international conference on intelligent user interfaces (pp. 275\u2013285).","DOI":"10.1145\/3301275.3302310"},{"issue":"5","key":"9591_CR11","first-page":"329","volume":"47","author":"J Dur\u00e1n","year":"2021","unstructured":"Dur\u00e1n, J., & Jongsma, K. (2021). Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI. Journal of Medical Ethics, 47(5), 329\u2013335.","journal-title":"Journal of Medical Ethics"},{"key":"9591_CR12","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1093\/oso\/9780198847205.003.0010","volume-title":"Reasons, justification, and defeat","author":"J Dutant","year":"2021","unstructured":"Dutant, J., & Littlejohn, C. (2021). Defeaters as Indicators of Ignorance. In M. Simion & J. Brown (Eds.), Reasons, justification, and defeat (pp. 223\u2013246). Oxford University Press."},{"key":"9591_CR13","doi-asserted-by":"crossref","unstructured":"Erlei, A., Nekdem, F., Meub, L., Anand, A., & Gadiraju, U. (2020). Impact of algorithmic decision making on human behavior: Evidence from ultimatum bargaining. In Proceedings of the AAAI conference on human computation and crowdsourcing (Vol. 8(1), pp. 43\u201352).","DOI":"10.1609\/hcomp.v8i1.7462"},{"key":"9591_CR14","doi-asserted-by":"crossref","unstructured":"Ferrario, A., & Loi, M. (2021). The meaning of \u201cExplainability fosters trust in AI\u201d. SSRN 3916396.","DOI":"10.2139\/ssrn.3916396"},{"key":"9591_CR15","doi-asserted-by":"publisher","DOI":"10.1136\/medethics-2021-107482","author":"A Ferrario","year":"2021","unstructured":"Ferrario, A. (2021). Design publicity of black box algorithms: A support to the epistemic and ethical justifications of medical AI systems. Journal of Medical Ethics. https:\/\/doi.org\/10.1136\/medethics-2021-107482","journal-title":"Journal of Medical Ethics"},{"key":"9591_CR16","doi-asserted-by":"crossref","unstructured":"Fong, R., & Vedaldi, A. (2017). Interpretable explanations of black boxes by meaningful perturbation. In 2017 IEEE international conference on computer vision (ICCV) (pp. 3449\u20133457), Venice, Italy, 2017.","DOI":"10.1109\/ICCV.2017.371"},{"key":"9591_CR17","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1007\/s11229-009-9655-0","volume":"178","author":"M Gerken","year":"2011","unstructured":"Gerken, M. (2011). Warrant and action. Synthese, 178, 529\u2013547.","journal-title":"Synthese"},{"key":"9591_CR18","doi-asserted-by":"publisher","first-page":"2863","DOI":"10.1007\/s11229-015-0662-z","volume":"194","author":"S Goldberg","year":"2017","unstructured":"Goldberg, S. (2017). Should have known. Synthese, 194, 2863\u20132894.","journal-title":"Synthese"},{"key":"9591_CR19","unstructured":"Guidotti, R., Monreale, A., Ruggieri, S., Pedreschi, D., Turini, F., & Giannotti, F. (2018). Local rule-based explanations of black box decision systems. arXiv preprint. arXiv:1805.10820."},{"issue":"1","key":"9591_CR20","doi-asserted-by":"publisher","first-page":"335","DOI":"10.2307\/2026523","volume":"82","author":"J Hardwig","year":"1985","unstructured":"Hardwig, J. (1985). Epistemic dependence. The Journal of Philosophy, 82(1), 335\u2013349.","journal-title":"the Journal of Philosophy"},{"issue":"10","key":"9591_CR21","doi-asserted-by":"publisher","first-page":"571","DOI":"10.5840\/jphil20081051022","volume":"105","author":"J Hawthorne","year":"2008","unstructured":"Hawthorne, J., & Stanley, J. (2008). Knowledge and action. Journal of Philosophy, 105(10), 571\u2013590.","journal-title":"Journal of Philosophy"},{"issue":"3\u20134","key":"9591_CR22","doi-asserted-by":"publisher","first-page":"6665","DOI":"10.1007\/s11229-021-03086-8","volume":"199","author":"T Henning","year":"2021","unstructured":"Henning, T. (2021). An epistemic modal norm of practical reasoning. Synthese, 199(3\u20134), 6665\u20136686.","journal-title":"Synthese"},{"issue":"3","key":"9591_CR23","doi-asserted-by":"publisher","first-page":"615","DOI":"10.1007\/s11229-008-9435-2","volume":"169","author":"P Humphreys","year":"2009","unstructured":"Humphreys, P. (2009). The philosophical novelty of computer simulation methods. Synthese, 169(3), 615\u2013626.","journal-title":"Synthese"},{"issue":"2","key":"9591_CR24","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1080\/00048402.2011.587438","volume":"90","author":"A Jackson","year":"2012","unstructured":"Jackson, A. (2012). Two ways to put knowledge first. Australasian Journal of Philosophy, 90(2), 353\u2013369.","journal-title":"Australasian Journal of Philosophy"},{"key":"9591_CR25","doi-asserted-by":"crossref","unstructured":"Jacovi, A., Marasovi\u0107, A., Miller, T., & Goldberg, Y. (2021). Formalizing trust in artificial intelligence: Prerequisites, causes and goals of human trust in AI. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency (pp. 624\u2013635).","DOI":"10.1145\/3442188.3445923"},{"key":"9591_CR26","doi-asserted-by":"publisher","first-page":"2715","DOI":"10.1007\/s11229-014-0415-4","volume":"191","author":"R Klein","year":"2014","unstructured":"Klein, R. (2014). Where there are internal defeaters, there are \u201cconfirmers.\u201d Synthese, 191, 2715\u20132728.","journal-title":"Synthese"},{"key":"9591_CR27","doi-asserted-by":"crossref","unstructured":"Kotzen M. (2019) A Formal Account of Epistemic Defeat. In: B. Fitelson, R. Borges & C. Braden (Eds.) Themes from Klein. Synthese library (Studies in epistemology, logic, methodology, and philosophy of science) (Vol. 404, pp. 213\u2013234). Springer.","DOI":"10.1007\/978-3-030-04522-7_14"},{"key":"9591_CR28","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1111\/j.1933-1592.2006.tb00605.x","volume":"73","author":"J Lackey","year":"2006","unstructured":"Lackey, J. (2006). Learning from words. Philosophy and Phenomenological Research, 73, 77\u2013101.","journal-title":"Philosophy and Phenomenological Research"},{"key":"9591_CR29","volume-title":"Theory of knowledge","author":"K Lehrer","year":"1990","unstructured":"Lehrer, K. (1990). Theory of knowledge. Routledge."},{"issue":"8","key":"9591_CR30","doi-asserted-by":"publisher","first-page":"463","DOI":"10.5840\/jphil2009106841","volume":"106","author":"C Littlejohn","year":"2009","unstructured":"Littlejohn, C. (2009). Must we act only on what we know? Journal of Philosophy, 106(8), 463\u2013473.","journal-title":"Journal of Philosophy"},{"key":"9591_CR31","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1016\/j.obhdp.2018.12.005","volume":"151","author":"J Logg","year":"2019","unstructured":"Logg, J., Minson, J., & Moore, D. (2019). Algorithm appreciation: People prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes, 151, 90\u2013103.","journal-title":"Organizational Behavior and Human Decision Processes"},{"issue":"CSCW1","key":"9591_CR32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3449180","volume":"5","author":"H Lyons","year":"2021","unstructured":"Lyons, H., Velloso, E., & Miller, T. (2021). Conceptualising contestability: Perspectives on contesting algorithmic decisions. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), 1\u201325.","journal-title":"Proceedings of the ACM on Human-Computer Interaction"},{"key":"9591_CR33","doi-asserted-by":"publisher","DOI":"10.7208\/chicago\/9780226511993.001.0001","volume-title":"Error and the growth of experimental knowledge","author":"D Mayo","year":"1996","unstructured":"Mayo, D. (1996). Error and the growth of experimental knowledge. The University of Chicago Press."},{"key":"9591_CR34","doi-asserted-by":"publisher","DOI":"10.1017\/9781107286184","volume-title":"Statistical inference as severe testing: How to get beyond the statistics wars","author":"D Mayo","year":"2018","unstructured":"Mayo, D. (2018). Statistical inference as severe testing: How to get beyond the statistics wars. Cambridge University Press."},{"issue":"3","key":"9591_CR35","doi-asserted-by":"publisher","first-page":"681","DOI":"10.1111\/phpr.12222","volume":"93","author":"N Mehta","year":"2016","unstructured":"Mehta, N. (2016). Knowledge and other norms for assertion, action, and belief: A teleological account. Philosophy and Phenomenological Research, 93(3), 681\u2013705.","journal-title":"Philosophy and Phenomenological Research"},{"issue":"1\u20132","key":"9591_CR36","doi-asserted-by":"publisher","first-page":"5395","DOI":"10.1007\/s11229-021-03030-w","volume":"199","author":"A Mueller","year":"2021","unstructured":"Mueller, A. (2021). The knowledge norm of apt practical reasoning. Synthese, 199(1\u20132), 5395\u20135414.","journal-title":"Synthese"},{"issue":"511","key":"9591_CR37","doi-asserted-by":"publisher","first-page":"887","DOI":"10.1093\/mind\/fzy043","volume":"128","author":"D Mu\u00f1oz","year":"2019","unstructured":"Mu\u00f1oz, D. (2019). Defeaters and Disqualifiers. Mind, 128(511), 887\u2013906.","journal-title":"Mind"},{"key":"9591_CR38","unstructured":"Pearce, T., Brintrup, A., Zaki, M., & Neely, A. (2018). High-quality prediction intervals for deep learning: A distribution-free, ensembled approach. In International conference on machine learning (pp. 4075\u20134084)."},{"key":"9591_CR39","volume-title":"Causality: Models, reasoning and inference","author":"J Pearl","year":"2000","unstructured":"Pearl, J. (2000). Causality: Models, reasoning and inference. Cambridge University Press."},{"key":"9591_CR40","volume-title":"Contemporary theories of knowledge","author":"J Pollock","year":"1986","unstructured":"Pollock, J., & Cruz, J. (1986). Contemporary theories of knowledge. Rowman and Littlefield."},{"key":"9591_CR41","doi-asserted-by":"crossref","unstructured":"Poursabzi-Sangdeh, F., Goldstein, D., Hofman, J., Wortman Vaughan, J., & Wallach, H. (2021). Manipulating and measuring model interpretability. In Proceedings of the 2021 CHI conference on human factors in computing systems (pp. 1\u201352).","DOI":"10.1145\/3411764.3445315"},{"key":"9591_CR42","doi-asserted-by":"crossref","unstructured":"Raji, I., Smart, A., White, R., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020). Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In FAT* \u201920: Proceedings of the 2020 conference on fairness, accountability, and transparency (pp. 33\u201344), January 2020.","DOI":"10.1145\/3351095.3372873"},{"key":"9591_CR43","series-title":"NATO ASI Series (Series D: Behavioural and Social Sciences","volume-title":"Social responses to large technical systems","author":"G Rochlin","year":"1991","unstructured":"Rochlin, G. (1991). Iran Air Flight 655 and the USS Vincennes. NATO ASI Series (Series D: Behavioural and Social SciencesIn T. R. La Porte (Ed.), Social responses to large technical systems.  (Vol. 58). Springer."},{"key":"9591_CR44","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1007\/s10676-009-9183-0","volume":"11","author":"E Rooksby","year":"2009","unstructured":"Rooksby, E. (2009). How to be a responsible slave: Managing the use of expert information systems. Ethics and Information Technology, 11, 81\u201390.","journal-title":"Ethics and Information Technology"},{"key":"9591_CR45","unstructured":"Spangher, A., Ustun, B., & Liu, Y. (2018). Actionable recourse in linear classification. In Proceedings of the 5th workshop on fairness, accountability and transparency in machine learning."},{"key":"9591_CR46","volume-title":"Failure mode and effect analysis: FMEA from theory to execution","author":"D Stamatis","year":"2003","unstructured":"Stamatis, D. (2003). Failure mode and effect analysis: FMEA from theory to execution. American Society for Quality, Quality Press."},{"key":"9591_CR47","first-page":"97","volume-title":"Public administration in an information age: A handbook","author":"J van den Hoven","year":"1998","unstructured":"van den Hoven, J. (1998). Moral responsibility, public office and information technology. In I. Snellen & W. van de Donk (Eds.), Public administration in an information age: A handbook (pp. 97\u2013112). IOS Press."},{"key":"9591_CR48","unstructured":"van Huffelen, A. C. (2020). Kamerstuk II 2019\/20, 31 066, Nr. 683. https:\/\/zoek.officielebekendmakingen.nl\/kst-31066-683.html"},{"issue":"2","key":"9591_CR49","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1215\/00318108-114-2-227","volume":"114","author":"M Weiner","year":"2005","unstructured":"Weiner, M. (2005). Must we know what we say? Philosophical Review, 114(2), 227\u2013251.","journal-title":"Philosophical Review"},{"issue":"219","key":"9591_CR50","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1111\/j.0031-8094.2005.00396.x","volume":"55","author":"T Williamson","year":"2005","unstructured":"Williamson, T. (2005). Contextualism, subject-sensitive invariantism and knowledge of knowledge. The Philosophical Quarterly, 55(219), 213\u2013235.","journal-title":"The Philosophical Quarterly"},{"key":"9591_CR51","unstructured":"Wu, X., & Zhang, X. (2016). Automated inference on criminality using face images, pp. 4038\u20134052. arXiv preprint. arXiv:1611.04135"}],"container-title":["Minds and Machines"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11023-022-09591-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11023-022-09591-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11023-022-09591-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,18]],"date-time":"2024-03-18T15:08:53Z","timestamp":1710774533000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11023-022-09591-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,21]]},"references-count":51,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["9591"],"URL":"https:\/\/doi.org\/10.1007\/s11023-022-09591-0","relation":{},"ISSN":["1572-8641"],"issn-type":[{"value":"1572-8641","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,21]]},"assertion":[{"value":"31 August 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 January 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 January 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}