{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T14:11:29Z","timestamp":1778940689819,"version":"3.51.4"},"reference-count":27,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2022,3,16]],"date-time":"2022-03-16T00:00:00Z","timestamp":1647388800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,16]],"date-time":"2022-03-16T00:00:00Z","timestamp":1647388800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100007210","name":"RWTH Aachen University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007210","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["AI &amp; Soc"],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>This paper explores the role and resolution of disagreements between physicians and their diagnostic AI-based decision support systems (DSS). With an ever-growing number of applications for these independently operating diagnostic tools, it becomes less and less clear what a physician ought to do in case their diagnosis is in faultless conflict with the results of the DSS. The consequences of such uncertainty can ultimately lead to effects detrimental to the intended purpose of such machines, e.g. by shifting the burden of proof towards a physician. Thus, we require normative clarity for integrating these machines without affecting established, trusted, and relied upon workflows. In reconstructing different causes of conflicts between physicians and their AI-based tools\u2014inspired by the approach of \u201cmeaningful human control\u201d over autonomous systems and the challenges to resolve them\u2014we will delineate normative conditions for \u201cmeaningful disagreements\u201d. These incorporate the potential of DSS to take on more tasks and outline how the moral responsibility of a physician can be preserved in an increasingly automated clinical work environment.<\/jats:p>","DOI":"10.1007\/s00146-022-01418-x","type":"journal-article","created":{"date-parts":[[2022,3,16]],"date-time":"2022-03-16T08:02:33Z","timestamp":1647417753000},"page":"1407-1414","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["\u201cI\u2019m afraid I can\u2019t let you do that, Doctor\u201d: meaningful disagreements with AI in medical contexts"],"prefix":"10.1007","volume":"38","author":[{"given":"Hendrik","family":"Kempt","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jan-Christoph","family":"Heilinger","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Saskia K.","family":"Nagel","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,3,16]]},"reference":[{"issue":"2","key":"1418_CR1","doi-asserted-by":"publisher","first-page":"207","DOI":"10.1001\/journalofethics.2017.19.2.sect1-1702","volume":"19","author":"AI Applbaum","year":"2017","unstructured":"Applbaum AI (2017) The idea of legitimate authority in the practice of medicine. AMA J Ethics 19(2):207\u2013213","journal-title":"AMA J Ethics"},{"issue":"2","key":"1418_CR2","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1080\/1359432X.2019.1620328","volume":"29","author":"S Bankins","year":"2019","unstructured":"Bankins S, Formosa P (2019) When AI meets PC: exploring the implications of workplace social robots and a human-robot psychological contract. Eur J Work Organ Psychol 29(2):215\u2013229","journal-title":"Eur J Work Organ Psychol"},{"key":"1418_CR3","doi-asserted-by":"publisher","DOI":"10.1136\/medethics-2019-105860","author":"M Braun","year":"2020","unstructured":"Braun M, Hummel P, Beck S, Dabrock P (2020) Primer on an ethics of AI-based decision support systems in the clinic. J Med Eth. https:\/\/doi.org\/10.1136\/medethics-2019-105860","journal-title":"J Med Eth"},{"key":"1418_CR4","doi-asserted-by":"publisher","first-page":"17025","DOI":"10.1057\/palcomms.2017.25","volume":"3","author":"Q Cassam","year":"2017","unstructured":"Cassam Q (2017) Diagnostic error, overconfidence and self-knowledge. Palgrave Commun 3:17025. https:\/\/doi.org\/10.1057\/palcomms.2017.25","journal-title":"Palgrave Commun"},{"key":"1418_CR5","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1215\/00318108-2006-035","volume":"116","author":"D Christensen","year":"2007","unstructured":"Christensen D (2007) Epistemology of disagreement: the good news. Philos Rev 116:187\u2013218","journal-title":"Philos Rev"},{"key":"1418_CR6","doi-asserted-by":"crossref","unstructured":"Denecke K, Hochreutner SL, P\u00f6pel A, May R (2018) Talking to ana: a mobile self-anamnesis application with conversational user interface. In: Proceedings of the 2018 international conference on digital health, pp 85\u201389","DOI":"10.1145\/3194658.3194670"},{"key":"1418_CR7","doi-asserted-by":"publisher","DOI":"10.1037\/14663-000","volume-title":"Human nature and conduct. An introduction to social psychology","author":"J Dewey","year":"1922","unstructured":"Dewey J (1922) Human nature and conduct. An introduction to social psychology. Holt, New York"},{"issue":"1","key":"1418_CR8","first-page":"9808","volume":"33","author":"P Flach","year":"2019","unstructured":"Flach P (2019) Performance evaluation in machine learning: the good, the bad, the ugly, and the way forward. Proc AAAI Conf Artif Intell 33(1):9808\u20139814","journal-title":"Proc AAAI Conf Artif Intell"},{"issue":"4","key":"1418_CR9","doi-asserted-by":"publisher","first-page":"329","DOI":"10.1016\/j.carj.2019.08.010","volume":"70","author":"JR Geis","year":"2019","unstructured":"Geis JR, Brady AP, Wu CC, Spencer J, Ranschaert E, Jaremko JL, Kohli M et al (2019) Ethics of artificial intelligence in radiology: summary of the joint European and North American multisociety statement. Can Assoc Radiol J 70(4):329\u2013334","journal-title":"Can Assoc Radiol J"},{"issue":"3","key":"1418_CR10","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1136\/medethics-2019-105586","volume":"46","author":"T Grote","year":"2020","unstructured":"Grote T, Berens P (2020) On the ethics of algorithmic decision-making in healthcare. J Med Eth 46(3):205\u2013211","journal-title":"J Med Eth"},{"key":"1418_CR11","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1515\/9783110379006-009","volume-title":"Moral, Wissenschaft Und Wahrheit","author":"J-C Heilinger","year":"2016","unstructured":"Heilinger J-C (2016) Konflikte in Der Ethik. Anmerkungen aus pragmatistischer Perspektive. In: Nida-R\u00fcmelin J, Heilinger J-C (eds) Moral, Wissenschaft Und Wahrheit. de Gruyter, Berlin, pp 145\u2013159"},{"key":"1418_CR12","volume-title":"Cosmopolitan responsibility. Global injustice, relational equality, and individual agency","author":"J-C Heilinger","year":"2020","unstructured":"Heilinger J-C (2020) Cosmopolitan responsibility. Global injustice, relational equality, and individual agency. de Gruyter, Berlin"},{"key":"1418_CR13","unstructured":"International Medical Device Regulators Forum-IMDRF (2013) Software as a Medical Device (SaMD): Key Definitions. https:\/\/www.imdrf.org\/sites\/default\/files\/docs\/imdrf\/final\/technical\/imdrf-tech-131209-samd-key-definitions-140901.pdf"},{"issue":"6","key":"1418_CR14","doi-asserted-by":"publisher","first-page":"3121","DOI":"10.1007\/s11948-020-00259-5","volume":"26","author":"S K\u00f6hler","year":"2020","unstructured":"K\u00f6hler S (2020) Instrumental robots. Sci Eng Ethics 26(6):3121\u20133141","journal-title":"Sci Eng Ethics"},{"issue":"1","key":"1418_CR15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-020-00367-3","volume":"4","author":"B Kompa","year":"2021","unstructured":"Kompa B, Snoek J, Beam AL (2021) Second opinion needed: communicating uncertainty in medical machine learning. NPJ Digit Med 4(1):1\u20136","journal-title":"NPJ Digit Med"},{"key":"1418_CR16","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1007\/978-3-662-45854-9_4","volume-title":"Autonomes Fahren","author":"P Lin","year":"2015","unstructured":"Lin P (2015) Why ethics matters for autonomous cars. Autonomes Fahren. Springer, Berlin, pp 69\u201385"},{"issue":"3","key":"1418_CR17","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1136\/medethics-2018-105118","volume":"45","author":"RJ McDougall","year":"2019","unstructured":"McDougall RJ (2019) Computer knows best? The need for value-flexibility in medical AI. J Med Eth 45(3):156\u2013160","journal-title":"J Med Eth"},{"key":"1418_CR18","doi-asserted-by":"crossref","unstructured":"Mittelstadt B, Russel C, Wachter S (2019) Explaining Explanations in AI. In: Proceedings of the conference on fairness, accountability, and transparency, pp 279\u2013288","DOI":"10.1145\/3287560.3287574"},{"key":"1418_CR19","unstructured":"Mozannar H, Sontag D (2020) Consistent estimators for learning to defer to an expert. In: International conference on machine learning. PMLR, pp 7076\u20137087"},{"key":"1418_CR100","doi-asserted-by":"publisher","first-page":"1201","DOI":"10.1007\/s11948-017-9943-x","volume":"24","author":"S Nyholm","year":"2018","unstructured":"Nyholm S (2018) Attributing agency to automated systems: reflections on human-robot collaborations and responsibility-loci. Sci Eng Ethics 24:1201\u20131219. https:\/\/doi.org\/10.1007\/s11948-017-9943-x","journal-title":"Sci Eng Ethics"},{"key":"1418_CR20","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1038\/s41746-019-0189-7","volume":"2","author":"BN Patel","year":"2019","unstructured":"Patel BN, Rosenberg L, Willcox G et al (2019) Human\u2013machine partnership with artificial intelligence for chest radiograph diagnosis. NPJ Digit Med 2:111. https:\/\/doi.org\/10.1038\/s41746-019-0189-7","journal-title":"NPJ Digit Med"},{"issue":"48","key":"1418_CR21","doi-asserted-by":"publisher","first-page":"1332","DOI":"10.1503\/cmaj.190506","volume":"191","author":"T Pelaccia","year":"2019","unstructured":"Pelaccia T, Forestier G, Wemmert C (2019) Deconstructing the diagnostic reasoning of human versus artificial intelligence. CMAJ 191(48):1332\u20131335","journal-title":"CMAJ"},{"key":"1418_CR22","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(20)31643-3","author":"W Rogers","year":"2020","unstructured":"Rogers W (2020) Moral responsibility in medicine: where are the boundaries? Lancet. https:\/\/doi.org\/10.1016\/S0140-6736(20)31643-3","journal-title":"Lancet"},{"key":"1418_CR23","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1038\/s41746-020-0221-y","volume":"3","author":"RT Sutton","year":"2020","unstructured":"Sutton RT, Pincock D, Baumgart DC et al (2020) An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 3:17. https:\/\/doi.org\/10.1038\/s41746-020-0221-y","journal-title":"NPJ Digit Med"},{"issue":"1","key":"1418_CR24","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","volume":"25","author":"EJ Topol","year":"2019","unstructured":"Topol EJ (2019) High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25(1):44\u201356","journal-title":"Nat Med"},{"issue":"12","key":"1418_CR25","doi-asserted-by":"publisher","first-page":"677","DOI":"10.1016\/S2589-7500(20)30200-4","volume":"2","author":"J Wilkinson","year":"2020","unstructured":"Wilkinson J, Arnold KF, Murray EJ et al (2020) Time to reality check the promises of machine learning-powered precision medicine. Lancet Digit Health 2(12):677\u2013680. https:\/\/doi.org\/10.1016\/S2589-7500(20)30200-4","journal-title":"Lancet Digit Health"},{"issue":"10","key":"1418_CR26","doi-asserted-by":"publisher","first-page":"719","DOI":"10.1038\/s41551-018-0305-z","volume":"2","author":"K Yu","year":"2019","unstructured":"Yu K, Beam AL, Kohane IS (2019) artificial intelligence in healthcare. Nat Biomed Eng 2(10):719\u2013731","journal-title":"Nat Biomed Eng"}],"container-title":["AI &amp; SOCIETY"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00146-022-01418-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00146-022-01418-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00146-022-01418-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,13]],"date-time":"2023-07-13T18:13:34Z","timestamp":1689272014000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00146-022-01418-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,16]]},"references-count":27,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,8]]}},"alternative-id":["1418"],"URL":"https:\/\/doi.org\/10.1007\/s00146-022-01418-x","relation":{},"ISSN":["0951-5666","1435-5655"],"issn-type":[{"value":"0951-5666","type":"print"},{"value":"1435-5655","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,16]]},"assertion":[{"value":"30 May 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 February 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 March 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"We have no conflicts of interests to declare.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}