{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T02:13:24Z","timestamp":1780366404462,"version":"3.54.1"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2025,1,23]],"date-time":"2025-01-23T00:00:00Z","timestamp":1737590400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,1,23]],"date-time":"2025-01-23T00:00:00Z","timestamp":1737590400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100008131","name":"Rheinische Friedrich-Wilhelms-Universit\u00e4t Bonn","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100008131","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["AI &amp; Soc"],"published-print":{"date-parts":[[2025,6]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Meaningful human control (MHC) is increasingly becoming an important topic in AI ethics beyond the domain of autonomous weapons systems. MHC has been conceptualized, analyzed, and applied. However, in this article, I show how all the current attempts at realizing MHC have fallen short because we have not taken the important first step of deciding what machines should and should not be doing in the first place. We must first ensure that the output we have delegated to the machine is appropriate \u2013 only then do we have to do the work required to ensure that MHC is realized. Here, I argue that machines should not be evaluating \u2013 that is, we should not be delegating evaluative outputs to machines. This is practically important because machines that have evaluative outputs cannot be evaluated for efficacy. We can\u2019t say how effective they are. This is ethically important because machines don\u2019t have the moral agency required to make evaluations. Furthermore, machines should not be in a position to change our values \u2013 which they would be if they were evaluating. Finally, evaluations are judged based on the considerations used to reach a particular judgment. Contemporary AI cannot provide these justifying considerations \u2013 so we have no way of evaluating their evaluative outputs.<\/jats:p>","DOI":"10.1007\/s00146-024-02169-7","type":"journal-article","created":{"date-parts":[[2025,1,23]],"date-time":"2025-01-23T04:25:04Z","timestamp":1737606304000},"page":"4093-4104","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["What machines shouldn\u2019t do"],"prefix":"10.1007","volume":"40","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5338-295X","authenticated-orcid":false,"given":"Scott","family":"Robbins","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,1,23]]},"reference":[{"key":"2169_CR1","doi-asserted-by":"publisher","first-page":"52138","DOI":"10.1109\/ACCESS.2018.2870052","volume":"6","author":"A Adadi","year":"2018","unstructured":"Adadi A, Berrada M (2018) Peeking Inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6:52138\u201352160. https:\/\/doi.org\/10.1109\/ACCESS.2018.2870052","journal-title":"IEEE Access"},{"issue":"4","key":"2169_CR2","first-page":"15","volume":"28","author":"M Anderson","year":"2007","unstructured":"Anderson M, Anderson SL (2007) Machine Ethics: Creating an Ethical Intelligent Agent. AI Mag 28(4):15\u201326","journal-title":"AI Mag"},{"key":"2169_CR3","unstructured":"Article 36. (2015, April). Killing by Machine: Key Issues for Understanding Meaningful Human Control. http:\/\/www.article36.org\/autonomous-weapons\/killing-by-machine-key-issues-for-understanding-meaningful-human-control\/"},{"key":"2169_CR4","unstructured":"Branwen, G. (2011). The Neural Net Tank Urban Legend. https:\/\/gwern.net\/tank. Accessed 13 November 2024."},{"key":"2169_CR5","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1075\/nlp.8.11bry","volume-title":"Close engagements with artificial companions: key social, psychological, ethical and design issues","author":"JJ Bryson","year":"2010","unstructured":"Bryson JJ (2010) Robots Should Be Slaves. In: Wilks Y (ed) Close engagements with artificial companions: key social, psychological, ethical and design issues. John Benjamins Publishing, pp 63\u201374"},{"key":"2169_CR6","doi-asserted-by":"publisher","DOI":"10.2514\/6.2004-6313","author":"M Cummings","year":"2012","unstructured":"Cummings M (2012) Automation bias in intelligent time critical decision support systems in aiaa 1st intelligent systems technical conference. Am Inst Aeronautics Astronautics. https:\/\/doi.org\/10.2514\/6.2004-6313","journal-title":"Am Inst Aeronautics Astronautics"},{"key":"2169_CR7","unstructured":"Dastin, J. (2018, October 10). Amazon scraps secret AI recruiting tool that showed bias against women. Reuters. https:\/\/www.reuters.com\/article\/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G. Accessed 13 November 2024."},{"key":"2169_CR8","doi-asserted-by":"publisher","first-page":"196","DOI":"10.1016\/j.trf.2014.06.016","volume":"27","author":"JCF de Winter","year":"2014","unstructured":"de Winter JCF, Happee R, Martens MH, Stanton NA (2014) Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence. Transport Res F: Traffic Psychol Behav 27:196\u2013217. https:\/\/doi.org\/10.1016\/j.trf.2014.06.016","journal-title":"Transport Res F: Traffic Psychol Behav"},{"key":"2169_CR9","doi-asserted-by":"publisher","DOI":"10.1111\/1758-5899.12665","author":"M Ekelhof","year":"2019","unstructured":"Ekelhof M (2019) Moving beyond semantics on autonomous weapons meaningful human control in operation. Global Pol. https:\/\/doi.org\/10.1111\/1758-5899.12665","journal-title":"Global Pol"},{"issue":"4","key":"2169_CR10","doi-asserted-by":"publisher","first-page":"689","DOI":"10.1007\/s11023-018-9482-5","volume":"28","author":"L Floridi","year":"2018","unstructured":"Floridi L, Cowls J, Beltrametti M, Chatila R, Chazerand P, Dignum V, Luetge C, Madelin R, Pagallo U, Rossi F, Schafer B, Valcke P, Vayena E (2018) AI4People\u2014an ethical framework for a good ai society: opportunities, risks, principles, and recommendations. Mind Mach 28(4):689\u2013707. https:\/\/doi.org\/10.1007\/s11023-018-9482-5","journal-title":"Mind Mach"},{"key":"2169_CR11","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(22)00063-2","author":"JW Gichoya","year":"2022","unstructured":"Gichoya JW, Banerjee I, Bhimireddy AR, Burns JL, Celi LA, Chen L-C, Correa R, Dullerud N, Ghassemi M, Huang S-C, Kuo P-C, Lungren MP, Palmer LJ, Price BJ, Purkayastha S, Pyrros AT, Oakden-Rayner L, Okechukwu C, Seyyed-Kalantari L, Zhang H (2022) AI recognition of patient race in medical imaging: A modelling study. Lancet Dig Health. https:\/\/doi.org\/10.1016\/S2589-7500(22)00063-2","journal-title":"Lancet Dig Health"},{"issue":"4","key":"2169_CR12","doi-asserted-by":"publisher","first-page":"814","DOI":"10.1037\/0033-295X.108.4.814","volume":"108","author":"J Haidt","year":"2001","unstructured":"Haidt J (2001) The emotional dog and its rational tail: a social intuitionist approach to moral judgment. Psychol Rev 108(4):814\u2013834. https:\/\/doi.org\/10.1037\/0033-295X.108.4.814","journal-title":"Psychol Rev"},{"issue":"2","key":"2169_CR13","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1007\/s13347-022-00546-y","volume":"35","author":"C Herzog","year":"2022","unstructured":"Herzog C (2022) On the ethical and epistemological utility of explicable ai in medicine. Philosophy Technol 35(2):50. https:\/\/doi.org\/10.1007\/s13347-022-00546-y","journal-title":"Philosophy Technol"},{"key":"2169_CR14","unstructured":"Human Rights Watch. (2016, April 11). Killer Robots and the Concept of Meaningful Human Control. Human Rights Watch. https:\/\/www.hrw.org\/news\/2016\/04\/11\/killer-robots-and-concept-meaningful-human-control"},{"issue":"4","key":"2169_CR15","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1007\/s10676-006-9111-5","volume":"8","author":"DG Johnson","year":"2006","unstructured":"Johnson DG (2006) Computer systems: Moral entities but not moral agents. Ethics Inf Technol 8(4):195\u2013204. https:\/\/doi.org\/10.1007\/s10676-006-9111-5","journal-title":"Ethics Inf Technol"},{"key":"2169_CR16","volume-title":"Thinking, Fast and Slow","author":"D Kahneman","year":"2013","unstructured":"Kahneman D (2013) Thinking, Fast and Slow, Reprint. Farrar, Straus and Giroux","edition":"Reprint"},{"key":"2169_CR17","unstructured":"King, A. (2017, July 18). Cameras with inbuilt detective skills to catch rural criminals. Horizon: EU Research and Innovation Magazine. https:\/\/ec.europa.eu\/research-and-innovation\/en\/horizon-magazine\/cameras-inbuilt-detective-skills-catch-rural-criminals"},{"key":"2169_CR18","unstructured":"Loitering. (2020). In Wikipedia. https:\/\/en.wikipedia.org\/w\/index.php?title=Loitering&oldid=948660532"},{"issue":"1","key":"2169_CR19","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1002\/hast.973","volume":"49","author":"AJ London","year":"2019","unstructured":"London AJ (2019) Artificial intelligence and black-box medical decisions: accuracy versus explainability. Hastings Cent Rep 49(1):15\u201321. https:\/\/doi.org\/10.1002\/hast.973","journal-title":"Hastings Cent Rep"},{"issue":"3","key":"2169_CR20","doi-asserted-by":"publisher","first-page":"421","DOI":"10.1007\/s11023-021-09570-x","volume":"31","author":"J Maclure","year":"2021","unstructured":"Maclure J (2021) AI, explainability and public reason: the argument from the limitations of the human mind. Mind Mach 31(3):421\u2013438. https:\/\/doi.org\/10.1007\/s11023-021-09570-x","journal-title":"Mind Mach"},{"key":"2169_CR21","first-page":"1257","volume":"105","author":"AER Prince","year":"2019","unstructured":"Prince AER, Schwarcz D (2019) Proxy discrimination in the age of artificial intelligence and big data. Iowa Law Review 105:1257","journal-title":"Iowa Law Review"},{"key":"2169_CR22","unstructured":"Riedl, M. O., & Harrison, B. (2016, March 29). Using Stories to Teach Human Values to Artificial Agents. Workshops at the Thirtieth AAAI Conference on Artificial Intelligence. Workshops at the Thirtieth AAAI Conference on Artificial Intelligence. https:\/\/www.aaai.org\/ocs\/index.php\/WS\/AAAIW16\/paper\/view\/12624"},{"issue":"4","key":"2169_CR23","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1007\/s11023-019-09509-3","volume":"29","author":"S Robbins","year":"2019","unstructured":"Robbins S (2019a) A misdirected principle with a catch: explicability for AI. Mind Mach 29(4):495\u2013514. https:\/\/doi.org\/10.1007\/s11023-019-09509-3","journal-title":"Mind Mach"},{"key":"2169_CR24","doi-asserted-by":"publisher","DOI":"10.1007\/s00146-019-00891-1","author":"S Robbins","year":"2019","unstructured":"Robbins S (2019b) AI and the path to envelopment: Knowledge as a first step towards the responsible regulation and use of AI-powered machines. AI & Soc. https:\/\/doi.org\/10.1007\/s00146-019-00891-1","journal-title":"AI & Soc"},{"key":"2169_CR25","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1007\/978-3-031-34804-4_8","volume-title":"Recommender systems: legal and ethical issues","author":"S Robbins","year":"2023","unstructured":"Robbins S (2023a) Recommending ourselves to death: values in the age of algorithms. In: Genovesi S, Kaesling K, Robbins S (eds) Recommender systems: legal and ethical issues. Springer International Publishing, NY, pp 147\u2013161"},{"key":"2169_CR26","doi-asserted-by":"publisher","DOI":"10.1007\/s43681-023-00320-6","author":"S Robbins","year":"2023","unstructured":"Robbins S (2023b) The many meanings of meaningful human control. AI and Ethics. https:\/\/doi.org\/10.1007\/s43681-023-00320-6","journal-title":"AI and Ethics"},{"issue":"15","key":"2169_CR27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/frobt.2018.00015","volume":"5","author":"F Santoni de Sio","year":"2018","unstructured":"Santoni de Sio F, van den Hoven J (2018) Meaningful human control over autonomous systems: a philosophical account. Front Robot AI 5(15):1\u201314. https:\/\/doi.org\/10.3389\/frobt.2018.00015","journal-title":"Front Robot AI"},{"key":"2169_CR28","first-page":"515","volume-title":"Fundamental issues of artificial intelligence","author":"M Scheutz","year":"2016","unstructured":"Scheutz M (2016) The need for moral competency in autonomous agent architectures. In: M\u00fcller VC (ed) Fundamental issues of artificial intelligence. Springer International Publishing, NY, pp 515\u2013525"},{"issue":"886","key":"2169_CR29","doi-asserted-by":"publisher","first-page":"787","DOI":"10.1017\/S1816383112000732","volume":"94","author":"N Sharkey","year":"2012","unstructured":"Sharkey N (2012) The evitability of autonomous robot warfare. International Review of the Red Cross 94(886):787\u2013799. https:\/\/doi.org\/10.1017\/S1816383112000732","journal-title":"International Review of the Red Cross"},{"key":"2169_CR30","doi-asserted-by":"publisher","DOI":"10.4476\/77105","author":"N Sharkey","year":"2014","unstructured":"Sharkey N (2014) Towards a principle for the human supervisory control of robot weapons. Politica Societ\u00e0. https:\/\/doi.org\/10.4476\/77105","journal-title":"Politica Societ\u00e0"},{"issue":"4","key":"2169_CR31","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1007\/s10676-017-9425-5","volume":"22","author":"A Sharkey","year":"2020","unstructured":"Sharkey A (2020) Can we program or train robots to be good? Ethics Inf Technol 22(4):283\u2013295. https:\/\/doi.org\/10.1007\/s10676-017-9425-5","journal-title":"Ethics Inf Technol"},{"issue":"5","key":"2169_CR32","doi-asserted-by":"publisher","first-page":"991","DOI":"10.1006\/ijhc.1999.0252","volume":"51","author":"LJ Skitka","year":"1999","unstructured":"Skitka LJ, Mosier KL, Burdick M (1999) Does automation bias decision-making? Int J Hum Comput Stud 51(5):991\u20131006. https:\/\/doi.org\/10.1006\/ijhc.1999.0252","journal-title":"Int J Hum Comput Stud"},{"issue":"3","key":"2169_CR33","doi-asserted-by":"publisher","first-page":"421","DOI":"10.1007\/s11023-009-9159-1","volume":"19","author":"R Tonkens","year":"2009","unstructured":"Tonkens R (2009) A challenge for machine ethics. Mind Mach 19(3):421. https:\/\/doi.org\/10.1007\/s11023-009-9159-1","journal-title":"Mind Mach"},{"issue":"3","key":"2169_CR34","doi-asserted-by":"publisher","first-page":"719","DOI":"10.1007\/s11948-018-0030-8","volume":"25","author":"A van Wynsberghe","year":"2019","unstructured":"van Wynsberghe A, Robbins S (2019) Critiquing the reasons for making artificial moral agents. Sci Eng Ethics 25(3):719\u2013735. https:\/\/doi.org\/10.1007\/s11948-018-0030-8","journal-title":"Sci Eng Ethics"},{"key":"2169_CR35","unstructured":"Verma, P. (2022, July 18). The never-ending quest to predict crime using AI. Washington Post. https:\/\/www.washingtonpost.com\/technology\/2022\/07\/15\/predictive-policing-algorithms-fail\/"},{"key":"2169_CR36","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aan6080","author":"S Wachter","year":"2017","unstructured":"Wachter S, Mittelstadt B, Floridi L (2017) Transparent, explainable, and accountable AI for robotics. Sci Robot. https:\/\/doi.org\/10.1126\/scirobotics.aan6080","journal-title":"Sci Robot"},{"issue":"4","key":"2169_CR37","doi-asserted-by":"publisher","first-page":"463","DOI":"10.1007\/s00146-007-0093-6","volume":"22","author":"W Wallach","year":"2007","unstructured":"Wallach W (2007) Implementing moral decision making faculties in computers and robots. AI & Soc 22(4):463\u2013475. https:\/\/doi.org\/10.1007\/s00146-007-0093-6","journal-title":"AI & Soc"},{"key":"2169_CR38","volume-title":"Moral machines: teaching robots right from wrong (1 edition)","author":"W Wallach","year":"2010","unstructured":"Wallach W, Allen C (2010) Moral machines: teaching robots right from wrong (1 edition). Oxford University Press"},{"key":"2169_CR39","doi-asserted-by":"publisher","DOI":"10.1007\/s43681-024-00566-8","author":"M Zafar","year":"2024","unstructured":"Zafar M (2024) Normativity and AI moral agency. AI Ethics. https:\/\/doi.org\/10.1007\/s43681-024-00566-8","journal-title":"AI Ethics"}],"container-title":["AI &amp; SOCIETY"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00146-024-02169-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00146-024-02169-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00146-024-02169-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,10]],"date-time":"2025-06-10T04:18:23Z","timestamp":1749529103000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00146-024-02169-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,23]]},"references-count":39,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["2169"],"URL":"https:\/\/doi.org\/10.1007\/s00146-024-02169-7","relation":{},"ISSN":["0951-5666","1435-5655"],"issn-type":[{"value":"0951-5666","type":"print"},{"value":"1435-5655","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,23]]},"assertion":[{"value":"2 July 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 December 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 January 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"On behalf of all authors, the corresponding author states that there is no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}