{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T11:06:48Z","timestamp":1779102408889,"version":"3.51.4"},"reference-count":52,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,8,1]],"date-time":"2020-08-01T00:00:00Z","timestamp":1596240000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,8,1]],"date-time":"2020-08-01T00:00:00Z","timestamp":1596240000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Minds &amp; Machines"],"published-print":{"date-parts":[[2021,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Accountability and responsibility are key concepts in the academic and societal debate on Autonomous Weapon Systems, but these notions are often used as high-level overarching constructs and are not operationalised to be useful in practice. \u201cMeaningful Human Control\u201d is often mentioned as a requirement for the deployment of Autonomous Weapon Systems, but a common definition of what this notion means in practice, and a clear understanding of its relation with responsibility and accountability is also lacking. In this paper, we present a definition of these concepts and describe the relations between accountability, responsibility, control and oversight in order to show how these notions are distinct but also connected. We focus on accountability as a particular form of responsibility\u2014the obligation to explain one\u2019s action to a forum\u2014and we present three ways in which the introduction of Autonomous Weapon Systems may create \u201caccountability gaps\u201d. We propose a Framework for Comprehensive Human Oversight based on an engineering, socio-technical and governance perspective on control. Our main claim is that combining the control mechanisms at technical, socio-technical and governance levels will lead to comprehensive human oversight over Autonomous Weapon Systems which may ensure solid controllability and accountability for the behaviour of Autonomous Weapon Systems. Finally, we give an overview of the military control instruments that are currently used in the Netherlands and show the applicability of the comprehensive human oversight Framework to Autonomous Weapon Systems. Our analysis reveals two main gaps in the current control mechanisms as applied to Autonomous Weapon Systems. We have identified three first options as future work for the design of a control mechanism, one in the technological layer, one in the socio-technical layer and one the governance layer, in order to achieve comprehensive human oversight and ensure accountability over Autonomous Weapon Systems.<\/jats:p>","DOI":"10.1007\/s11023-020-09532-9","type":"journal-article","created":{"date-parts":[[2020,8,1]],"date-time":"2020-08-01T09:13:51Z","timestamp":1596273231000},"page":"137-163","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":43,"title":["Accountability and Control Over Autonomous Weapon Systems: A Framework for Comprehensive Human Oversight"],"prefix":"10.1007","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0296-1844","authenticated-orcid":false,"given":"Ilse","family":"Verdiesen","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3940-9885","authenticated-orcid":false,"given":"Filippo","family":"Santoni de Sio","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7409-5813","authenticated-orcid":false,"given":"Virginia","family":"Dignum","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2020,8,1]]},"reference":[{"key":"9532_CR1","unstructured":"AIV and CAVV. (2015). Autonomous weapon systems: the need for meaningful human control. (No. 97, No. 26). Retrieved from https:\/\/www.advisorycouncilinternationalaffairs.nl\/binaries\/advisorycouncilinternationalaffairs\/documents\/publications\/2015\/10\/02\/autonomous-weapon-systems\/Autonomous_Weapon_Systems_AIV-Advice-97_CAVV-Advisory-report-26_ENG_201510.pdf"},{"key":"9532_CR2","unstructured":"Alberts, D. S., Hayes, R. E. (2006). Understanding command and control."},{"key":"9532_CR3","unstructured":"Alston, P. (2010). Study on Targeted Killings\u2019, Report of the UN Special Rapporteur on Extrajudicial, Summary or Arbitrary Executions, UN Doc A."},{"issue":"886","key":"9532_CR4","doi-asserted-by":"publisher","first-page":"687","DOI":"10.1017\/S1816383112000768","volume":"94","author":"P Asaro","year":"2012","unstructured":"Asaro, P. (2012). On banning autonomous weapon systems: human rights, automation, and the dehumanization of lethal decision-making. International Review of the Red Cross, 94(886), 687\u2013709.","journal-title":"International Review of the Red Cross"},{"key":"9532_CR5","series-title":"Robot law","doi-asserted-by":"publisher","DOI":"10.4337\/9781783476732.00024","volume-title":"Jus nascendi, robotic weapons and the Martens Clause","author":"P Asaro","year":"2016","unstructured":"Asaro, P. (2016). Jus nascendi, robotic weapons and the Martens Clause., Robot law Cheltenham: Edward Elgar Publishing."},{"issue":"1","key":"9532_CR6","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1016\/j.automatica.2013.10.012","volume":"50","author":"KJ \u00c5str\u00f6m","year":"2014","unstructured":"\u00c5str\u00f6m, K. J., & Kumar, P. R. (2014). Control: A perspective. Automatica, 50(1), 3\u201343.","journal-title":"Automatica"},{"issue":"4","key":"9532_CR7","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1111\/j.1468-0386.2007.00378.x","volume":"13","author":"M Bovens","year":"2007","unstructured":"Bovens, M. (2007). Analysing and assessing accountability: A conceptual framework 1. European Law Journal, 13(4), 447\u2013468.","journal-title":"European Law Journal"},{"key":"9532_CR8","first-page":"1","volume":"1","author":"M Bovens","year":"2014","unstructured":"Bovens, M., Schillemans, T., & Goodin, R. E. (2014). Public accountability. The Oxford Handbook of Public Accountability, 1, 1\u201320.","journal-title":"The Oxford Handbook of Public Accountability"},{"issue":"3","key":"9532_CR9","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1109\/MIS.2013.70","volume":"28","author":"JM Bradshaw","year":"2013","unstructured":"Bradshaw, J. M., Hoffman, R. R., Woods, D. D., & Johnson, M. (2013). The seven deadly myths of \u201cautonomous systems\u201d. IEEE Intelligent Systems, 28(3), 54\u201361.","journal-title":"IEEE Intelligent Systems"},{"key":"9532_CR10","unstructured":"Busuioc, M. (2007). Autonomy, Accountability and Control. The Case of European Agencies. In Paper presented at the 4TH ECPR General Conference, Pisa, Italy."},{"key":"9532_CR11","unstructured":"Caparini, M. (2004). Media and the security sector: Oversight and accountability. Geneva Centre for the Democratic Control of Armed Forces (DCAF) Publication, 1\u201349."},{"key":"9532_CR12","series-title":"Agent autonomy","first-page":"103","volume-title":"From automaticity to autonomy: the frontier of artificial agents","author":"C Castelfranchi","year":"2003","unstructured":"Castelfranchi, C., & Falcone, R. (2003). From automaticity to autonomy: the frontier of artificial agents (pp. 103\u2013136)., Agent autonomy Berlin: Springer."},{"key":"9532_CR13","first-page":"1347","volume":"164","author":"R Crootof","year":"2015","unstructured":"Crootof, R. (2015). War torts: Accountability for autonomous weapons. University of Pennsylvania Law Review, 164, 1347.","journal-title":"University of Pennsylvania Law Review"},{"key":"9532_CR14","doi-asserted-by":"crossref","unstructured":"Cummings, M. L. (2006). Automation and accountability in decision support system interface design.","DOI":"10.21061\/jots.v32i1.a.4"},{"issue":"11","key":"9532_CR15","doi-asserted-by":"publisher","first-page":"1987","DOI":"10.1017\/S2071832200000729","volume":"9","author":"E De Wet","year":"2008","unstructured":"De Wet, E. (2008). Holding international institutions accountable: The complementary role of non-judicial oversight mechanisms and judicial review. German Law Journal, 9(11), 1987\u20132012.","journal-title":"German Law Journal"},{"key":"9532_CR16","doi-asserted-by":"crossref","unstructured":"Dickinson, L. (2018). Lethal Autonomous Weapons Systems: The Overlooked Importance of Administrative Accountability. Lethal Autonomous Weapons Systems: The overlooked importance of administrative accountability, in The Impact of Emerging Technologies on the Law of Armed Conflict (Eric Talbot Jensen & Ronald Alcala eds., Oxford University Press 2018 Forthcoming).","DOI":"10.1093\/oso\/9780190915322.003.0004"},{"issue":"3","key":"9532_CR17","doi-asserted-by":"publisher","first-page":"343","DOI":"10.1111\/1758-5899.12665","volume":"10","author":"M Ekelhof","year":"2019","unstructured":"Ekelhof, M. (2019). Moving beyond semantics on autonomous weapons: Meaningful human control in operation. Global Policy, 10(3), 343\u2013348.","journal-title":"Global Policy"},{"key":"9532_CR18","doi-asserted-by":"crossref","unstructured":"Fischer, J. M., & Ravizza, M. (1998). Responsibility and control: A theory of moral responsibility.","DOI":"10.1017\/CBO9780511814594"},{"key":"9532_CR19","first-page":"289","volume":"12","author":"S Fitzsimmons","year":"2010","unstructured":"Fitzsimmons, S., & Sangha, K. (2010). Killing in high definition. Technology, 12, 289\u2013292.","journal-title":"Technology"},{"key":"9532_CR20","volume-title":"Military robots: Mapping the moral landscape","author":"J Galliott","year":"2015","unstructured":"Galliott, J. (2015). Military robots: Mapping the moral landscape. Farnham: Ashgate Publishing Ltd."},{"key":"9532_CR21","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1093\/acprof:oso\/9780199239351.003.0009","volume-title":"Offences and defences","author":"J Gardner","year":"2007","unstructured":"Gardner, J. (2007). The mark of responsibility. Offences and defences (pp. 177\u2013200). Oxford: Oxford University Press."},{"key":"9532_CR22","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511625053","volume-title":"Utilitarianism as a public philosophy","author":"RE Goodin","year":"1995","unstructured":"Goodin, R. E. (1995). Utilitarianism as a public philosophy. Cambridge: Cambridge University Press."},{"key":"9532_CR23","first-page":"27","volume":"22","author":"SL Greer","year":"2016","unstructured":"Greer, S. L., Wismar, M., Figueras, J., & McKee, C. (2016). Governance: a framework. Strengthening Health System Governance, 22, 27\u201356.","journal-title":"Strengthening Health System Governance"},{"key":"9532_CR24","volume-title":"Meaningful human control in weapon systems: A primer","author":"M Horowitz","year":"2015","unstructured":"Horowitz, M., & Scharre, P. (2015). Meaningful human control in weapon systems: A primer. Washington: Center for a New American Security."},{"key":"9532_CR25","series-title":"Handbook of ethics, values, and technological design: sources, theory, values and application domains","first-page":"1","volume-title":"Design for the values of accountability and transparency","author":"J Hulstijn","year":"2014","unstructured":"Hulstijn, J., & Burgemeestre, B. (2014). Design for the values of accountability and transparency (pp. 1\u201325)., Handbook of ethics, values, and technological design: sources, theory, values and application domains Berlin: Springer."},{"key":"9532_CR26","unstructured":"ICRC. (2018). Ethics and autonomous weapon systems: An ethical basis for human control? Retrieved 31 January 2020 from Geneva. https:\/\/www.icrc.org\/en\/download\/file\/69961\/icrc_ethics_and_autonomous_weapon_systems_report_3_april_2018.pdf"},{"key":"9532_CR27","volume-title":"Global governance and democratic accountability","author":"RO Keohane","year":"2003","unstructured":"Keohane, R. O. (2003). Global governance and democratic accountability. Citeseer: Princeton."},{"issue":"2","key":"9532_CR28","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1016\/0005-1098(96)85546-4","volume":"32","author":"N Kheir","year":"1996","unstructured":"Kheir, N., \u00c5str\u00f6m, K. J., Auslander, D., Cheok, K. C., Franklin, G. F., Masten, M., et al. (1996). Control systems engineering education. Automatica, 32(2), 147\u2013166.","journal-title":"Automatica"},{"issue":"1","key":"9532_CR29","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1111\/j.1540-6210.2005.00434.x","volume":"65","author":"JG Koppell","year":"2005","unstructured":"Koppell, J. G. (2005). Pathologies of accountability: ICANN and the challenge of \u201cmultiple accountabilities disorder\u201d. Public Administration Review, 65(1), 94\u2013108.","journal-title":"Public Administration Review"},{"issue":"3","key":"9532_CR30","doi-asserted-by":"publisher","first-page":"645","DOI":"10.1016\/j.eswa.2007.07.012","volume":"35","author":"S-H Liao","year":"2008","unstructured":"Liao, S.-H. (2008). Problem structuring methods in military command and control. Expert Systems with Applications, 35(3), 645\u2013653.","journal-title":"Expert Systems with Applications"},{"issue":"3","key":"9532_CR31","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1007\/s10676-004-3422-1","volume":"6","author":"A Matthias","year":"2004","unstructured":"Matthias, A. (2004). The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics and Information Technology, 6(3), 175\u2013183.","journal-title":"Ethics and Information Technology"},{"key":"9532_CR32","doi-asserted-by":"crossref","unstructured":"Mecacci, G., Santoni De Sio, F. (2019). Meaningful human control as reason-responsiveness: the case of dual-mode vehicles. Ethics and Information Technology.","DOI":"10.1007\/s10676-019-09519-w"},{"key":"9532_CR33","first-page":"47","volume-title":"State and individual responsibility for targeted killings by drones","author":"C Meloni","year":"2016","unstructured":"Meloni, C. (2016). State and individual responsibility for targeted killings by drones (p. 47). Drones and Responsibility: Legal, philosophical and socio-technical perspectives on remotely controlled weapons."},{"issue":"3","key":"9532_CR34","doi-asserted-by":"publisher","first-page":"555","DOI":"10.1111\/1467-9299.00218","volume":"78","author":"R Mulgan","year":"2000","unstructured":"Mulgan, R. (2000). \u2018Accountability\u2019: An ever-expanding concept? Public Administration, 78(3), 555\u2013573.","journal-title":"Public Administration"},{"key":"9532_CR35","unstructured":"NATO. (2016). Allied joint doctrine for joint targeting Edition A Version 1 Retrieved from https:\/\/assets.publishing.service.gov.uk\/government\/uploads\/system\/uploads\/attachment_data\/file\/628215\/20160505-nato_targeting_ajp_3_9.pdf"},{"key":"9532_CR36","unstructured":"NATO. (2017). AJP-01 ALLIED JOINT DOCTRINE. Retrieved from https:\/\/www.gov.uk\/government\/publications\/ajp-01-d-allied-joint-doctrine"},{"key":"9532_CR37","unstructured":"Pelizzo, R., Stapenhurst, R., Olson, D., Parliamentary Oversight for Government Accountability. (2006). Research collection school of social sciences. Paper 137. https:\/\/ink.library.smu.edu.sg\/soss_research\/137"},{"issue":"4","key":"9532_CR38","doi-asserted-by":"publisher","first-page":"925","DOI":"10.1007\/s11948-014-9571-7","volume":"21","author":"U Pesch","year":"2015","unstructured":"Pesch, U. (2015). Engineers and active responsibility. Science and Engineering Ethics, 21(4), 925\u2013939.","journal-title":"Science and Engineering Ethics"},{"key":"9532_CR39","volume-title":"Re-conceptualizing command and control","author":"R Pigeau","year":"2002","unstructured":"Pigeau, R., & McCann, C. (2002). Re-conceptualizing command and control. Toronto: Defence and Civil Institute of Environmental Medicine."},{"issue":"2","key":"9532_CR40","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1093\/oxfordjournals.pubjof.a029855","volume":"26","author":"BA Radin","year":"1996","unstructured":"Radin, B. A., & Romzek, B. S. (1996). Accountability expectations in an intergovernmental arena: The national rural development partnership. The Journal of Federalism, 26(2), 59\u201381.","journal-title":"The Journal of Federalism"},{"key":"9532_CR41","series-title":"Routledge handbook of ethics and war: just war theory in the 21st century","first-page":"352","volume-title":"Responsibility, liability, and lethal autonomous robots","author":"HM Roff","year":"2013","unstructured":"Roff, H. M. (2013). Responsibility, liability, and lethal autonomous robots (pp. 352\u2013364)., Routledge handbook of ethics and war: just war theory in the 21st century Abingdon: Routledge."},{"key":"9532_CR42","unstructured":"Roff, H. M., Moyes, R. (2016). Meaningful human control, artificial intelligence and autonomous weapons. Paper presented at the Briefing Paper Prepared for the Informal Meeting of Experts on Lethal Au-Tonomous Weapons Systems, UN Convention on Certain Conventional Weapons."},{"key":"9532_CR43","doi-asserted-by":"publisher","first-page":"227","DOI":"10.2307\/975901","volume":"1","author":"BS Romzek","year":"1987","unstructured":"Romzek, B. S., & Dubnick, M. J. (1987). Accountability in the public sector: Lessons from the challenger tragedy. Public administration review, 1, 227\u2013238.","journal-title":"Public administration review"},{"key":"9532_CR44","doi-asserted-by":"publisher","first-page":"15","DOI":"10.3389\/frobt.2018.00015","volume":"5","author":"F Santoni de Sio","year":"2018","unstructured":"Santoni de Sio, F., & Van den Hoven, J. (2018). Meaningful human control over autonomous systems: A philosophical account. Frontiers in Robotics and AI, 5, 15.","journal-title":"Frontiers in Robotics and AI"},{"key":"9532_CR45","series-title":"The self-restraining state","first-page":"14","volume-title":"Conceptualizing accountability","author":"A Schedler","year":"1999","unstructured":"Schedler, A. (1999). Conceptualizing accountability (p. 14)., The self-restraining state New York: Power and accountability in new democracies."},{"issue":"1","key":"9532_CR46","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1111\/1467-6478.00146","volume":"27","author":"C Scott","year":"2000","unstructured":"Scott, C. (2000). Accountability in the regulatory state. Journal of Law and Society, 27(1), 38\u201360.","journal-title":"Journal of Law and Society"},{"issue":"1","key":"9532_CR47","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1017\/S0892679415000647","volume":"30","author":"R Sparrow","year":"2016","unstructured":"Sparrow, R. (2016). Robots and respect: Assessing the case against autonomous weapon systems. Ethics & International Affairs, 30(1), 93\u2013116.","journal-title":"Ethics & International Affairs"},{"key":"9532_CR48","unstructured":"UN GGE LAWS. (2018). Emerging Commonalities, Conclusions and Recommendations. https:\/\/www.unog.ch\/80256EDD006B8954\/(httpAssets)\/EB4EC9367D3B63B1C12582FD0057A9A4\/$file\/GGE+LAWS+August_EC,+C+and+Rs_final.pdf"},{"key":"9532_CR49","series-title":"Moral responsibility","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1163\/9789401200714","volume-title":"The relation between forward-looking and backward-looking responsibility","author":"I Van de Poel","year":"2011","unstructured":"Van de Poel, I. (2011). The relation between forward-looking and backward-looking responsibility (pp. 37\u201352)., Moral responsibility Berlin: Springer."},{"key":"9532_CR50","first-page":"2015","volume":"2","author":"J Van den Berg","year":"2015","unstructured":"Van den Berg, J. (2015). Wat maakt cyber security anders dan informatiebeveiliging? Magazine Nationale Veiligheid en Crisisbeheersing, 2, 2015.","journal-title":"Magazine Nationale Veiligheid en Crisisbeheersing"},{"key":"9532_CR51","first-page":"1371","volume":"47","author":"M Wagner","year":"2014","unstructured":"Wagner, M. (2014). The dehumanization of international humanitarian law: Legal, ethical, and political implications of autonomous weapon systems. Vanderbilt Journal of Transnational Law, 47, 1371.","journal-title":"Vanderbilt Journal of Transnational Law"},{"issue":"4","key":"9532_CR52","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1007\/s10677-008-9109-7","volume":"11","author":"G Williams","year":"2008","unstructured":"Williams, G. (2008). Responsibility as a virtue. Ethical Theory and Moral Practice, 11(4), 455\u2013470.","journal-title":"Ethical Theory and Moral Practice"}],"container-title":["Minds and Machines"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11023-020-09532-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11023-020-09532-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11023-020-09532-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,31]],"date-time":"2021-07-31T23:42:21Z","timestamp":1627774941000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11023-020-09532-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,1]]},"references-count":52,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,3]]}},"alternative-id":["9532"],"URL":"https:\/\/doi.org\/10.1007\/s11023-020-09532-9","relation":{},"ISSN":["0924-6495","1572-8641"],"issn-type":[{"value":"0924-6495","type":"print"},{"value":"1572-8641","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8,1]]},"assertion":[{"value":"17 May 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 July 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 August 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}