{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T16:47:24Z","timestamp":1781369244279,"version":"3.54.1"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2021,7,20]],"date-time":"2021-07-20T00:00:00Z","timestamp":1626739200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[2021,7,20]],"date-time":"2021-07-20T00:00:00Z","timestamp":1626739200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/100000161","name":"United States Department of Commerce | National Institute of Standards and Technology","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000161","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Icelandic Fulbright Commission"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Nat Mach Intell"],"DOI":"10.1038\/s42256-021-00370-7","type":"journal-article","created":{"date-parts":[[2021,7,20]],"date-time":"2021-07-20T16:03:34Z","timestamp":1626797014000},"page":"566-571","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":180,"title":["Governing AI safety through independent audits"],"prefix":"10.1038","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6463-7719","authenticated-orcid":false,"given":"Gregory","family":"Falco","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ben","family":"Shneiderman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julia","family":"Badger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ryan","family":"Carrier","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anton","family":"Dahbura","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Danks","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Eling","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alwyn","family":"Goodloe","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jerry","family":"Gupta","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"Hart","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6088-3955","authenticated-orcid":false,"given":"Marina","family":"Jirotka","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Henric","family":"Johnson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cara","family":"LaPointe","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ashley J.","family":"Llorens","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0380-105X","authenticated-orcid":false,"given":"Alan K.","family":"Mackworth","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carsten","family":"Maple","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sigur\u00f0ur Emil","family":"P\u00e1lsson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Frank","family":"Pasquale","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1476-3127","authenticated-orcid":false,"given":"Alan","family":"Winfield","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zee Kin","family":"Yeong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,7,20]]},"reference":[{"key":"370_CR1","unstructured":"Oliver, N., Calvard, T. & Poto\u010dnik, K. The tragic crash of flight AF447 shows the unlikely but catastrophic consequences of automation. Harvard Business Review (15 September 2017)."},{"key":"370_CR2","unstructured":"Sumwalt, R., Landsberg, B. & Homendy, J. Assumptions Used in the Safety Assessment Process and the Effects of Multiple Alerts and Indications on Pilot Performance (National Transportation Safety Board, 2019)."},{"key":"370_CR3","unstructured":"Sumwalt, R. NHTSA-2020-0106-0617 Comment from Robert L. Sumwalt, III (National Transportation Safety Board, 2021)."},{"key":"370_CR4","doi-asserted-by":"publisher","first-page":"370","DOI":"10.1037\/h0054346","volume":"50","author":"AH Maslow","year":"1943","unstructured":"Maslow, A. H. A theory of human motivation. Psychol. Rev. 50, 370\u2013396 (1943).","journal-title":"Psychol. Rev."},{"key":"370_CR5","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1080\/10447318.2020.1741118","volume":"36","author":"B Shneiderman","year":"2020","unstructured":"Shneiderman, B. Human-centered artificial intelligence: reliable, safe & trustworthy. Int. J. Human Comput. Interact. 36, 495\u2013504 (2020).","journal-title":"Int. J. Human Comput. Interact."},{"key":"370_CR6","doi-asserted-by":"publisher","first-page":"1066","DOI":"10.1126\/science.aaz4795","volume":"366","author":"G Falco","year":"2019","unstructured":"Falco, G. et al. Cyber risk research impeded by disciplinary barriers. Science 366, 1066\u20131069 (2019).","journal-title":"Science"},{"key":"370_CR7","doi-asserted-by":"crossref","unstructured":"Fjeld, J., Achten, N., Hilligoss, H., Nagy, A. & Srikumar, M. Principled Artificial Intelligence: Mapping Consensus in Ethical and Rights-Based Approaches to Principles for AI (Berkman Klein Center, 2020).","DOI":"10.2139\/ssrn.3518482"},{"key":"370_CR8","unstructured":"Brundage, M. et al. Toward trustworthy AI development: mechanisms for supporting verifiable claims. Preprint at https:\/\/arxiv.org\/abs\/2004.07213 (2020)."},{"key":"370_CR9","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1038\/s42256-019-0088-2","volume":"1","author":"A Jobin","year":"2019","unstructured":"Jobin, A., Ienca, M. & Vayena, E. The global landscape of AI ethics guidelines. Nat. Mach. Intell. 1, 389\u2013399 (2019).","journal-title":"Nat. Mach. Intell."},{"key":"370_CR10","doi-asserted-by":"crossref","unstructured":"Winfield, A. & Jirotka, M. Ethical governance is essential to building trust in robotics and artificial intelligence systems. Phil. Trans. R. Soc. A 376, 20180085 (2018)..","DOI":"10.1098\/rsta.2018.0085"},{"key":"370_CR11","doi-asserted-by":"publisher","first-page":"501","DOI":"10.1038\/s42256-019-0114-4","volume":"1","author":"B Mittelstadt","year":"2019","unstructured":"Mittelstadt, B. Principles alone cannot guarantee ethical AI. Nat. Mach. Intell. 1, 501\u2013507 (2019).","journal-title":"Nat. Mach. Intell."},{"key":"370_CR12","unstructured":"IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems Ethically Aligned Design: A Vision for Prioritizing Human Well-Being with Autonomous and Intelligent Systems 1st edn (IEEE, 2019)."},{"key":"370_CR13","doi-asserted-by":"publisher","first-page":"286","DOI":"10.1057\/jdg.2010.22","volume":"7","author":"RJ Kueppers","year":"2010","unstructured":"Kueppers, R. J. & Sullivan, K. B. How and why an independent audit matters. Int. J. Discl. Gov. 7, 286\u2013293 (2010).","journal-title":"Int. J. Discl. Gov."},{"key":"370_CR14","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1145\/3170706","volume":"25","author":"G Fischer","year":"2017","unstructured":"Fischer, G. Exploring design trade-offs for quality of life in human-centered design. Interactions 25, 26\u201333 (2017).","journal-title":"Interactions"},{"key":"370_CR15","doi-asserted-by":"publisher","first-page":"986","DOI":"10.1109\/JPROC.2003.814621","volume":"91","author":"CJ Tomlin","year":"2003","unstructured":"Tomlin, C. J., Mitchell, I., Bayen, A. M. & Oishi, M. Computational techniques for the verification of hybrid systems. Proc. IEEE 91, 986\u20131001 (2003).","journal-title":"Proc. IEEE"},{"key":"370_CR16","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1023\/A:1025697810124","volume":"8","author":"AK Mackworth","year":"2003","unstructured":"Mackworth, A. K. & Zhang, Y. A formal approach to agent design: an overview of constraint-based agents. Constraints 8, 229\u2013242 (2003).","journal-title":"Constraints"},{"key":"370_CR17","unstructured":"Topcu, U. et al. Assured autonomy: path toward living with autonomous systems we can trust. Preprint at https:\/\/arxiv.org\/abs\/2010.14443 (2020)."},{"key":"370_CR18","doi-asserted-by":"crossref","unstructured":"Falco, G. & Gilpin, L. A stress testing framework for autonomous system verification and validation (v&v). In IEEE International Conference on Autonomous Systems (IEEE, 2021).","DOI":"10.1109\/ICAS49788.2021.9551154"},{"key":"370_CR19","unstructured":"ISO 13849-1:2015 Safety of Machinery \u2014 Safety-Related Parts of Control Systems (International Organization for Standardization Technical Committee, 2015)."},{"key":"370_CR20","unstructured":"IEC 31010:2019 Risk Management\u2014Risk Assessment Techniques (ISO, 2019); https:\/\/www.iso.org\/standard\/72140.html"},{"key":"370_CR21","unstructured":"BS8611:2016 Robots and Robotic Devices. Guide to the Ethical Design and Application of Robots and Robotic Systems (British Standards Institute, 2016)."},{"key":"370_CR22","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1038\/s41928-019-0213-6","volume":"2","author":"A Winfield","year":"2019","unstructured":"Winfield, A. Ethical standards in robotics and AI. Nat. Electron. 2, 46\u201348 (2019).","journal-title":"Nat. Electron."},{"key":"370_CR23","doi-asserted-by":"crossref","unstructured":"Landwehr, C. E. A building code for building code: putting what we know works to work. In Proc. 29th Annual Computer Security Applications Conference 139\u2013147 (ACM, 2013).","DOI":"10.1145\/2523649.2530278"},{"key":"370_CR24","unstructured":"Haigh, T. & Landwehr, C. Building Code for Medical Device Software Security (IEEE Cybersecurity, 2015)."},{"key":"370_CR25","unstructured":"Gebru, T. et al. Datasheets for datasets. Preprint at https:\/\/arxiv.org\/abs\/1803.09010 (2018)."},{"key":"370_CR26","doi-asserted-by":"crossref","unstructured":"Mitchell, M. et al. Model cards for model reporting. In Proc. Conference on Fairness, Accountability, and Transparency 220\u2013229 (ACM, 2019).","DOI":"10.1145\/3287560.3287596"},{"key":"370_CR27","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1147\/JRD.2019.2942288","volume":"63","author":"M Arnold","year":"2019","unstructured":"Arnold, M. et al. FactSheets: Increasing trust in AI services through supplier\u2019s declarations of conformity. IBM J. Res. Dev. 63, 6 (2019).","journal-title":"IBM J. Res. Dev."},{"key":"370_CR28","unstructured":"Personal Data Protection Commission of Singapore Model Artificial Intelligence Governance Framework 2nd edn (Infocomm Media Development Authority of Singapore, 2020)."},{"key":"370_CR29","doi-asserted-by":"crossref","unstructured":"Cummings, M. & Britton, D. in Living with Robots 119\u2013140 (Elsevier, 2020).","DOI":"10.1016\/B978-0-12-815367-3.00006-2"},{"key":"370_CR30","unstructured":"Grossi, D. R. Aviation recorder overview. In International Symposium On Transportation Recorders 153\u2013164 (ISTR, 1999)."},{"key":"370_CR31","doi-asserted-by":"crossref","unstructured":"Winfield, A. & Jirotka, M. The case for an ethical black box. In Annual Conference Towards Autonomous Robotic Systems 262\u2013273 (Springer, 2017).","DOI":"10.1007\/978-3-319-64107-2_21"},{"key":"370_CR32","doi-asserted-by":"publisher","first-page":"1484","DOI":"10.1109\/TITS.2020.2971385","volume":"22","author":"Y Yao","year":"2020","unstructured":"Yao, Y. & Atkins, E. The smart black box: a value-driven high-bandwidth automotive event data recorder. IEEE Trans. Intell. Transport. Syst. 22, 1484\u20131496 (2020).","journal-title":"IEEE Trans. Intell. Transport. Syst."},{"key":"370_CR33","doi-asserted-by":"crossref","unstructured":"Falco, G. & Siegel, J. E. A distributed \u2018black box\u2019 audit trail design specification for connected and automated vehicle data and software assurance. SAE Int. J. Transport. Cybersecur. Privacy 3, 11-03-02-0006 (2020).","DOI":"10.4271\/11-03-02-0006"},{"key":"370_CR34","doi-asserted-by":"crossref","unstructured":"Winfield, A. et al. in Software Engineering for Robotics (eds Cavalcanti, A. et al.) 165\u2013187 (Springer, 2021).","DOI":"10.1007\/978-3-030-66494-7_6"},{"key":"370_CR35","first-page":"46","volume":"58","author":"M Williamsen","year":"2013","unstructured":"Williamsen, M. Near-miss reporting: a missing link in safety culture. Prof. Safety 58, 46\u201350 (2013).","journal-title":"Prof. Safety"},{"key":"370_CR36","unstructured":"Principles to Promote Fairness, Ethics, Accountability and Transparency (Feat) in the Use of Artificial Intelligence and Data Analytics in Singapore\u2019s Financial Sector (Monetary Authority of Singapore & Fairness, Ethics, Accountability and Transparency Committee, 2018)."},{"key":"370_CR37","unstructured":"World Economic Forum Companion to the Model AI Governance Framework\u2014Implementation and Self-Assessment for Organizations (Infocomm Media Development Authority of Singapore, 2020)."},{"key":"370_CR38","unstructured":"Preliminary Study on the Ethics of Artificial Intelligence (World Commission on the Ethics of Scientific Knowledge and Technology, 2019)."},{"key":"370_CR39","doi-asserted-by":"crossref","unstructured":"Falco, G. A smart city internet for autonomous systems. In 2020 Symposium on Security and Privacy Workshops 215\u2013220 (IEEE, 2020).","DOI":"10.1109\/SPW50608.2020.00051"},{"key":"370_CR40","unstructured":"Drogkaris, P. & Bourka, A. Towards a Framework for Policy Development in Cybersecurity: Security and Privacy Considerations in Autonomous Agents (European Union Agency for Cybersecurity, 2018)."},{"key":"370_CR41","unstructured":"High-Level Expert Group on AI Ethics Guidelines for Trustworthy AI (European Commission, 2019)."},{"key":"370_CR42","unstructured":"Regulation of the European Parliament and of the Council Laying Down Harmonized Rules of Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts (European Commission, 2021)."},{"key":"370_CR43","unstructured":"Personal Data Protection Commission of Singapore Model Artificial Intelligence Governance Framework 1st edn (Infocomm Media Development Authority of Singapore, 2019)."},{"key":"370_CR44","unstructured":"Schmidt, E. et al. National Security Commission on Artificial Intelligence: Final Report (National Security Commission on Artificial Intelligence, 2021)."},{"key":"370_CR45","doi-asserted-by":"crossref","unstructured":"Pasquale, F. The Black Box Society (Harvard Univ. Press, 2015).","DOI":"10.4159\/harvard.9780674736061"},{"key":"370_CR46","unstructured":"The American Recovery and Reinvestment Act of 2009 (US Government, 2009)."},{"key":"370_CR47","doi-asserted-by":"crossref","unstructured":"Spiekermann, S. & Winkler, T. Value-based engineering for ethics by design. Preprint at https:\/\/arxiv.org\/abs\/2004.13676l (2020).","DOI":"10.2139\/ssrn.3598911"}],"container-title":["Nature Machine Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s42256-021-00370-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s42256-021-00370-7","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s42256-021-00370-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,3]],"date-time":"2022-12-03T20:34:14Z","timestamp":1670099654000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s42256-021-00370-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,20]]},"references-count":47,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2021,7]]}},"alternative-id":["370"],"URL":"https:\/\/doi.org\/10.1038\/s42256-021-00370-7","relation":{},"ISSN":["2522-5839"],"issn-type":[{"value":"2522-5839","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,20]]},"assertion":[{"value":"15 December 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 June 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 July 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"G.F. is a consultant for the World Bank Group on autonomous vehicle regulation and is a ForHumanity fellow; he thanks the US National Institute for Standards and Technology (NIST), the Icelandic Fulbright Commission and the National Science Foundation for research funding. B.S. is a ForHumanity fellow. J.B. is a US government civil servant working for and funded by NASA. R.C. is the Executive Director of ForHumanity, a registered 501(c)(3) not-for-profit organization. A.D. is a member of the Maryland Cybersecurity Council, established by the Maryland State Legislature to work with the National Institute of Standards and Technology and other federal agencies, private sector businesses and private cybersecurity experts to improve cybersecurity in Maryland. D.D. is an external member of the Salesforce Advisory Council on Ethical & Humane Use of Technology. A.G. is a US government civil servant working for and funded by NASA; he is a voting member of SAE 34 working group on AI in Aviation that is writing guidelines for AI in aviation. C.H. is the Chairman of the Washington Metrorail Safety Commission where the opinions expressed in this article are his and not those of the Commission; he is bound by confidentiality agreements that prevent him from disclosing other competing interests in this work. M.J. holds an EPSRC Fellowship investigating ethical data recorders in robots, leads a project on legality and ethics of data recorders in autonomous vehicles and is a member of the All-Party Parliamentary Group on Data Analytics; she is a Director of ORBIT-RRI Ltd. H.J. is the Swedish Science and Innovation Counsellor to the United States. A.J.L. is Vice President and Global Outreach Director at Microsoft Research and currently serves as the Science Representative on the steering committee of the Global Partnership on Artificial Intelligence. A.K.M. is a Director of Minerva Intelligence Inc; he is a member of the Centre for AI Decision-making and Action (CAIDA) Steering Committee, the AI network of British Columbia (AInBC) Board and The Confederation of Laboratories for Artificial Intelligence Research in Europe (CLAIRE) international advisory board. C.M. is a Fellow of the Alan Turing Institute, a member of the Strategic Advisory Board of Zenzic and of the ENISA CarSec Expert Group. S.E.P. is the Chief National Cyber Security Adviser of the Icelandic Government, chairs the Icelandic Cyber Security Council and is a member of the Board of the European Union Agency for Cybersecurity (ENISA); he leads the development and implementation of Iceland\u2019s cybersecurity strategy including the cybersecurity aspects of AI. A.W. sits on British Standards Institute committee AMT\/010\/01 Ethics for Robots and Autonomous Systems, the executive committee of the IEEE Standards Association Global Initiative on Ethics of Autonomous and Intelligent Systems and the WEF Global AI Council; he is a member of the Advisory Committee of robotics company Karakuri Ltd. Z.K.Y. leads the development and implementation of Singapore\u2019s AI Governance Framework and is a member of OECD\u2019s Network of Experts in AI and the Global Partnership on Artificial Intelligence\u2019s expert working group on Data Governance. Authors not mentioned declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}