{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T00:02:22Z","timestamp":1760659342155,"version":"3.37.3"},"reference-count":46,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2021,5,29]],"date-time":"2021-05-29T00:00:00Z","timestamp":1622246400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,5,29]],"date-time":"2021-05-29T00:00:00Z","timestamp":1622246400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003246","name":"Nederlandse Organisatie voor Wetenschappelijk Onderzoek","doi-asserted-by":"publisher","award":["400.17.605"],"award-info":[{"award-number":["400.17.605"]}],"id":[{"id":"10.13039\/501100003246","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["AI Ethics"],"published-print":{"date-parts":[[2021,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The collection and use of personal data on citizens in the design and deployment of algorithms in the domain of justice and security is a sensitive topic. Values like fairness, autonomy, privacy, accuracy, transparency and property are at stake. Negative examples of algorithms that propagate or exacerbate biases, inequalities or injustices have received ample attention, both in academia and in popular media. To supplement this view, we will discuss two positive examples of Responsible Innovation (RI): the design and deployment of algorithms in decision support, with good intentions and careful approaches. We then explore potential, unintended, undesirable, higher-order effects of algorithms\u2014effects that may occur despite good intentions and careful approaches. We do that by engaging with<jats:italic>anticipation<\/jats:italic>and<jats:italic>responsiveness<\/jats:italic>, two key dimensions of Responsible Innovation. We close the paper with proposing a framework and a series of tentative recommendations to promote anticipation and responsiveness in the design and deployment of algorithms in decision support in the domain of justice and security.<\/jats:p>","DOI":"10.1007\/s43681-021-00063-2","type":"journal-article","created":{"date-parts":[[2021,5,29]],"date-time":"2021-05-29T06:03:38Z","timestamp":1622268218000},"page":"501-515","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Responsible innovation, anticipation and responsiveness: case studies of algorithms in decision support in justice and security, and an exploration of potential, unintended, undesirable, higher-order effects"],"prefix":"10.1007","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7915-5459","authenticated-orcid":false,"given":"Marc","family":"Steen","sequence":"first","affiliation":[]},{"given":"Tjerk","family":"Timan","sequence":"additional","affiliation":[]},{"given":"Ibo","family":"van de Poel","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,5,29]]},"reference":[{"key":"63_CR1","volume-title":"On organizational learning","author":"C Argyris","year":"1992","unstructured":"Argyris, C.: On organizational learning. Blackwell Publishing, Cambridge, Massachusetts (1992)"},{"issue":"2","key":"63_CR2","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1007\/s10676-016-9389-x","volume":"18","author":"T Arnold","year":"2016","unstructured":"Arnold, T., Scheutz, M.: Against the moral turing test: accountable design and the moral reasoning of autonomous systems. Ethics Inf. Technol. 18(2), 103\u2013115 (2016). https:\/\/doi.org\/10.1007\/s10676-016-9389-x","journal-title":"Ethics Inf. Technol."},{"key":"63_CR3","volume-title":"The Oxford Handbook of Ethics of AI","author":"C Barabas","year":"2020","unstructured":"Barabas, C.: Beyond bias: \u201cEthical AI\u201d in criminal law. In: Dubber, M.D., Pasquale, F., Das, S. (eds.) The Oxford Handbook of Ethics of AI. Oxford University Press, Oxford, UK (2020)"},{"key":"63_CR4","doi-asserted-by":"crossref","unstructured":"Barabas, C., Doyle, C., Rubinovitz, J.B., Dinakar K.: \"Studying Up: Reorienting the study of algorithmic fairness around issues of power.\" ACM Conference on Fairness, Accountability, and Transparency, January 27\u201330, 2020, Barcelona, Spain (2020)","DOI":"10.1145\/3351095.3372859"},{"key":"63_CR5","first-page":"149","volume":"81","author":"R Binns","year":"2018","unstructured":"Binns, R.: Fairness in machine learning: lessons from political philosophy. Proc. Mach. Learn. Res. 81, 149\u2013159 (2018)","journal-title":"Proc. Mach. Learn. Res."},{"issue":"1","key":"63_CR6","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1007\/s10676-018-9444-x","volume":"20","author":"V Bonnemains","year":"2018","unstructured":"Bonnemains, V., Saurel, C., Tessier, C.: Embedded ethics: some technical and ethical challenges. Ethics Inf. Technol. 20(1), 41\u201358 (2018). https:\/\/doi.org\/10.1007\/s10676-018-9444-x","journal-title":"Ethics Inf. Technol."},{"issue":"1","key":"63_CR7","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1007\/s41125-017-0012-x","volume":"2","author":"S Brinkhoff","year":"2017","unstructured":"Brinkhoff, S.: Big data data mining by the Dutch police: criteria for a future method of investigation. Eur. J. Secur. Res. 2(1), 57\u201369 (2017). https:\/\/doi.org\/10.1007\/s41125-017-0012-x","journal-title":"Eur. J. Secur. Res."},{"issue":"4","key":"63_CR8","doi-asserted-by":"publisher","first-page":"2051","DOI":"10.1007\/s11948-019-00146-8","volume":"26","author":"M Coeckelbergh","year":"2020","unstructured":"Coeckelbergh, M.: Artificial intelligence, responsibility attribution, and a relational justification of explainability. Sci. Eng. Ethics 26(4), 2051\u20132068 (2020). https:\/\/doi.org\/10.1007\/s11948-019-00146-8","journal-title":"Sci. Eng. Ethics"},{"issue":"3","key":"63_CR9","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1007\/s13347-015-0211-1","volume":"29","author":"J Danaher","year":"2016","unstructured":"Danaher, J.: The threat of algocracy: reality, resistance and accommodation. Philos. Technol. 29(3), 245\u2013268 (2016). https:\/\/doi.org\/10.1007\/s13347-015-0211-1","journal-title":"Philos. Technol."},{"key":"63_CR10","doi-asserted-by":"publisher","DOI":"10.1007\/s10676-020-09537-z","author":"M de Reuver","year":"2020","unstructured":"de Reuver, M., van Wynsberghe, A., Janssen, M., van de Poel, I.: Digital platforms and responsible innovation: expanding value sensitive design to overcome ontological uncertainty. Ethics Inf. Technol. (2020). https:\/\/doi.org\/10.1007\/s10676-020-09537-z","journal-title":"Ethics Inf. Technol."},{"key":"63_CR11","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-30371-6","volume-title":"Responsible artificial intelligence: how to develop and use AI in a responsible way","author":"V Dignum","year":"2019","unstructured":"Dignum, V.: Responsible artificial intelligence: how to develop and use AI in a responsible way. Springer Nature, Cham, Switzerland (2019)"},{"key":"63_CR12","volume-title":"Automating inequality","author":"V Eubanks","year":"2017","unstructured":"Eubanks, V.: Automating inequality. St. Martin\u2019s Press, New York (2017)"},{"issue":"5","key":"63_CR13","first-page":"1109","volume":"94","author":"AG Ferguson","year":"2017","unstructured":"Ferguson, A.G.: Policing predictive policing. Washington Univ. Law Rev. 94(5), 1109\u20131189 (2017)","journal-title":"Washington Univ. Law Rev."},{"key":"63_CR14","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780199641321.001.0001","volume-title":"The ethics of information","author":"L Floridi","year":"2013","unstructured":"Floridi, L.: The ethics of information. Oxford University Press, Oxford, UK (2013)"},{"issue":"2","key":"63_CR15","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1007\/s13347-019-00354-x","volume":"32","author":"L Floridi","year":"2019","unstructured":"Floridi, L.: Translating principles into practices of digital ethics: five risks of being unethical. Philos. Technol. 32(2), 185\u2013193 (2019). https:\/\/doi.org\/10.1007\/s13347-019-00354-x","journal-title":"Philos. Technol."},{"key":"63_CR16","doi-asserted-by":"publisher","first-page":"689","DOI":"10.1007\/s11023-018-9482-5","volume":"28","author":"L Floridi","year":"2018","unstructured":"Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., Schafer, B., Valcke, P., Vayena, E.: AI4People\u2014an ethical framework for a good ai society: opportunities, risks, principles, and recommendations. Mind. Mach. 28, 689\u2013707 (2018)","journal-title":"Mind. Mach."},{"key":"63_CR17","first-page":"348","volume-title":"Human-computer interaction and management information systems","author":"B Friedman","year":"2006","unstructured":"Friedman, B., Kahn, P., Borning, A.: Value sensitive design and information systems. In: Zhang, P., Galletta, D. (eds.) Human-computer interaction and management information systems, pp. 348\u2013372. M. E. Sharpe, Armonk, NY (2006)"},{"key":"63_CR18","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1007\/978-94-017-8956-1_2","volume-title":"Responsible innovation 1: innovative solutions for global issues","author":"A Grunwald","year":"2014","unstructured":"Grunwald, A.: Technology assessment for responsible innovation. In: Van den Hoven, J., Doorn, N., Swierstra, T., Koops, B.-J., Romijn, H. (eds.) Responsible innovation 1: innovative solutions for global issues, pp. 15\u201332. Springer Science+Business Media, Dordrecht, The Netherlands (2014)"},{"key":"63_CR19","doi-asserted-by":"publisher","DOI":"10.1007\/s10676-017-9428-2","author":"DJ Gunkel","year":"2017","unstructured":"Gunkel, D.J.: Mind the gap: responsible robotics and the problem of responsibility. Ethics Inf. Technol. (2017). https:\/\/doi.org\/10.1007\/s10676-017-9428-2","journal-title":"Ethics Inf. Technol."},{"key":"63_CR20","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1007\/s00146-019-00932-9","volume":"35","author":"P Hayes","year":"2020","unstructured":"Hayes, P., van de Poel, I., Steen, M.: Algorithms and values in justice and security. AI Soc. 35, 533\u2013555 (2020). https:\/\/doi.org\/10.1007\/s00146-019-00932-9","journal-title":"AI Soc."},{"key":"63_CR21","volume-title":"Ethics guidelines for trustworthy AI","author":"High-Level Expert Group on Artificial Intelligence","year":"2019","unstructured":"High-Level Expert Group on Artificial Intelligence: Ethics guidelines for trustworthy AI. European Commission, Brussels (2019)"},{"key":"63_CR22","volume-title":"Basisboek aanpak schulden","author":"N Jungmann","year":"2017","unstructured":"Jungmann, N., Madern, T.: Basisboek aanpak schulden, Eerste druk Noordhoff Uitgevers, Groningen (2017)","edition":"Eerste druk"},{"issue":"1","key":"63_CR23","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1080\/23299460.2014.992769","volume":"2","author":"AH Kiran","year":"2015","unstructured":"Kiran, A.H., Oudshoorn, N., Verbeek, P.-P.: Beyond checklists: toward an ethical-constructive technology assessment. J. Responsib. Innov. 2(1), 5\u201319 (2015)","journal-title":"J. Responsib. Innov."},{"key":"63_CR24","volume-title":"Juridische aspecten van algoritmen die besluiten nemen: Een verkennend onderzoek","author":"S Kulk","year":"2020","unstructured":"Kulk, S., Van Deursen, S., Boekema, M., Breemen, V., Heeger, S., Philipsen, S., Snijders, T., Wouters, A.: Juridische aspecten van algoritmen die besluiten nemen: Een verkennend onderzoek. Boom Juridisch, Den Haag (2020)"},{"issue":"1_suppl","key":"63_CR25","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1111\/j.1467-954X.1999.tb03480.x","volume":"47","author":"B Latour","year":"1999","unstructured":"Latour, B.: On recalling ANT. Sociol. Rev. 47(1_suppl), 15\u201325 (1999). https:\/\/doi.org\/10.1111\/j.1467-954X.1999.tb03480.x","journal-title":"Sociol. Rev."},{"key":"63_CR26","volume-title":"Rebooting AI: building artificial intelligence we can trust","author":"G Marcus","year":"2019","unstructured":"Marcus, G., Davis, E.: Rebooting AI: building artificial intelligence we can trust. Pantheon, Toronto, Canada (2019)"},{"key":"63_CR27","volume-title":"Thinking in systems: a primer","author":"DH Meadows","year":"2008","unstructured":"Meadows, D.H.: Thinking in systems: a primer. Chelsea Publishing, White River Junction, Vermont (2008)"},{"issue":"11","key":"63_CR28","doi-asserted-by":"publisher","first-page":"501","DOI":"10.1038\/s42256-019-0114-4","volume":"1","author":"B Mittelstadt","year":"2019","unstructured":"Mittelstadt, B.: Principles alone cannot guarantee ethical AI. Nat. Mach. Intell. 1(11), 501\u2013507 (2019). https:\/\/doi.org\/10.1038\/s42256-019-0114-4","journal-title":"Nat. Mach. Intell."},{"key":"63_CR29","doi-asserted-by":"publisher","DOI":"10.1177\/2053951716679679","author":"BD Mittelstadt","year":"2016","unstructured":"Mittelstadt, B.D., Allo, P., Taddeo, M., Wachter, S., Floridi, L.: The ethics of algorithms: mapping the debate. Big Data Soc. (2016). https:\/\/doi.org\/10.1177\/2053951716679679","journal-title":"Big Data Soc."},{"key":"63_CR30","doi-asserted-by":"publisher","DOI":"10.1007\/s11948-019-00165-5","author":"J Morley","year":"2019","unstructured":"Morley, J., Floridi, L., Kinsey, L., Elhalal, A.: From what to how: an initial review of publicly available AI ethics tools, methods and research to translate principles into practices. Sci. Eng. Ethics (2019). https:\/\/doi.org\/10.1007\/s11948-019-00165-5","journal-title":"Sci. Eng. Ethics"},{"key":"63_CR31","volume-title":"CHI '07 Extended Abstracts on Human Factors in Computing Systems","author":"LP Nathan","year":"2007","unstructured":"Nathan, L.P., Klasnja, P.V., Friedman, B.: Value scenarios: a technique for envisioning systemic effects of new technologies. In: CHI \u201907 Extended Abstracts on Human Factors in Computing Systems. ACM, San Jose, CA (2007)"},{"key":"63_CR32","volume-title":"Weapons of math destruction","author":"C O'Neil","year":"2016","unstructured":"O\u2019Neil, C.: Weapons of math destruction. Penguin, London (2016)"},{"key":"63_CR33","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1007\/978-94-007-7844-3_3","volume-title":"Early engagement and new technologies: opening up the laboratory","author":"A Rip","year":"2013","unstructured":"Rip, A., Robinson, D.K.R.: Constructive technology assessment and the methodology of insertion. In: Doorn, N., Schuurbiers, D., Van de Poel, I., Gorman, M.E. (eds.) Early engagement and new technologies: opening up the laboratory, pp. 37\u201353. Springer Science+Business Media, Dordrecht, The Netherlands (2013)"},{"key":"63_CR34","first-page":"49","volume-title":"Yearbook of nanotechnology in society","author":"A Rip","year":"2008","unstructured":"Rip, A., Te Kulve, H.: Constructive technology assessment and socio-technical scenarios. In: Fisher, E., Selin, C., Wetmore, J.M. (eds.) Yearbook of nanotechnology in society, pp. 49\u201370. Springer, Berlin, Germany (2008)"},{"key":"63_CR35","volume-title":"Human compatible: AI and the problem of control","author":"S Russell","year":"2019","unstructured":"Russell, S.: Human compatible: AI and the problem of control. Allen Lane, London, UK (2019)"},{"key":"63_CR36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/frobt.2018.00001","volume":"5","author":"FS de Sio","year":"2018","unstructured":"de Sio, F.S., Van den Hoven, J.: Meaningful human control over autonomous systems: a philosophical account. Front. Robot. AI 5, 1\u201315 (2018)","journal-title":"Front. Robot. AI"},{"key":"63_CR37","volume-title":"The fifth discipline: the art and practice of the learning organization","author":"P Senge","year":"1990","unstructured":"Senge, P.: The fifth discipline: the art and practice of the learning organization. Doubleday, New York (1990)"},{"issue":"6","key":"63_CR38","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1080\/10447318.2020.1741118","volume":"36","author":"B Shneiderman","year":"2020","unstructured":"Shneiderman, B.: Human-centered artificial intelligence: reliable, safe & trustworthy. Int. J. Human Comput. Interact. 36(6), 495\u2013504 (2020). https:\/\/doi.org\/10.1080\/10447318.2020.1741118","journal-title":"Int. J. Human Comput. Interact."},{"key":"63_CR39","doi-asserted-by":"publisher","DOI":"10.1080\/23299460.2020.1813864","author":"M Steen","year":"2020","unstructured":"Steen, M., Nauta, J.: Advantages and disadvantages of societal engagement: a case study in a research and technology organization. J. Responsib. Innov. (2020). https:\/\/doi.org\/10.1080\/23299460.2020.1813864","journal-title":"J. Responsib. Innov."},{"key":"63_CR40","doi-asserted-by":"publisher","first-page":"1568","DOI":"10.1016\/j.respol.2013.05.008","volume":"42","author":"J Stilgoe","year":"2013","unstructured":"Stilgoe, J., Owen, R., Macnaghten, P.: Developing a framework for responsible innovation. Res. Policy 42, 1568\u20131580 (2013)","journal-title":"Res. Policy"},{"issue":"3","key":"63_CR41","doi-asserted-by":"publisher","first-page":"667","DOI":"10.1007\/s11948-015-9724-3","volume":"22","author":"I van de Poel","year":"2016","unstructured":"van de Poel, I.: An ethical framework for evaluating experimental technology. Sci. Eng. Ethics 22(3), 667\u2013686 (2016). https:\/\/doi.org\/10.1007\/s11948-015-9724-3","journal-title":"Sci. Eng. Ethics"},{"key":"63_CR42","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1201\/9781315379593-4","volume-title":"Embedding new technologies into society: a regulatory, ethical and societal perspective","author":"I van de Poel","year":"2017","unstructured":"van de Poel, I.: Society as a laboratory to experiment with new technologies. In: Bowman, D.M., Stokes, E., Rip, A. (eds.) Embedding new technologies into society: a regulatory, ethical and societal perspective, pp. 61\u201387. Pan Stanford Publishing, Singapore (2017)"},{"key":"63_CR43","doi-asserted-by":"publisher","DOI":"10.1080\/23299460.2020.1791506","author":"I van de Poel","year":"2020","unstructured":"van de Poel, I., Asveld, L., Flipse, S., Klaassen, P., Kwee, Z., Maia, M., Mantovani, E., Nathan, C., Porcari, A., Yaghmaei, E.: Learning to do responsible innovation in industry: six lessons. J. Responsib. Innov. (2020). https:\/\/doi.org\/10.1080\/23299460.2020.1791506","journal-title":"J. Responsib. Innov."},{"key":"63_CR44","doi-asserted-by":"publisher","DOI":"10.1108\/TG-09-2019-0095","author":"AF van Veenstra","year":"2020","unstructured":"van Veenstra, A.F., Gromm\u00e9, F., Djafari, S.: The use of public sector data analytics in the Netherlands. Transform. Gov. (2020). https:\/\/doi.org\/10.1108\/TG-09-2019-0095. (ahead-of-print)","journal-title":"Transform. Gov."},{"issue":"3","key":"63_CR45","doi-asserted-by":"publisher","first-page":"719","DOI":"10.1007\/s11948-018-0030-8","volume":"25","author":"A Van Wynsberghe","year":"2019","unstructured":"Van Wynsberghe, A., Robbins, S.: Critiquing the reasons for making artificial moral agents. Sci. Eng. Ethics 25(3), 719\u2013735 (2019)","journal-title":"Sci. Eng. Ethics"},{"key":"63_CR46","volume-title":"Case study research","author":"R Yin","year":"1994","unstructured":"Yin, R.: Case study research, 2nd edn. Sage, Thousand Oaks (1994)","edition":"2"}],"container-title":["AI and Ethics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s43681-021-00063-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s43681-021-00063-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s43681-021-00063-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,31]],"date-time":"2024-08-31T19:59:59Z","timestamp":1725134399000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s43681-021-00063-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,29]]},"references-count":46,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,11]]}},"alternative-id":["63"],"URL":"https:\/\/doi.org\/10.1007\/s43681-021-00063-2","relation":{},"ISSN":["2730-5953","2730-5961"],"issn-type":[{"type":"print","value":"2730-5953"},{"type":"electronic","value":"2730-5961"}],"subject":[],"published":{"date-parts":[[2021,5,29]]},"assertion":[{"value":"5 January 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 May 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 May 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}