{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T16:53:30Z","timestamp":1776099210134,"version":"3.50.1"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2022,3,21]],"date-time":"2022-03-21T00:00:00Z","timestamp":1647820800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,21]],"date-time":"2022-03-21T00:00:00Z","timestamp":1647820800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Ethics Inf Technol"],"published-print":{"date-parts":[[2022,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Rapid developments in Artificial Intelligence are leading to an increasing human reliance on machine decision making. Even in collaborative efforts with Decision Support Systems (DSSs), where a human expert is expected to make the final decisions, it can be hard to keep the expert actively involved throughout the decision process. DSSs suggest their own solutions and thus invite passive decision making. To keep humans actively \u2018on\u2019 the decision-making loop and counter overreliance on machines, we propose a \u2018reflection machine\u2019 (RM). This system asks users questions about their decision strategy and thereby prompts them to evaluate their own decisions critically. We discuss what forms RMs can take and present a proof-of-concept implementation of a RM that can produce feedback on users\u2019 decisions in the medical and law domains. We show that the prototype requires very little domain knowledge to create reasonably intelligent critiquing questions. With this prototype, we demonstrate the technical feasibility to develop RMs and hope to pave the way for future research into their effectiveness and value.<\/jats:p>","DOI":"10.1007\/s10676-022-09645-y","type":"journal-article","created":{"date-parts":[[2022,4,7]],"date-time":"2022-04-07T18:03:51Z","timestamp":1649354631000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Reflection machines: increasing meaningful human control over Decision Support Systems"],"prefix":"10.1007","volume":"24","author":[{"given":"N. A. J.","family":"Cornelissen","sequence":"first","affiliation":[]},{"given":"R. J. M.","family":"van Eerdt","sequence":"additional","affiliation":[]},{"given":"H. K.","family":"Schraffenberger","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4077-9560","authenticated-orcid":false,"given":"W. F. G.","family":"Haselager","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,3,21]]},"reference":[{"issue":"4","key":"9645_CR1","first-page":"199","volume":"52","author":"B Budgell","year":"2008","unstructured":"Budgell, B. (2008). Commentary guidelines to the writing of case studies. The Journal of the Canadian Chiropractic Association, 52(4), 199\u2013204.","journal-title":"The Journal of the Canadian Chiropractic Association"},{"key":"9645_CR2","first-page":"1","volume-title":"The Truncated Normal Distribution","author":"J Burkardt","year":"2014","unstructured":"Burkardt, J. (2014). The Truncated Normal Distribution (pp. 1\u201335). Department of Scientific Computing Florida State University."},{"issue":"4","key":"9645_CR3","doi-asserted-by":"publisher","first-page":"688","DOI":"10.1016\/j.clinthera.2015.12.001","volume":"38","author":"Y Chen","year":"2016","unstructured":"Chen, Y., Argentinis, E., & Weber, G. (2016). IBM Watson: How cognitive computing can be applied to big data challenges in life sciences research. Clinical Therapeutics, 38(4), 688\u2013701. https:\/\/doi.org\/10.1016\/j.clinthera.2015.12.001","journal-title":"Clinical Therapeutics"},{"key":"9645_CR100","doi-asserted-by":"crossref","unstructured":"Cox, A. L., Gould, S. J., Cecchinato, M. E., Iacovides, I., & Renfree, I. (2016). Design frictions for mindful interactions: The case for microboundaries. In Proceedings of the 2016 CHI conference extended abstracts on human factors in computing systems (pp. 1389\u20131397).","DOI":"10.1145\/2851581.2892410"},{"key":"9645_CR4","volume-title":"AI and ethics at the police: Towards responsible use of artificial intelligence in the Dutch Police","author":"F Dechesne","year":"2019","unstructured":"Dechesne, F., Dignum, V., Zardiashvili, L., & Bieger, J. (2019). AI and ethics at the police: Towards responsible use of artificial intelligence in the Dutch Police. Leiden University."},{"key":"9645_CR16","doi-asserted-by":"publisher","DOI":"10.1007\/s13347-021-00450-x","author":"F de Santoni","year":"2021","unstructured":"de Santoni, F., & Mecacci, G. (2021). Four responsibility gaps with artificial intelligence: Why they matter and how to address them. Philosophy and Technology. https:\/\/doi.org\/10.1007\/s13347-021-00450-x","journal-title":"Philosophy and Technology"},{"issue":"FEB","key":"9645_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/frobt.2018.00015","volume":"5","author":"F de Santoni","year":"2018","unstructured":"de Santoni, F., & van den Hoven, J. (2018). Meaningful human control over autonomous systems: A philosophical account. Frontiers Robotics AI, 5(FEB), 1\u201314. https:\/\/doi.org\/10.3389\/frobt.2018.00015","journal-title":"Frontiers Robotics AI"},{"key":"9645_CR5","volume-title":"Attention and performance limitations.\u00a0Foundations of cognitive psychology:\u00a0Core readings","author":"MW Eysenck","year":"2002","unstructured":"Eysenck, M. W., & Keane, M. T. (2002). Attention and performance limitations.\u00a0Foundations of cognitive psychology:\u00a0Core readings. MIT Press."},{"key":"9645_CR105","unstructured":"Friedman, B., Kahn, P. H., Borning, A., & Huldtgren, A. (2013). Value sensitive design and information systems. In K. E. Himma & H. T. Tavani (Eds.), The handbook of information and computer ethics (pp. 55\u201395). Hoboken, NJ: Wiley."},{"issue":"6","key":"9645_CR6","first-page":"320","volume":"44","author":"M Grissinger","year":"2019","unstructured":"Grissinger, M. (2019). Understanding human over-reliance on technology. Pharmacy and Therapeutics, 44(6), 320\u2013321.","journal-title":"Pharmacy and Therapeutics"},{"issue":"2","key":"9645_CR7","doi-asserted-by":"publisher","first-page":"149","DOI":"10.17705\/1thci.00093","volume":"9","author":"S Kieffer","year":"2017","unstructured":"Kieffer, S. (2017). Representative design in user experience evaluations. AIS Transactions on Human-Computer Interaction, 9(2), 149\u2013172.","journal-title":"AIS Transactions on Human-Computer Interaction"},{"issue":"8","key":"9645_CR8","doi-asserted-by":"publisher","first-page":"1223","DOI":"10.2196\/17283","volume":"8","author":"CS Kruse","year":"2020","unstructured":"Kruse, C. S., & Ehrbar, N. (2020). Effects of computerized decision support systems on practitioner performance and patient outcomes: Systematic review. JMIR Medical Informatics, 8(8), 1223\u20131238. https:\/\/doi.org\/10.2196\/17283","journal-title":"JMIR Medical Informatics"},{"key":"9645_CR103","doi-asserted-by":"publisher","unstructured":"Lauermann, F., & Karabenick, S. A. (2013). The meaning and measure of teachers\u2019 sense of responsibility for educational outcomes. Teaching and Teacher Education, 30, 13\u201326. https:\/\/doi.org\/10.1016\/j.tate.2012.10.001","DOI":"10.1016\/j.tate.2012.10.001"},{"issue":"2","key":"9645_CR9","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1080\/17579961.2018.1527480","volume":"10","author":"HY Liu","year":"2018","unstructured":"Liu, H. Y. (2018). The power structure of artificial intelligence. Law, Innovation and Technology, 10(2), 197\u2013229. https:\/\/doi.org\/10.1080\/17579961.2018.1527480","journal-title":"Law, Innovation and Technology"},{"key":"9645_CR10","doi-asserted-by":"publisher","unstructured":"Matthias, A. (2004). The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics and Information Technology, 6(3), 175\u2013183. https:\/\/doi.org\/10.1007\/s10676-004-3422-1","DOI":"10.1007\/s10676-004-3422-1"},{"key":"9645_CR11","doi-asserted-by":"publisher","unstructured":"Mecacci, G., & de Santoni, F. (2020). Meaningful human control as reason-responsiveness: The case of dual-mode vehicles. Ethics and Information Technology, 22(2), 103\u2013115. https:\/\/doi.org\/10.1007\/s10676-019-09519-w","DOI":"10.1007\/s10676-019-09519-w"},{"issue":"FEB","key":"9645_CR12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fpsyg.2019.00225","volume":"10","author":"SM Merritt","year":"2019","unstructured":"Merritt, S. M., Ako-Brew, A., Bryant, W. J., Staley, A., McKenna, M., Leone, A., & Shirase, L. (2019). Automation-induced complacency potential: Development and validation of a new \nscale. Frontiers in Psychology, 10(FEB), 1\u201313. https:\/\/doi.org\/10.3389\/fpsyg.2019.00225","journal-title":"Frontiers in Psychology"},{"issue":"3","key":"9645_CR13","doi-asserted-by":"publisher","first-page":"642","DOI":"10.1016\/j.ejor.2007.11.001","volume":"195","author":"G Phillips-Wren","year":"2009","unstructured":"Phillips-Wren, G., Mora, M., Forgionne, G. A., & Gupta, J. N. D. (2009). An integrative evaluation framework for intelligent decision support systems. European Journal of Operational Research, 195(3), 642\u2013652. https:\/\/doi.org\/10.1016\/j.ejor.2007.11.001","journal-title":"European Journal of Operational Research"},{"key":"9645_CR14","unstructured":"Robbins, J. E. (1998). Design Critiquing Systems. Technical Report UCI-98-41."},{"issue":"3","key":"9645_CR15","first-page":"1","volume":"8","author":"A Roig","year":"2017","unstructured":"Roig, A. (2017). Safeguards for the right not to be subject to a decision based solely on automated processing (Article 22 GDPR). European Journal of Law and Technology, 8(3), 1\u201317.","journal-title":"European Journal of Law and Technology"},{"key":"9645_CR104","doi-asserted-by":"publisher","unstructured":"Tapal, A., Oren, E., Dar, R., & Eitam, B. (2017). The sense of agency scale: A measure of consciously perceived control over one\u2019s mind, body, and the immediate environment. Frontiers in Psychology, 8, 1552. https:\/\/doi.org\/10.3389\/fpsyg.2017.01552","DOI":"10.3389\/fpsyg.2017.01552"},{"key":"9645_CR101","unstructured":"van der Stigchel, B., van den Bosch, K., van Diggelen, J., & Haselager, W. (submitted) Intelligent decision support in medical triage: are people robust to biased advice? Journal of Public Health."},{"key":"9645_CR102","unstructured":"van der Stigchel, B. (2021). Resilience towards bias in artificial agents: Human-Agent team performing a triage task. MSc Thesis, dpt. of Artificial Intelligence, Radboud University, Nijmegen, The Netherlands."},{"key":"9645_CR18","volume-title":"Falsification machines in medical decision making","author":"R van Eerdt","year":"2021","unstructured":"van Eerdt, R. (2021). Falsification machines in medical decision making. Radboud University."},{"key":"9645_CR19","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v38i3.2741","author":"S Wachter","year":"2017","unstructured":"Wachter, S., Mittelstadt, B., & Floridi, L. (2017). Why a right to explanation of automated decision-making does not exist in the General Data Protection Regulation. SSRN. https:\/\/doi.org\/10.1609\/aimag.v38i3.2741","journal-title":"SSRN"}],"container-title":["Ethics and Information Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10676-022-09645-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10676-022-09645-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10676-022-09645-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,29]],"date-time":"2022-06-29T04:29:40Z","timestamp":1656476980000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10676-022-09645-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,21]]},"references-count":25,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,6]]}},"alternative-id":["9645"],"URL":"https:\/\/doi.org\/10.1007\/s10676-022-09645-y","relation":{},"ISSN":["1388-1957","1572-8439"],"issn-type":[{"value":"1388-1957","type":"print"},{"value":"1572-8439","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,21]]},"assertion":[{"value":"23 February 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 March 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no conflicts of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"19"}}