{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,20]],"date-time":"2026-04-20T23:42:13Z","timestamp":1776728533566,"version":"3.51.2"},"reference-count":53,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,8,23]],"date-time":"2023-08-23T00:00:00Z","timestamp":1692748800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p><jats:bold>Introduction:<\/jats:bold> Collaboration in teams composed of both humans and automation has an interdependent nature, which demands calibrated trust among all the team members. For building suitable autonomous teammates, we need to study how trust and trustworthiness function in such teams. In particular, automation occasionally fails to do its job, which leads to a decrease in a human\u2019s trust. Research has found interesting effects of such a reduction of trust on the human\u2019s trustworthiness, i.e., human characteristics that make them more or less reliable. This paper investigates how automation failure in a human-automation collaborative scenario affects the human\u2019s trust in the automation, as well as a human\u2019s trustworthiness towards the automation.<\/jats:p><jats:p><jats:bold>Methods:<\/jats:bold> We present a 2 \u00d7 2 mixed design experiment in which the participants perform a simulated task in a 2D grid-world, collaborating with an automation in a \u201cmoving-out\u201d scenario. During the experiment, we measure the participants\u2019 trustworthiness, trust, and liking regarding the automation, both subjectively and objectively.<\/jats:p><jats:p><jats:bold>Results:<\/jats:bold> Our results show that automation failure negatively affects the human\u2019s trustworthiness, as well as their trust in and liking of the automation.<\/jats:p><jats:p><jats:bold>Discussion:<\/jats:bold> Learning the effects of automation failure in trust and trustworthiness can contribute to a better understanding of the nature and dynamics of trust in these teams and improving human-automation teamwork.<\/jats:p>","DOI":"10.3389\/frobt.2023.1143723","type":"journal-article","created":{"date-parts":[[2023,8,23]],"date-time":"2023-08-23T19:24:13Z","timestamp":1692818653000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":14,"title":["Exploring the effect of automation failure on the human\u2019s trustworthiness in human-agent teamwork"],"prefix":"10.3389","volume":"10","author":[{"given":"Carolina","family":"Centeio Jorge","sequence":"first","affiliation":[]},{"given":"Nikki H.","family":"Bouman","sequence":"additional","affiliation":[]},{"given":"Catholijn M.","family":"Jonker","sequence":"additional","affiliation":[]},{"given":"Myrthe L.","family":"Tielman","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2023,8,23]]},"reference":[{"key":"B1","volume-title":"Trust in automated systems","author":"Adams","year":"2003"},{"key":"B2","doi-asserted-by":"publisher","first-page":"815","DOI":"10.1111\/srt.12726","article-title":"Data augmentation in dermatology image recognition using machine learning","volume":"25","author":"Aggarwal","year":"2019","journal-title":"Skin Res. Technol."},{"key":"B3","doi-asserted-by":"publisher","first-page":"103858","DOI":"10.1016\/j.apergo.2022.103858","article-title":"Differential biases in human-human versus human-robot interactions","volume":"106","author":"Alarcon","year":"2023","journal-title":"Appl. Ergon."},{"key":"B4","doi-asserted-by":"publisher","first-page":"103350","DOI":"10.1016\/j.apergo.2020.103350","article-title":"Exploring the differential effects of trust violations in human-human and human-robot interactions","volume":"93","author":"Alarcon","year":"2021","journal-title":"Appl. Ergon."},{"key":"B5","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1037\/1082-989x.10.3.317","article-title":"An alternative to cohen\u2019s standardized mean difference effect size: a robust parameter and confidence interval in the two independent groups case","volume":"10","author":"Algina","year":"2005","journal-title":"Psychol. methods"},{"key":"B6","doi-asserted-by":"publisher","first-page":"835","DOI":"10.1007\/s00426-008-0197-8","article-title":"Anticipation of cognitive demand during decision-making","volume":"73","author":"Botvinick","year":"2009","journal-title":"Psychol. Res."},{"key":"B7","doi-asserted-by":"publisher","first-page":"1260","DOI":"10.1016\/j.ssci.2009.03.015","article-title":"Does projection into use improve trust and exploration? An example with a cruise control system","volume":"47","author":"Cahour","year":"2009","journal-title":"Saf. Sci."},{"key":"B8","article-title":"Trust should correspond to trustworthiness: a formalization of appropriate mutual trust in human-agent teams","volume-title":"Proceedings of the international workshop in agent societies","author":"Centeio Jorge","year":"2021"},{"key":"B9","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1109\/THMS.2013.2293535","article-title":"Human - agent teaming for multirobot control: a review of human factors issues","volume":"44","author":"Chen","year":"2014","journal-title":"IEEE Trans. Human-Machine Syst."},{"key":"B10","volume-title":"Designing for trust exploring trust and collaboration in conversational agents for E-commerce. Ph.D. Thesis, school of design","author":"Dagli","year":"2018"},{"key":"B11","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1177\/001872676001300202","article-title":"The effect of motivational orientation upon trust and suspicion","volume":"13","author":"Deutschi","year":"1960","journal-title":"Hum. Relat."},{"key":"B12","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1518\/0018720024494856","article-title":"The perceived utility of human and automated aids in a visual detection task","volume":"44","author":"Dzindolet","year":"2002","journal-title":"Hum. Factors"},{"key":"B13","doi-asserted-by":"publisher","first-page":"193124312311581","DOI":"10.1177\/19312431231158120","article-title":"Human or machine? How much difference in understanding and trust does a human element make in storm forecasts?","author":"Eicher","year":"2023","journal-title":"Electron. News"},{"key":"B14","first-page":"740","article-title":"Trust dynamics: how trust is influenced by direct experiences and by trust itself","volume-title":"Proceedings of the third international joint conference on autonomous agents and multiagent systems","author":"Falcone","year":"2004"},{"key":"B15","first-page":"226","article-title":"\u201cIt\u2019s the computer\u2019s fault\u201d -reasoning about computers as moral agents","volume-title":"Conference companion on Human factors in computing systems","author":"Frieainan","year":"1995"},{"key":"B16","doi-asserted-by":"publisher","first-page":"483","DOI":"10.1075\/is.8.3.10gro","article-title":"Can robots be teammates? Benchmarks in human-robot teams","volume":"8","author":"Groom","year":"2007","journal-title":"Interact. Stud."},{"key":"B17","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1177\/0018720811417254","article-title":"A meta-analysis of factors affecting trust in human-robot interaction","volume":"53","author":"Hancock","year":"2011","journal-title":"Hum. Factors"},{"key":"B18","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1177\/0018720814547570","article-title":"Trust in automation: integrating empirical evidence on factors that influence trust","volume":"57","author":"Hoff","year":"2015","journal-title":"Hum. factors"},{"key":"B19","volume-title":"Metrics for explainable AI: challenges and prospects","author":"Hoffman","year":"2018"},{"key":"B20","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1207\/s15327566ijce0401_04","article-title":"Foundations for an empirically determined scale of trust in automated systems","volume":"4","author":"Jian","year":"2000","journal-title":"Int. J. cognitive ergonomics"},{"key":"B21","doi-asserted-by":"publisher","first-page":"531","DOI":"10.5465\/amr.1998.926625","article-title":"The experience and evolution of trust: implications for cooperation and teamwork","volume":"3","author":"Jones","year":"1998","journal-title":"Tech. Rep."},{"key":"B22","doi-asserted-by":"publisher","first-page":"515","DOI":"10.1037\/a0016755","article-title":"Conditions for intuitive expertise: a failure to disagree","volume":"64","author":"Kahneman","year":"2009","journal-title":"Am. Psychol."},{"key":"B23","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1109\/mis.2004.74","article-title":"Ten challenges for making automation a \u201cteam player\u201d in joint human-agent activity","volume":"19","author":"Klein","year":"2004","journal-title":"IEEE Intell. Syst."},{"key":"B24","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1177\/00187208211021711","article-title":"The effect of informing participants of the response bias of an automated target recognition system on trust and reliance behavior","volume":"65","author":"Knocton","year":"2023","journal-title":"Hum. Factors"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2021.604977","article-title":"Measurement of trust in automation: a narrative review and reference guide","author":"Kohn","year":"2021"},{"key":"B26","doi-asserted-by":"publisher","first-page":"103095","DOI":"10.1016\/j.ijhcs.2023.103095","article-title":"\u201cIt\u2019s not Paul, it\u2019s a robot\u201d: the impact of linguistic framing and the evolution of trust and distrust in a collaborative robot during a human-robot interaction","volume":"178","author":"Kopp","year":"2023","journal-title":"Int. J. Human-Computer Stud."},{"key":"B27","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1007\/s10458-021-09515-9","article-title":"Trust repair in human-agent teams: the effectiveness of explanations and expressing regret","volume":"35","author":"Kox","year":"2021","journal-title":"Aut. Agents Multi-Agent Syst."},{"key":"B28","article-title":"Current and emerging applications of innovative artificial intelligence in modern medicine and Technology","volume":"1","author":"Laurent","year":"2019","journal-title":"Tech. Rep."},{"key":"B29","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1518\/hfes.46.1.50_30392","article-title":"Trust in automation: designing for appropriate reliance","volume":"46","author":"Lee","year":"2004","journal-title":"Hum. Factors"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1243","DOI":"10.1080\/00140139208967392","article-title":"Trust, control strategies and allocation of function in human-machine systems","volume":"35","author":"Lee","year":"1992","journal-title":"Ergonomics"},{"key":"B31","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1518\/001872006777724408","article-title":"Automation failures on tasks easily performed by operators undermine trust in automated aids","volume":"48","author":"Madhavan","year":"2006","journal-title":"Hum. factors"},{"key":"B32","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1080\/14639220500337708","article-title":"Similarities and differences between human\u2013human and human\u2013automation trust: an integrative review","volume":"8","author":"Madhavan","year":"2007","journal-title":"Theor. Issues Ergonomics Sci."},{"key":"B33","first-page":"6","article-title":"Measuring human-computer trust","volume-title":"11th australasian conference on information systems","author":"Madsen","year":"2000"},{"key":"B34","doi-asserted-by":"publisher","first-page":"464","DOI":"10.3758\/s13428-019-01246-w","article-title":"Robust statistical methods in R using the WRS2 package","volume":"52","author":"Mair","year":"2020","journal-title":"Behav. Res. Methods"},{"key":"B35","doi-asserted-by":"publisher","first-page":"709","DOI":"10.5465\/amr.1995.9508080335","article-title":"An integrative model of organizational trust","volume":"20","author":"Mayer","year":"1995","journal-title":"Acad. Manag. Rev."},{"key":"B36","doi-asserted-by":"publisher","first-page":"356","DOI":"10.1177\/0018720811411912","article-title":"Affective processes in human-automation interactions","volume":"53","author":"Merritt","year":"2011","journal-title":"Hum. Factors"},{"key":"B37","doi-asserted-by":"publisher","first-page":"520","DOI":"10.1177\/0018720812465081","article-title":"I trust it, but i don\u2019t know why: effects of implicit attitudes toward automation on trust in an automated system","volume":"55","author":"Merritt","year":"2013","journal-title":"Hum. Factors"},{"key":"B38","doi-asserted-by":"publisher","first-page":"740","DOI":"10.1177\/0018720815581247","article-title":"Measuring individual differences in the perfect automation schema","volume":"57","author":"Merritt","year":"2015","journal-title":"Hum. factors"},{"key":"B39","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1037\/1076-898x.6.1.44","article-title":"Adaptive automation, trust, and self-confidence in fault management of time-critical tasks","volume":"6","author":"Moray","year":"2000","journal-title":"J. Exp. Psychol. Appl."},{"key":"B40","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1007\/bf00150235","article-title":"Competence and responsibility in intelligent systems","volume":"6","author":"Morgan","year":"1992","journal-title":"Artif. Intell. Rev."},{"key":"B41","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1177\/0092070301291001","article-title":"The role of interpersonal liking in building trust in long-term channel relationships","volume":"29","author":"Nicholson","year":"2001","journal-title":"Tech. Rep."},{"key":"B42","article-title":"Metrics for evaluating human-robot interactions","volume-title":"Proceedings of PERMIS","author":"Olsen","year":"2003"},{"key":"B43","doi-asserted-by":"publisher","first-page":"286","DOI":"10.1109\/3468.844354","article-title":"A model for types and levels of human interaction with automation","volume":"30","author":"Parasuraman","year":"2000","journal-title":"IEEE Trans. Syst. Man, Cybern. Part A:Systems Humans"},{"key":"B44","doi-asserted-by":"publisher","first-page":"425","DOI":"10.1109\/THMS.2017.2648849","article-title":"Effect of robot performance on human-robot trust in time-critical situations","volume":"47","author":"Robinette","year":"2017","journal-title":"IEEE Trans. Human-Machine Syst."},{"key":"B45","doi-asserted-by":"publisher","first-page":"555","DOI":"10.1177\/1046496405277134","article-title":"Is there a \u201cbig five\u201d in teamwork?","volume":"36","author":"Salas","year":"2005","journal-title":"Small group Res."},{"key":"B46","first-page":"141","article-title":"Would you trust a (faulty) robot?","volume-title":"ACM\/IEEE international conference on human-robot interaction","author":"Salem","year":"2015"},{"key":"B47","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1207\/s15327108ijap0302_2","article-title":"Automation-induced \u201ccomplacency\u201d: development of the complacency-potential rating scale","volume":"3","author":"Singh","year":"1993","journal-title":"Int. J. Aviat. Psychol."},{"key":"B48","doi-asserted-by":"publisher","first-page":"2059","DOI":"10.1016\/j.socec.2007.10.004","article-title":"Trust-The importance of trustfulness versus trustworthiness","volume":"37","author":"Tullberg","year":"2008","journal-title":"J. Socio-Economics"},{"key":"B49","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3152890","article-title":"Modeling the human-robot trust phenomenon: a conceptual framework based on risk","volume":"8","author":"Wagner","year":"2018","journal-title":"ACM Trans. Interact. Intelligent Syst."},{"key":"B50","doi-asserted-by":"publisher","first-page":"746","DOI":"10.1177\/1046496408323569","article-title":"Development of cognitive and affective trust in teams: a longitudinal study","volume":"39","author":"Webber","year":"2008","journal-title":"Small Group Res."},{"key":"B51","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2018\/8145713","article-title":"Skin disease recognition method based on image color and texture features","volume":"2018","author":"Wei","year":"2018","journal-title":"Comput. Math. Methods Med."},{"key":"B52","first-page":"465","article-title":"The effect of agent reasoning transparency on automation bias: an analysis of response performance","volume-title":"Lect. Notes Comput. Sci. Incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinforma.","author":"Wright","year":"2016"},{"key":"B53","doi-asserted-by":"publisher","first-page":"103031","DOI":"10.1016\/j.ijhcs.2023.103031","article-title":"\u201cSorry, it was my fault\u201d: repairing trust in human-robot interactions","volume":"175","author":"Zhang","year":"2023","journal-title":"Int. J. Human-Computer Stud."}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2023.1143723\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,5]],"date-time":"2024-08-05T07:25:02Z","timestamp":1722842702000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2023.1143723\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,23]]},"references-count":53,"alternative-id":["10.3389\/frobt.2023.1143723"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2023.1143723","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,23]]},"article-number":"1143723"}}