{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T13:25:03Z","timestamp":1777728303251,"version":"3.51.4"},"reference-count":28,"publisher":"SAGE Publications","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IA"],"published-print":{"date-parts":[[2022,7,8]]},"abstract":"<jats:p>Autonomy is crucial in cooperation. The complexity of HRI scenarios requires autonomous robots able to exploit their superhuman computations (based on DNN, Machine Learning techniques and Big Data) in a trustworthy way. Trustworthiness is not only a matter of accuracy, privacy or security, but it is becoming more and more a matter of adaptation to humans agency. As claimed by Falcone and Castelfranchi, autonomy means the possibility of dislaying or providing an unexpected behavior (including refusal) that departs from a requested (agreed upon or not) behavior. In this sense, the autonomy to decide how to adopt a task delegated by the user, with respect to her\/his own real needs and goals, distinguishes intelligent and trustworthy robots from highly performing robots. This kind of smart help can be provided only by cognitive robots able to represent and ascribe mental states (beliefs, goals, intentions, desires etc.) to their interlocutors. The mental states attribution can be the result of complex reasoning mechanisms or can be fast and automatic, based on scripts, roles, categories or stereotypes typically exploited by humans every time they interact in everyday life. In all these cases, robots that build and use cognitive models of humans (that have a Theory of Mind of their interlocutors), have to operate also a meta-evaluation of their own predictive skills to build those models. Robots have to be endowed with the capability to self-trust their skills to interpret the interlocutors and the context, for producing smart and effective decisions towards humans. After exploring the main concepts that make collaboration between humans and robots trustworthy and effective, we present the first of a series of experiments draw for testing different aspects of a designed cognitive architecture for trustworthy HRI. This architecture, based on consolidated theoretical principles (theory of social adjustable autonomy, theory of mind, theory of trust) has the main goal to build cognitive robots that provide smart, trustworthy collaboration, every time a human requires their help. In particular, the experiment has been designed in order to demonstrate how the robot\u2019s capability to learn its own level of self-trust on its predictive abilities in perceiving the user and building a model of her\/him, allows it to establish a trustworthy collaboration and to maintain a high level of user\u2019s satisfaction, with respect to the robot\u2019s performance, also when these abilities progressively degrade.<\/jats:p>","DOI":"10.3233\/ia-210124","type":"journal-article","created":{"date-parts":[[2022,7,8]],"date-time":"2022-07-08T11:09:51Z","timestamp":1657278591000},"page":"69-79","source":"Crossref","is-referenced-by-count":3,"title":["Human-robot interaction through adjustable social autonomy"],"prefix":"10.1177","volume":"16","author":[{"given":"Filippo","family":"Cantucci","sequence":"first","affiliation":[{"name":"Institute of Cognitive Science and Technology, National Research Council of Italy, (ISTC-CNR), Rome"}]},{"given":"Rino","family":"Falcone","sequence":"additional","affiliation":[{"name":"Institute of Cognitive Science and Technology, National Research Council of Italy, (ISTC-CNR), Rome"}]},{"given":"Cristiano","family":"Castelfranchi","sequence":"additional","affiliation":[{"name":"Institute of Cognitive Science and Technology, National Research Council of Italy, (ISTC-CNR), Rome"}]}],"member":"179","reference":[{"issue":"6","key":"10.3233\/IA-210124_ref1","doi-asserted-by":"crossref","first-page":"747","DOI":"10.1016\/j.scico.2011.10.004","article-title":"Multi-agent oriented programming with jacamo","volume":"78","author":"Boissier","year":"2013","journal-title":"Science of Computer Programming"},{"key":"10.3233\/IA-210124_ref2","doi-asserted-by":"crossref","unstructured":"Rafael H. Bordini and Jomi F. H\u00fcbner, Bdi agent programming in agentspeak using jason. In International Workshop on Computational Logic in Multi-Agent Systems, pp. 143\u2013164. Springer, 2005.","DOI":"10.1007\/11750734_9"},{"key":"10.3233\/IA-210124_ref3","doi-asserted-by":"crossref","unstructured":"Braunhofer M. , Elahi M. and Ricci F. , User personality and the new user problem in a context-aware point of interest recommender system. In Information and Communication Technologies in Tourism 2015, pp. 537\u2013549. Springer, 2015.","DOI":"10.1007\/978-3-319-14343-9_39"},{"key":"10.3233\/IA-210124_ref4","first-page":"1","article-title":"Towards trustworthiness and transparency in social human-robot interaction","author":"Cantucci","year":"2020","journal-title":"2020 IEEE International Conference on Human-Machine Systems (ICHMS)"},{"issue":"3-4","key":"10.3233\/IA-210124_ref5","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1016\/S0921-8890(98)00028-1","article-title":"Towards a theory of delegation for agent-based systems","volume":"24","author":"Castelfranchi","year":"1998","journal-title":"Robotics and Autonomous Systems"},{"key":"10.3233\/IA-210124_ref6","doi-asserted-by":"crossref","unstructured":"Castelfranchi C. and Falcone R. , Trust theory: A socio-cognitive and computational model, volume 18. John Wiley & Sons, 2010.","DOI":"10.1002\/9780470519851"},{"key":"10.3233\/IA-210124_ref7","first-page":"307","article-title":"Planning with trust for human-robot collaboration","author":"Chen","year":"2018","journal-title":"Proceedings of the 2018 ACM\/IEEE International Conference on Human-Robot Interaction"},{"key":"10.3233\/IA-210124_ref8","unstructured":"Commission E. , White paper on artificial intelligencea european approach to excellence and trust. Com, (2020) 65 Final, 2020."},{"key":"10.3233\/IA-210124_ref9","doi-asserted-by":"crossref","unstructured":"Cooper S. , Fava A.D. , Vivas C. , Marchionni L. and Ferro F. , Ari: The social assistive robot and companion. In 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), (2020), pp. 745\u2013751. IEEE.","DOI":"10.1109\/RO-MAN47096.2020.9223470"},{"key":"10.3233\/IA-210124_ref10","doi-asserted-by":"crossref","unstructured":"D\u2019Onofrio G. , Sancarlo D. , Raciti M. , Reforgiato D. , Mangiacotti A. , Russo A. , Ricciardi F. , Vitanza A. , Cantucci F. , Presutti V. , et al., Mario project: Experimentation in the hospital setting. In Italian Forum of Ambient Assisted Living, pp. 289\u2013303. Springer, 2017.","DOI":"10.1007\/978-3-030-04672-9_20"},{"key":"10.3233\/IA-210124_ref11","doi-asserted-by":"crossref","unstructured":"Eder K. , Gaining confidence in the trustworthiness of robotic and autonomous systems. In Software Engineering for Robotics, pp. 139\u2013164. Springer, 2021.","DOI":"10.1007\/978-3-030-66494-7_5"},{"issue":"5","key":"10.3233\/IA-210124_ref12","doi-asserted-by":"crossref","first-page":"406","DOI":"10.1109\/3468.952715","article-title":"The human in the loop of a delegated agent: The theory of adjustable social autonomy","volume":"31","author":"Falcone","year":"2001","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans"},{"issue":"1","key":"10.3233\/IA-210124_ref13","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1007\/s12194-019-00552-4","article-title":"Ai-based computer-aided diagnosis (ai-cad): the latest review to read first","volume":"13","author":"Fujita","year":"2020","journal-title":"Radiological Physics and Technology"},{"key":"10.3233\/IA-210124_ref14","unstructured":"Michael P. Georgeff and Amy L. Lansky, Reactive reasoning and planning. In volume pp, AAAI 87 (1987), 677\u2013682."},{"key":"10.3233\/IA-210124_ref15","doi-asserted-by":"crossref","unstructured":"Hadfield-Menell D. , Dragan A. , Abbeel P. and Russell S. , The off-switch game. arXiv preprint arXiv:1611.08219, 2016.","DOI":"10.24963\/ijcai.2017\/32"},{"key":"10.3233\/IA-210124_ref16","doi-asserted-by":"crossref","unstructured":"Hardalov M. , Koychev I. and Nakov P. , Towards automated customer support. In International Conference on Artificial Intelligence: Methodology, Systems, and Applications, pp. 48\u201359. Springer, 2018.","DOI":"10.1007\/978-3-319-99344-7_5"},{"key":"10.3233\/IA-210124_ref17","doi-asserted-by":"crossref","unstructured":"Israelsen B. , Ahmed N. , Frew E. , Lawrence D. , and Brian Argrow. Machine self-confidence in autonomous systems via meta-analysis of decision processes. In International Conference on Applied Human Factors and Ergonomics, pp. 213\u2013223. Springer, 2019.","DOI":"10.1007\/978-3-030-20454-9_21"},{"key":"10.3233\/IA-210124_ref18","doi-asserted-by":"crossref","unstructured":"Jarvis B. and Jain L. , Trust in lora: Towards a formal definition of trust in bdi agents. In International Conference on Knowledge-Based and Intelligent Information and Engineering Systems, pp. 458\u2013463. Springer, 2006.","DOI":"10.1007\/11893004_59"},{"key":"10.3233\/IA-210124_ref19","first-page":"1","article-title":"Trust in robots: Challenges and opportunities","author":"Cai Kok","year":"2020","journal-title":"Current Robotics Reports"},{"key":"10.3233\/IA-210124_ref20","unstructured":"Levin S. and Carrie J. , Wong, Self-driving uber kills arizona woman in first fatal crash involving pedestrian, The Guardian, 19, 2018."},{"key":"10.3233\/IA-210124_ref21","doi-asserted-by":"crossref","unstructured":"Mohamed M.H. , Khafagy M.H. and Ibrahim M.H. , Recommender systems challenges and solutions survey. In 2019 International Conference on Innovative Trends in Computer Engineering (ITCE), pp. 149\u2013155. IEEE, 2019.","DOI":"10.1109\/ITCE.2019.8646645"},{"key":"10.3233\/IA-210124_ref22","doi-asserted-by":"crossref","first-page":"102888","DOI":"10.1016\/j.annals.2020.102888","article-title":"Multifaceted trust in tourism service robots","volume":"81","author":"Park","year":"2020","journal-title":"Annals of Tourism Research"},{"key":"10.3233\/IA-210124_ref23","doi-asserted-by":"crossref","unstructured":"Michael J. Pazzani, A framework for collaborative, contentbased and demographic filtering, Artificial Intelligence Review 13(5) (1999), 393\u2013408.","DOI":"10.1023\/A:1006544522159"},{"key":"10.3233\/IA-210124_ref24","unstructured":"Anand S. Rao, Michael P. Georgeff, et al., Bdi agents: from theory to practice. In ICMAS, volume 95 (1995), pp. 312\u2013319."},{"key":"10.3233\/IA-210124_ref25","first-page":"1","article-title":"Let\u2019s talk about race: Identity, chatbots, and ai","author":"Schlesinger","year":"2018","journal-title":"Proceedings of the 2018 chi conference on human factors in computing systems"},{"issue":"7839","key":"10.3233\/IA-210124_ref26","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1038\/s41586-020-03051-4","article-title":"Mastering atari, go, chess and shogi by planning with a learned model","volume":"588","author":"Schrittwieser","year":"2020","journal-title":"Nature"},{"key":"10.3233\/IA-210124_ref27","doi-asserted-by":"crossref","unstructured":"Nathalie A. Smuha, The eu approach to ethics guidelines for trustworthy artificial intelligence, Computer Law Review International 20(4) (2019), 97\u2013106.","DOI":"10.9785\/cri-2019-200402"},{"key":"10.3233\/IA-210124_ref28","doi-asserted-by":"crossref","unstructured":"Hannah J. Swift, D. Abrams, L. Drury and Ruth A. Lamont, Categorization by age, Encyclopedia of Evolutionary Psychological Science, 2018.","DOI":"10.1007\/978-3-319-16999-6_2431-1"}],"container-title":["Intelligenza Artificiale"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/IA-210124","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:51:44Z","timestamp":1777459904000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/IA-210124"}},"subtitle":[],"editor":[{"given":"Roberta","family":"Calegari","sequence":"additional","affiliation":[]},{"given":"Giovanni","family":"Ciatto","sequence":"additional","affiliation":[]},{"given":"Andrea","family":"Omicini","sequence":"additional","affiliation":[]},{"given":"Giuseppe","family":"Vizzari","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2022,7,8]]},"references-count":28,"journal-issue":{"issue":"1"},"URL":"https:\/\/doi.org\/10.3233\/ia-210124","relation":{},"ISSN":["1724-8035","2211-0097"],"issn-type":[{"value":"1724-8035","type":"print"},{"value":"2211-0097","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,8]]}}}