{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T22:14:38Z","timestamp":1758924878982,"version":"3.41.0"},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2022,3,4]],"date-time":"2022-03-04T00:00:00Z","timestamp":1646352000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004359","name":"Swedish Research Council","doi-asserted-by":"crossref","award":["2016-03698"],"award-info":[{"award-number":["2016-03698"]}],"id":[{"id":"10.13039\/501100004359","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100011898","name":"Marcus and Amalia Wallenberg foundation","doi-asserted-by":"crossref","award":["MAW 2020.0052"],"award-info":[{"award-number":["MAW 2020.0052"]}],"id":[{"id":"10.13039\/501100011898","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Hum.-Robot Interact."],"published-print":{"date-parts":[[2022,6,30]]},"abstract":"<jats:p>The main aim of this study is to investigate if verbal, vocal, and facial information can be used to identify low-engaged second language learners in robot-led conversation practice. The experiments were performed on voice recordings and video data from 50 conversations, in which a robotic head talks with pairs of adult language learners using four different interaction strategies with varying robot-learner focus and initiative. It was found that these robot interaction strategies influenced learner activity and engagement. The verbal analysis indicated that learners with low activity rated the robot significantly lower on two out of four scales related to social competence. The acoustic vocal and video-based facial analysis, based on manual annotations or machine learning classification, both showed that learners with low engagement rated the robot\u2019s social competencies consistently, and in several cases significantly, lower, and in addition rated the learning effectiveness lower. The agreement between manual and automatic identification of low-engaged learners based on voice recordings or face videos was further found to be adequate for future use. These experiments constitute a first step towards enabling adaption to learners\u2019 activity and engagement through within- and between-strategy changes of the robot\u2019s interaction with learners.<\/jats:p>","DOI":"10.1145\/3503799","type":"journal-article","created":{"date-parts":[[2022,3,4]],"date-time":"2022-03-04T10:14:11Z","timestamp":1646388851000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Identification of Low-engaged Learners in Robot-led Second Language Conversations with Adults"],"prefix":"10.1145","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4532-014X","authenticated-orcid":false,"given":"Olov","family":"Engwall","sequence":"first","affiliation":[{"name":"KTH Royal Institute of Technology, Stockholm, Sweden"}]},{"given":"Ronald","family":"Cumbal","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology, Stockholm, Sweden"}]},{"given":"Jos\u00e9","family":"Lopes","sequence":"additional","affiliation":[{"name":"Heriot-Watt University, Edinburgh, United Kingdom"}]},{"given":"Mikael","family":"Ljung","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology, Stockholm, Sweden"}]},{"given":"Linnea","family":"M\u00e5nsson","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology, Stockholm, Sweden"}]}],"member":"320","published-online":{"date-parts":[[2022,3,4]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2017.1300750"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1177\/1059712319844182"},{"key":"e_1_3_1_4_2","first-page":"114","volume-title":"Furhat: A Back-projected Human-like Robot Head for Multiparty Human-Machine Interaction","author":"Moubayed Samer Al","year":"2012","unstructured":"Samer Al Moubayed, Jonas Beskow, Gabriel Skantze, and Bjorn Granstr\u00f6m. 2012. Furhat: A Back-projected Human-like Robot Head for Multiparty Human-Machine Interaction. Springer, 114\u2013130."},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/2818346.2820733"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.3233\/978-1-60750-028-5-17"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2016.7477553"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-019-00591-2"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1080\/08839514.2010.492259"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/2678025.2701397"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2005-446"},{"key":"e_1_3_1_12_2","first-page":"217","volume-title":"Proceedings of the Speech Prosody","author":"Campbell Nick","year":"2004","unstructured":"Nick Campbell. 2004. Accounting for voice-quality variation. In Proceedings of the Speech Prosody. 217\u2013220."},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12193-009-0033-5"},{"key":"e_1_3_1_14_2","first-page":"1","volume-title":"Proceedings of the Embodied Communication of Goals and Intentions Workshop","author":"Corrigan L. J.","year":"2013","unstructured":"L. J. Corrigan, Christopher Peters, Ginevra Castellano, Fotis Papadopoulos, Aidan Jones, Shweta Bhargava, Srini Janarthanam, Helen Hastie, Amol Deshmukh, and Ruth Aylett. 2013. Social-task engagement: Striking a balance between the robot and the task. In Proceedings of the Embodied Communication of Goals and Intentions Workshop. 1\u20137."},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1080\/01690960801920545"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3371382.3378306"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2020.00101"},{"key":"e_1_3_1_18_2","first-page":"75","volume-title":"Motivational Techniques of Expert Human Tutors: Lessons for the Design of Computer-based Tutors.","author":"Lepper Marc R. de","year":"1993","unstructured":"Marc R. de Lepper, Maria Woolverton, Donna L. Mumme, and Jean-Luc Gurtner. 1993. Motivational Techniques of Expert Human Tutors: Lessons for the Design of Computer-based Tutors.Lawrence Erlbaum Associates, Inc., 75\u2013105."},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2006-275"},{"key":"e_1_3_1_20_2","first-page":"34","volume-title":"Real-Life Emotion Recognition in Speech","author":"Devillers L.","year":"2007","unstructured":"L. Devillers and L. Vidrascu. 2007. Real-Life Emotion Recognition in Speech. Springer, Berlin, 34\u201342."},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1037\/a0032674"},{"key":"e_1_3_1_22_2","first-page":"45","volume-title":"Basic Emotions","author":"Ekman Paul","year":"1999","unstructured":"Paul Ekman. 1999. Basic Emotions. John Wiley & Sons, Chapter 3, 45\u201368."},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11031-008-9102-4"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1080\/09588221.2020.1799821"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-020-00635-y"},{"key":"e_1_3_1_26_2","article-title":"Learner and Teacher Perspectives on Robot-led L2 Conversation Practice","author":"Engwall Olov","unstructured":"Olov Engwall, Jos\u00e9 Lopes, Roland Cumbal, Gustav Berndtson, Ruben Lindstr\u00f6m, Patrik Ekman, Eric Hartmanis, Emelie Jin, Ella Johnston, Michael Mekonnen, and Gara Tahir. Accepted. Learner and Teacher Perspectives on Robot-led L2 Conversation Practice. ReCALL (Accepted).","journal-title":"ReCALL"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-017-0414-y"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3434073.3444670"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v30i1.9914"},{"key":"e_1_3_1_30_2","volume-title":"Proceedings of DiSS-LPSS Joint Workshop","author":"Gustafson Joakim","year":"2010","unstructured":"Joakim Gustafson and Daniel Neiberg. 2010. Prosodic cues to engagement in non-lexical response tokens in Swedish. In Proceedings of DiSS-LPSS Joint Workshop."},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.20982\/tqmp.08.1.p023"},{"key":"e_1_3_1_32_2","first-page":"323","volume-title":"IEEE Transactions on Affective Computing","author":"Hoque Ehsan","year":"2012","unstructured":"Ehsan Hoque, Rosalind Picard, Daniel McDuff, and Mohammed Hoque. 2012. Exploring temporal patterns in classifying frustrated and delighted smiles. In IEEE Transactions on Affective Computing 3, 3 323\u2013334."},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1111\/jcal.12325"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/34.574797"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1207\/s15327051hci1901&2_4"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2007.904904"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2007.02.003"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.21437\/SLaTE.2017-3"},{"key":"e_1_3_1_39_2","first-page":"341","volume-title":"Proceedings of the SIGDIAL 2011 Conference: 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue","author":"Klotz David","year":"2011","unstructured":"David Klotz, Johannes Wienke, Julia Peltason, Britta Wrede, Sebastian Wrede, Vasil Khalidov, and Jean-Marc Odobez. 2011. Engagement-based multi-party dialog with a humanoid robot. In Proceedings of the SIGDIAL 2011 Conference: 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue. 341\u2013343."},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.2307\/2529310"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/2696454.2696466"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1142\/S021984361550019X"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1093\/applin\/4.2.126"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.3390\/s18020416"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROMAN.2007.4415249"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/2818346.2820759"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2011-465"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_3_1_49_2","volume-title":"Proceedings of the Workshop on Affective Interaction in Natural Environments, ACM International Conference on Multimodal Interfaces","author":"Peters Christopher","year":"2008","unstructured":"Christopher Peters, Stylianos Asteriadis, Kostas Karpouzis, and Etienne de Sevin. 2008. Towards a real-time gaze-based shared attention for a virtual agent. In Proceedings of the Workshop on Affective Interaction in Natural Environments, ACM International Conference on Multimodal Interfaces."},{"key":"e_1_3_1_50_2","volume-title":"Mind, Hands, Face and Body: A Goal and Belief View of Multimodal Communication","author":"Poggi I.","year":"2007","unstructured":"I. Poggi. 2007. Mind, Hands, Face and Body: A Goal and Belief View of Multimodal Communication. Weidler."},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/1734454.1734580"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROMAN.2017.8172330"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/2388676.2388781"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366422"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/964442.964458"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2011-781"},{"key":"e_1_3_1_57_2","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1075\/lald.46.15uns","volume-title":"Current Trends in Child Second Language Acquisition: A Generative Perspective","author":"Unsworth Sharon","year":"2008","unstructured":"Sharon Unsworth. 2008. Comparing child L2 development with adult L2 development: How to measure L2 proficiency. In Current Trends in Child Second Language Acquisition: A Generative Perspective, Belma Haznedar and Elena Gavruseva (Eds.), 301\u2013333, John Benjamins Publishing Company."},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-012-0314-x"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASRU.2003.1318425"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2006.870737"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2004-327"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/2668956.2668958"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2008.52"}],"container-title":["ACM Transactions on Human-Robot Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3503799","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3503799","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:19Z","timestamp":1750186939000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3503799"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,4]]},"references-count":62,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,6,30]]}},"alternative-id":["10.1145\/3503799"],"URL":"https:\/\/doi.org\/10.1145\/3503799","relation":{},"ISSN":["2573-9522","2573-9522"],"issn-type":[{"type":"print","value":"2573-9522"},{"type":"electronic","value":"2573-9522"}],"subject":[],"published":{"date-parts":[[2022,3,4]]},"assertion":[{"value":"2020-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-03-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}