{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T07:02:07Z","timestamp":1773903727058,"version":"3.50.1"},"reference-count":71,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,3,18]],"date-time":"2021-03-18T00:00:00Z","timestamp":1616025600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,3,18]],"date-time":"2021-03-18T00:00:00Z","timestamp":1616025600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010661","name":"Horizon 2020 Framework Programme","doi-asserted-by":"publisher","award":["765955"],"award-info":[{"award-number":["765955"]}],"id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J of Soc Robotics"],"published-print":{"date-parts":[[2022,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In educational HRI, it is generally believed that a robots behavior has a direct effect on the engagement of a user with the robot, the task at hand and also their partner in case of a collaborative activity. Increasing this engagement is then held responsible for increased learning and productivity. The state of the art usually investigates the relationship between the behaviors of the robot and the engagement state of the user while assuming a linear relationship between engagement and the end goal: learning. However, is it correct to assume that to maximise learning, one needs to maximise engagement? Furthermore, conventional supervised models of engagement require human annotators to get labels. This is not only laborious but also introduces further subjectivity in an already subjective construct of engagement. Can we have machine-learning models for engagement detection where annotations do not rely on human annotators? Looking deeper at the behavioral patterns and the learning outcomes and a performance metric in a multi-modal data set collected in an educational human\u2013human\u2013robot setup with 68 students, we observe a hidden link that we term as Productive Engagement. We theorize a robot incorporating this knowledge will (1) distinguish teams based on engagement that is conducive of learning; and (2) adopt behaviors that eventually lead the users to increased learning by means of being productively engaged. Furthermore, this seminal link paves way for machine-learning models in educational HRI with automatic labelling based on the data.<\/jats:p>","DOI":"10.1007\/s12369-021-00766-w","type":"journal-article","created":{"date-parts":[[2021,3,18]],"date-time":"2021-03-18T19:12:30Z","timestamp":1616094750000},"page":"55-71","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["What if Social Robots Look for Productive Engagement?"],"prefix":"10.1007","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6203-1510","authenticated-orcid":false,"given":"Jauwairia","family":"Nasir","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Barbara","family":"Bruno","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohamed","family":"Chetouani","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pierre","family":"Dillenbourg","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,3,18]]},"reference":[{"issue":"1","key":"766_CR1","first-page":"1","volume":"1618","author":"N Alyuz","year":"2016","unstructured":"Alyuz N, Okur E, Oktay E, Genc U, Aslan S, Mete SE, Stanhill D, Arnrich B, Esme AA (2016) Towards an emotional engagement model: can affective states of a learner be automatically detected in a 1:1 learning scenario? CEUR Workshop Proc 1618(1):1\u20137","journal-title":"CEUR Workshop Proc"},{"key":"766_CR2","unstructured":"Amos B, Ludwiczuk B, Satyanarayanan M (2016) Openface: a general-purpose face recognition library with mobile applications. Technical report, CMU-CS-16-118. CMU School of Computer Science"},{"issue":"4","key":"766_CR3","doi-asserted-by":"publisher","first-page":"465","DOI":"10.1007\/s12369-015-0298-7","volume":"7","author":"SM Anzalone","year":"2015","unstructured":"Anzalone SM, Boucenna S, Ivaldi S, Chetouani M (2015) Evaluating the engagement with social robots. Int J Social Robot 7(4):465\u2013478. https:\/\/doi.org\/10.1007\/s12369-015-0298-7","journal-title":"Int J Social Robot"},{"key":"766_CR4","doi-asserted-by":"publisher","unstructured":"Baker R, Siemens G (2012) Educational data mining and learning analytics. In: Sawyer RK (ed) CHLS. Cambridge University Press, Cambridge, pp 253\u2013272. https:\/\/doi.org\/10.1017\/CBO9781139519526.016","DOI":"10.1017\/CBO9781139519526.016"},{"key":"766_CR5","doi-asserted-by":"publisher","unstructured":"Baker RS, Corbett AT, Koedinger KR, Wagner AZ (2004) Off-task behavior in the cognitive tutor classroom, pp 383\u2013390. https:\/\/doi.org\/10.1145\/985692.985741","DOI":"10.1145\/985692.985741"},{"key":"766_CR6","doi-asserted-by":"publisher","unstructured":"Baker RS, Corbett AT, Aleven V (2008) More accurate student modeling through contextual estimation of slip and guess probabilities in bayesian knowledge tracing. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) 5091 LNCS, pp 406\u2013415. https:\/\/doi.org\/10.1007\/978-3-540-69132-7-44","DOI":"10.1007\/978-3-540-69132-7-44"},{"key":"766_CR7","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0178126","author":"P Baxter","year":"2017","unstructured":"Baxter P, Ashurst E, Read R, Kennedy J, Belpaeme T (2017) Robot education peers in a situated primary school study: personalisation promotes child learning. PLoS ONE. https:\/\/doi.org\/10.1371\/journal.pone.0178126","journal-title":"PLoS ONE"},{"key":"766_CR8","unstructured":"Beal CR, Qu L, Lee H (2004) Basics of feedback control-elements of feedback control | instrumentation and control engineering, pp 151\u2013156"},{"issue":"21","key":"766_CR9","doi-asserted-by":"publisher","first-page":"5954","DOI":"10.1126\/scirobotics.aat5954","volume":"3","author":"T Belpaeme","year":"2018","unstructured":"Belpaeme T, Kennedy J, Ramachandran A, Scassellati B, Tanaka F (2018) Social robots for education: a review. Sci Robot 3(21):5954. https:\/\/doi.org\/10.1126\/scirobotics.aat5954","journal-title":"Sci Robot"},{"key":"766_CR10","doi-asserted-by":"publisher","unstructured":"Benitez-Quiroz CF, Srinivasan R, Martinez AM (2016) Emotionet: an accurate, real-time algorithm for the automatic annotation of a million facial expressions in the wild. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 5562\u20135570. https:\/\/doi.org\/10.1109\/CVPR.2016.600","DOI":"10.1109\/CVPR.2016.600"},{"key":"766_CR11","unstructured":"Benkaouar W, Vaufreydaz D (2012) Multi-sensors engagement detection with a robot companion in a home environment multi-sensors engagement detection with a robot companion in a home environment. Workshop on assistance and service robotics in a human environment, pp 45\u201352"},{"key":"766_CR12","unstructured":"Blaye A (1988) Confrontation socio-cognitive et r\u00e9solution de probl\u00e8mes. PhD thesis, Centre de Recherche en Psychologie Cognitive, Universit\u00e9 de Provence, 13261 Aix-en-Provence, France"},{"key":"766_CR13","doi-asserted-by":"publisher","unstructured":"Brown LV, Kerwin R, Howard AM (2013) Applying behavioral strategies for student engagement using a robotic educational agent. In: Proceedings\u20142013 IEEE international conference on systems, man, and cybernetics, SMC 2013, pp 4360\u20134365. https:\/\/doi.org\/10.1109\/SMC.2013.744","DOI":"10.1109\/SMC.2013.744"},{"key":"766_CR14","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1007\/s11257-009-9062-8","volume":"19","author":"C Conati","year":"2009","unstructured":"Conati C, Maclaren H (2009) Empirically building and evaluating a probabilistic model of user affect. User Model User Adap Inter 19:267\u2013303","journal-title":"User Model User Adap Inter"},{"key":"766_CR15","doi-asserted-by":"publisher","unstructured":"Castellano G, Pereira A, Leite I, Paiva A, Mcowan P (2009) Detecting user engagement with a robot companion using task and social interaction-based features, pp 119\u2013126. https:\/\/doi.org\/10.1145\/1647314.1647336","DOI":"10.1145\/1647314.1647336"},{"key":"766_CR16","doi-asserted-by":"publisher","unstructured":"Castellano G, Leite I, Pereira A, Martinho C, Paiva A, McOwan PW (2012) Detecting engagement in hri: an exploration of social and task-based context. In: Proceedings\u20142012 ASE\/IEEE international conference on privacy, security, risk and trust and 2012 ASE\/IEEE international conference on social computing, SocialCom\/PASSAT 2012, pp 421\u2013428. https:\/\/doi.org\/10.1109\/SocialCom-PASSAT.2012.51","DOI":"10.1109\/SocialCom-PASSAT.2012.51"},{"issue":"2","key":"766_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2622615","volume":"4","author":"G Castellano","year":"2014","unstructured":"Castellano G, Leite I, Pereira A, Martinho C, Paiva A, Mcowan PW (2014) Context-sensitive affect recognition for a robotic game companion. ACM Trans Interact Intell Syst 4(2):1\u201325. https:\/\/doi.org\/10.1145\/2622615","journal-title":"ACM Trans Interact Intell Syst"},{"key":"766_CR18","unstructured":"Chaouachi M, Chalfoun P, Jraidi I, Frasson C (2010) Affect and mental engagement: towards adaptability for intelligent systems. In: Proceedings of the twenty-third international Florida artificial intelligence research society conference (FLAIRS), pp 355\u2013360"},{"issue":"4","key":"766_CR19","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1080\/00461520.2014.965823","volume":"49","author":"MT Chi","year":"2014","unstructured":"Chi MT, Wylie R (2014) The ICAP framework: linking cognitive engagement to active learning outcomes. Educ Psychol 49(4):219\u2013243. https:\/\/doi.org\/10.1080\/00461520.2014.965823","journal-title":"Educ Psychol"},{"key":"766_CR20","doi-asserted-by":"publisher","unstructured":"Cocea M, Weibelzahl S (2009) Log file analysis for disengagement detection in e-Learning environments, vol\u00a019. https:\/\/doi.org\/10.1007\/s11257-009-9065-5","DOI":"10.1007\/s11257-009-9065-5"},{"key":"766_CR21","doi-asserted-by":"crossref","unstructured":"Corbett AT, Anderson JR (1995) Knowledge tracing: modeling the acquisition of student knowledge","DOI":"10.1007\/BF01099821"},{"issue":"13","key":"766_CR22","first-page":"1","volume":"13","author":"LJ Corrigan","year":"2013","unstructured":"Corrigan LJ, Peters C, Castellano G (2013) Social-task engagement: striking a balance between the robot and the task. Embodied Commun Goals Intentions Work ICSR 13(13):1\u20137","journal-title":"Embodied Commun Goals Intentions Work ICSR"},{"issue":"1\u20132","key":"766_CR23","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1007\/s11257-007-9037-6","volume":"18","author":"SD Craig","year":"2007","unstructured":"Craig SD, Witherspoon A, D\u2019Mello SK, Graesser A, McDaniel B (2007) Automatic detection of learner\u2019s affect from conversational cues. User Model User Adap Inter 18(1\u20132):45\u201380. https:\/\/doi.org\/10.1007\/s11257-007-9037-6","journal-title":"User Model User Adap Inter"},{"key":"766_CR24","doi-asserted-by":"publisher","unstructured":"Deci E (2017) Intrinsic motivation and self-determination. https:\/\/doi.org\/10.1016\/B978-0-12-809324-5.05613-3","DOI":"10.1016\/B978-0-12-809324-5.05613-3"},{"issue":"1\u20132","key":"766_CR25","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1007\/s11257-011-9106-8","volume":"22","author":"MC Desmarais","year":"2012","unstructured":"Desmarais MC, Baker RS (2012) A review of recent advances in learner and skill modeling in intelligent learning environments. User Model User Adap Inter 22(1\u20132):9\u201338. https:\/\/doi.org\/10.1007\/s11257-011-9106-8","journal-title":"User Model User Adap Inter"},{"issue":"1","key":"766_CR26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40561-018-0080-z","volume":"6","author":"MAA Dewan","year":"2019","unstructured":"Dewan MAA, Murshed M, Lin F (2019) Engagement detection in online learning: a review. Smart Learn Environ 6(1):1\u201320. https:\/\/doi.org\/10.1186\/s40561-018-0080-z","journal-title":"Smart Learn Environ"},{"key":"766_CR27","first-page":"189","volume-title":"Learning in humans and machines: towards an interdisciplinary learning science","author":"P Dillenbourg","year":"1996","unstructured":"Dillenbourg P, Baker M, Blaye A, O\u2019Malley C (1996) The evolution of research on collaborative learning. In: Spada H, Reimann P (eds) Learning in humans and machines: towards an interdisciplinary learning science. Elsevier, Oxford, pp 189\u2013211"},{"issue":"2","key":"766_CR28","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1016\/j.learninstruc.2011.10.001","volume":"22","author":"S D\u2019Mello","year":"2012","unstructured":"D\u2019Mello S, Graesser A (2012) Dynamics of affective states during complex learning. Learn Instruct 22(2):145\u2013157. https:\/\/doi.org\/10.1016\/j.learninstruc.2011.10.001","journal-title":"Learn Instruct"},{"key":"766_CR29","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1016\/j.learninstruc.2012.05.003","volume":"29","author":"S D\u2019Mello","year":"2014","unstructured":"D\u2019Mello S, Lehman B, Pekrun R, Graesser A (2014) Confusion can be beneficial for learning. Learn Instruct 29:153\u2013170. https:\/\/doi.org\/10.1016\/j.learninstruc.2012.05.003","journal-title":"Learn Instruct"},{"key":"766_CR30","doi-asserted-by":"publisher","unstructured":"Do-lenh S (2012) Supporting reflection and classroom orchestration with tangible tabletops 5313:241. https:\/\/doi.org\/10.5075\/epfl-thesis-5313","DOI":"10.5075\/epfl-thesis-5313"},{"issue":"5","key":"766_CR31","doi-asserted-by":"publisher","first-page":"659","DOI":"10.1007\/s12369-017-0414-y","volume":"9","author":"ME Foster","year":"2017","unstructured":"Foster ME, Gaschler A, Giuliani M (2017) Automatically classifying user engagement for dynamic multi-party human\u2013robot interaction. Int J Social Robot 9(5):659\u2013674. https:\/\/doi.org\/10.1007\/s12369-017-0414-y","journal-title":"Int J Social Robot"},{"key":"766_CR32","doi-asserted-by":"crossref","unstructured":"Gatica-Perez D, McCowan L, Zhang D, Bengio S (2005) Detecting group interest-level in meetings. In: Proceedings (ICASSP\u201905). IEEE international conference on acoustics, speech, and signal processing, vol\u00a01. IEEE, pp I\u2013489","DOI":"10.1109\/ICASSP.2005.1415157"},{"key":"766_CR33","unstructured":"Glachan M, Light P (1982) Peer interaction and learning: can two wrongs make a right. In: Social cognition: studies of the development of understanding, vol\u00a02 in developing body and mind. Harvester Press, pp 238\u2013262"},{"key":"766_CR34","doi-asserted-by":"crossref","unstructured":"Gordon G, Spaulding S, Westlund JK, Lee JJ, Plummer L, Martinez M, Das M, Breazeal C (2016) Affective personalization of a social robot tutor for children\u2019s second language skills. In: Proceedings of the 30th conference on artificial intelligence (AAAI 2016), vol 2011, pp 3951\u20133957","DOI":"10.1609\/aaai.v30i1.9914"},{"key":"766_CR35","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1016\/j.compedu.2015.09.005","volume":"90","author":"CR Henrie","year":"2015","unstructured":"Henrie CR, Halverson LR, Graham CR (2015) Measuring student engagement in technology-mediated learning: a review. Comput Educ 90:36\u201353. https:\/\/doi.org\/10.1016\/j.compedu.2015.09.005","journal-title":"Comput Educ"},{"key":"766_CR36","doi-asserted-by":"publisher","unstructured":"Huang CM, Mutlu B (2014) Learning-based modeling of multimodal behaviors for humanlike robots, pp 57\u201364. https:\/\/doi.org\/10.1145\/2559636.2559668","DOI":"10.1145\/2559636.2559668"},{"key":"766_CR37","doi-asserted-by":"publisher","unstructured":"Ishii R, Nakano YI (2010) An empirical study of eye-gaze behaviors. In: Proceedings of the 2010 workshop on eye gaze in intelligent human machine interaction\u2014EGIHMI \u201910, pp 33\u201340. https:\/\/doi.org\/10.1145\/2002333.2002339","DOI":"10.1145\/2002333.2002339"},{"key":"766_CR38","unstructured":"Ishii R, Shinohara Y, Nakano I, Nishida T (2011) Combining multiple types of eye-gaze information to predict user\u2019s conversational engagement. Hum Factors"},{"key":"766_CR39","doi-asserted-by":"publisher","unstructured":"Kapoor A, Picard RW (2006) Multimodal affect recognition in learning environments, p 677. https:\/\/doi.org\/10.1145\/1101149.1101300","DOI":"10.1145\/1101149.1101300"},{"issue":"3","key":"766_CR40","doi-asserted-by":"publisher","first-page":"379","DOI":"10.1080\/07370000802212669","volume":"26","author":"M Kapur","year":"2008","unstructured":"Kapur M (2008) Productive failure. Cognit Instruct 26(3):379\u2013424. https:\/\/doi.org\/10.1080\/07370000802212669","journal-title":"Cognit Instruct"},{"key":"766_CR41","doi-asserted-by":"publisher","unstructured":"Kim J, Co H, Truong K, Evers V, Truong KP (2016) Automatic detection of children\u2019s engagement using non-verbal features and ordinal learning expressive agents for symbiotic education and learning (EASEL) view project squirrel (clearing clutter bit by bit) view project automatic detection of children\u2019s engagement using non-verbal features and ordinal learning. https:\/\/doi.org\/10.21437\/WOCCI.2016-5","DOI":"10.21437\/WOCCI.2016-5"},{"issue":"6","key":"766_CR42","doi-asserted-by":"publisher","first-page":"99194","DOI":"10.1371\/journal.pone.0099194","volume":"9","author":"S Korb","year":"2014","unstructured":"Korb S, With S, Niedenthal P, Kaiser Wehrle S, Grandjean DM (2014) The perception and mimicry of facial movements predict judgments of smile authenticity. PLoS ONE 9(6):99194","journal-title":"PLoS ONE"},{"issue":"5","key":"766_CR43","doi-asserted-by":"publisher","first-page":"991","DOI":"10.1109\/TRO.2007.904899","volume":"23","author":"D Kul\u00edc","year":"2007","unstructured":"Kul\u00edc D, Croft E (2007) Affective state estimation for human\u2013robot interaction. IEEE Trans Rob 23(5):991\u20131000. https:\/\/doi.org\/10.1109\/TRO.2007.904899","journal-title":"IEEE Trans Rob"},{"key":"766_CR44","doi-asserted-by":"crossref","unstructured":"Maslow A (1943) A theory of human motivation 13:370\u2013396","DOI":"10.1037\/h0054346"},{"key":"766_CR45","doi-asserted-by":"publisher","unstructured":"Nasir J, Norman U, Bruno B, Chetouani M, Dillenbourg P (2020a) PE-HRI: a multimodal dataset for the study of productive engagement in a robot mediated collaborative educational setting. https:\/\/doi.org\/10.5281\/zenodo.4288833","DOI":"10.5281\/zenodo.4288833"},{"key":"766_CR46","doi-asserted-by":"publisher","unstructured":"Nasir J, Norman U, Bruno B, Dillenbourg P (2020b) When positive perception of the robot has no effect on learning. In: 2020 29th IEEE international conference on robot and human interactive communication (RO-MAN), pp 313\u2013320. https:\/\/doi.org\/10.1109\/RO-MAN47096.2020.9223343","DOI":"10.1109\/RO-MAN47096.2020.9223343"},{"key":"766_CR47","unstructured":"Nezami OM, Hamey L, Richards D, Dras M (2018) Engagement recognition using deep learning and facial expression 2013"},{"key":"766_CR48","doi-asserted-by":"publisher","unstructured":"O\u2019Brien H, Freund L, Kopak R (2016) Reading environments. In Proceedings of the 2016 ACM on conference on human information interaction and retrieval, pp 71\u201380. https:\/\/doi.org\/10.1145\/2854946.2854973","DOI":"10.1145\/2854946.2854973"},{"key":"766_CR49","doi-asserted-by":"publisher","first-page":"938","DOI":"10.1002\/asi.20801","volume":"59","author":"HL O\u2019Brien","year":"2008","unstructured":"O\u2019Brien HL, Toms E (2008) What is user engagement? A conceptual framework for defining user engagement with technology. JASIST 59:938\u2013955","journal-title":"JASIST"},{"key":"766_CR50","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1002\/asi.21229","volume":"61","author":"HL O\u2019Brien","year":"2010","unstructured":"O\u2019Brien HL, Toms E (2010) The development and evaluation of a survey to measure user engagement. JASIST 61:50\u201369","journal-title":"JASIST"},{"key":"766_CR51","doi-asserted-by":"crossref","unstructured":"Oertel C, Scherer S, Campbell N (2011) On the use of multimodal cues for the prediction of degrees of involvement in spontaneous conversation. In: Twelfth annual conference of the international speech communication association","DOI":"10.21437\/Interspeech.2011-465"},{"key":"766_CR52","doi-asserted-by":"publisher","first-page":"92","DOI":"10.3389\/frobt.2020.00092","volume":"7","author":"C Oertel","year":"2020","unstructured":"Oertel C, Castellano G, Chetouani M, Nasir J, Obaid M, Pelachaud C, Peters C (2020) Engagement in human\u2013agent interaction?: An overview. Front Robot AI 7:92. https:\/\/doi.org\/10.3389\/frobt.2020.00092","journal-title":"Front Robot AI"},{"key":"766_CR53","doi-asserted-by":"crossref","unstructured":"Oggi O, Rudovic, Park HW, Busche J, Schuller B, Breazeal C, Picard RW (2019) Personalized estimation of engagement from videos using active learning with deep reinforcement learning","DOI":"10.1109\/CVPRW.2019.00031"},{"key":"766_CR54","doi-asserted-by":"publisher","unstructured":"Pardos ZA, Heffernan NT (2010) Modeling individualization in a Bayesian networks implementation of knowledge tracing. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) 6075 LNCS, pp 255\u2013266. https:\/\/doi.org\/10.1007\/978-3-642-13470-8_24","DOI":"10.1007\/978-3-642-13470-8_24"},{"key":"766_CR55","unstructured":"Parsons J, Leah T (2011) Student engagement: what do we know and what should we do? University of Alberta, Technical report"},{"key":"766_CR56","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825\u20132830","journal-title":"J Mach Learn Res"},{"key":"766_CR57","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2020.2980275","author":"G Perugia","year":"2020","unstructured":"Perugia G, Boladeras M, Catal\u00e0 BE, Rauterberg M (2020) Engage-dem: a model of engagement of people with dementia. IEEE Trans Affect Comput. https:\/\/doi.org\/10.1109\/TAFFC.2020.2980275","journal-title":"IEEE Trans Affect Comput"},{"key":"766_CR58","unstructured":"Poggi I (2007) Mind, hands, face and body: a goal and belief view of multimodal communication. No. v. 19 = v. 19 in K\u00f6rper, Zeichen, Kultur ; Body, sign, culture, Weidler, Berlin, oCLC: ocn143609341"},{"issue":"1","key":"766_CR59","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3213768","volume":"9","author":"A Ramachandran","year":"2019","unstructured":"Ramachandran A, Huang CM, Scassellati B (2019) Toward effective robot\u2013child tutoring: internal motivation, behavioral intervention and learning outcomes. ACM Trans Interact Intell Syst 9(1):1\u201323. https:\/\/doi.org\/10.1145\/3213768","journal-title":"ACM Trans Interact Intell Syst"},{"key":"766_CR60","doi-asserted-by":"publisher","unstructured":"Rich C, Ponsler B, Holroyd A, Sidner CL (2010) Recognizing engagement in human\u2013robot interaction. In: 5th ACM\/IEEE International conference on human\u2013robot interaction (HRI), pp 375\u2013382. https:\/\/doi.org\/10.1109\/HRI.2010.5453163","DOI":"10.1109\/HRI.2010.5453163"},{"key":"766_CR61","doi-asserted-by":"crossref","unstructured":"Rudovic O, Zhang M, Schuller B, Picard R (2019) Multi-modal active learning from human data: A deep reinforcement learning approach. In: 2019 International conference on multimodal interaction. ACM, New York, pp 6\u201315","DOI":"10.1145\/3340555.3353742"},{"key":"766_CR62","doi-asserted-by":"publisher","unstructured":"Salam H, Chetouani M (2015) Engagement detection based on mutli-party cues for human robot interaction. In: International conference on affective computing and intelligent interaction, ACII 2015, pp 341\u2013347. https:\/\/doi.org\/10.1109\/ACII.2015.7344593","DOI":"10.1109\/ACII.2015.7344593"},{"key":"766_CR63","doi-asserted-by":"publisher","unstructured":"Salam H, Chetouani M (2015) A multi-level context-based modeling of engagement in human\u2013robot interaction. In: 11th IEEE international conference and workshops on automatic face and gesture recognition (FG), vol\u00a003, pp 1\u20136. https:\/\/doi.org\/10.1109\/FG.2015.7284845","DOI":"10.1109\/FG.2015.7284845"},{"key":"766_CR64","doi-asserted-by":"publisher","first-page":"705","DOI":"10.1109\/ACCESS.2016.2614525","volume":"5","author":"H Salam","year":"2017","unstructured":"Salam H, Celiktutan O, Hupont I, Gunes H, Chetouani M (2017) Fully automatic analysis of engagement and its relationship to personality in human\u2013robot interactions. IEEE Access 5:705\u2013721","journal-title":"IEEE Access"},{"key":"766_CR65","doi-asserted-by":"publisher","unstructured":"Sanghvi J, Castellano G, Leite I, Pereira A, McOwan PW, Paiva A (2011) Automatic analysis of affective postures and body motion to detect engagement with a game companion. In:Proceedings of the 6th international conference on Human\u2013robot interaction-HRI \u201911, p 305. https:\/\/doi.org\/10.1145\/1957656.1957781","DOI":"10.1145\/1957656.1957781"},{"issue":"4","key":"766_CR66","doi-asserted-by":"publisher","first-page":"461","DOI":"10.1207\/S1532690XCI1804_2","volume":"18","author":"BB Schwarz","year":"2000","unstructured":"Schwarz BB, Neuman Y, Biezuner S (2000) Two wrongs may make a right... if they argue together!. Cognit Instruct 18(4):461\u2013494. https:\/\/doi.org\/10.1207\/S1532690XCI1804_2","journal-title":"Cognit Instruct"},{"key":"766_CR67","doi-asserted-by":"publisher","unstructured":"Sharma K, Papamitsiou Z, Olsen J, Giannakos M (2020) Predicting learners\u2019 effortful behaviour in adaptive assessment using multimodal data. https:\/\/doi.org\/10.1145\/3375462.3375498","DOI":"10.1145\/3375462.3375498"},{"issue":"1\u20132","key":"766_CR68","doi-asserted-by":"publisher","first-page":"140","DOI":"10.1016\/j.artint.2005.03.005","volume":"166","author":"CL Sidner","year":"2005","unstructured":"Sidner CL, Lee C, Kidd CD, Lesh N, Rich C (2005) Explorations in engagement for humans and robots. Artif Intell 166(1\u20132):140\u2013164. https:\/\/doi.org\/10.1016\/j.artint.2005.03.005","journal-title":"Artif Intell"},{"key":"766_CR69","doi-asserted-by":"publisher","unstructured":"Szafir D, Mutlu B (2012) Pay attention! designing adaptive agents that monitor and improve user engagement. In: Conference on human factors in computing systems (CHI). https:\/\/doi.org\/10.1145\/2207676.2207679","DOI":"10.1145\/2207676.2207679"},{"issue":"1","key":"766_CR70","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1109\/TAFFC.2014.2316163","volume":"5","author":"J Whitehill","year":"2014","unstructured":"Whitehill J, Serpell Z, Lin YC, Foster A, Movellan JR (2014) The faces of engagement: automatic recognition of student engagement from facial expressions. IEEE Trans Affect Comput 5(1):86\u201398. https:\/\/doi.org\/10.1109\/TAFFC.2014.2316163","journal-title":"IEEE Trans Affect Comput"},{"issue":"3","key":"766_CR71","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1016\/S1041-6080(96)90015-1","volume":"8","author":"CA Wolters","year":"1996","unstructured":"Wolters CA, Yu SL, Pintrich PR (1996) The relation between goal orientation and students\u2019 motivational beliefs and self-regulated learning. Learn Individual Differ 8(3):211\u2013238. https:\/\/doi.org\/10.1016\/S1041-6080(96)90015-1","journal-title":"Learn Individual Differ"}],"container-title":["International Journal of Social Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12369-021-00766-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12369-021-00766-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12369-021-00766-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T22:22:00Z","timestamp":1671661320000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12369-021-00766-w"}},"subtitle":["Automated Assessment of Goal-Centric Engagement in Learning Applications"],"short-title":[],"issued":{"date-parts":[[2021,3,18]]},"references-count":71,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,1]]}},"alternative-id":["766"],"URL":"https:\/\/doi.org\/10.1007\/s12369-021-00766-w","relation":{},"ISSN":["1875-4791","1875-4805"],"issn-type":[{"value":"1875-4791","type":"print"},{"value":"1875-4805","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,18]]},"assertion":[{"value":"12 February 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 March 2021","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"We have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Code availability"}}]}}