{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T10:20:42Z","timestamp":1779358842773,"version":"3.51.4"},"reference-count":39,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,7,17]],"date-time":"2024-07-17T00:00:00Z","timestamp":1721174400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p><jats:bold>Introduction:<\/jats:bold> In this work we explore a potential approach to improve human-robot collaboration experience by adapting cobot behavior based on natural cues from the operator.<\/jats:p><jats:p><jats:bold>Methods:<\/jats:bold> Inspired by the literature on human-human interactions, we conducted a wizard-of-oz study to examine whether a gaze towards the cobot can serve as a trigger for initiating joint activities in collaborative sessions. In this study, 37 participants engaged in an assembly task while their gaze behavior was analyzed. We employed a gaze-based attention recognition model to identify when the participants look at the cobot.<\/jats:p><jats:p><jats:bold>Results:<\/jats:bold> Our results indicate that in most cases (83.74%), the joint activity is preceded by a gaze towards the cobot. Furthermore, during the entire assembly cycle, the participants tend to look at the cobot mostly around the time of the joint activity. Given the above results, a fully integrated system triggering joint action only when the gaze is directed towards the cobot was piloted with 10 volunteers, of which one characterized by high-functioning Autism Spectrum Disorder. Even though they had never interacted with the robot and did not know about the gaze-based triggering system, most of them successfully collaborated with the cobot and reported a smooth and natural interaction experience.<\/jats:p><jats:p><jats:bold>Discussion:<\/jats:bold> To the best of our knowledge, this is the first study to analyze the natural gaze behavior of participants working on a joint activity with a robot during a collaborative assembly task and to attempt the full integration of an automated gaze-based triggering system.<\/jats:p>","DOI":"10.3389\/frobt.2024.1394379","type":"journal-article","created":{"date-parts":[[2024,7,17]],"date-time":"2024-07-17T04:10:16Z","timestamp":1721189416000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Gaze detection as a social cue to initiate natural human-robot collaboration in an assembly task"],"prefix":"10.3389","volume":"11","author":[{"given":"Matteo","family":"Lavit Nicora","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pooja","family":"Prajod","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marta","family":"Mondellini","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Giovanni","family":"Tauro","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rocco","family":"Vertechy","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elisabeth","family":"Andr\u00e9","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matteo","family":"Malosio","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2024,7,17]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"25","DOI":"10.5898\/jhri.6.1.admoni","article-title":"Social eye gaze in human-robot interaction: a review","volume":"6","author":"Admoni","year":"2017","journal-title":"J. Human-Robot Interact."},{"key":"B2","doi-asserted-by":"publisher","first-page":"848","DOI":"10.1017\/s0007125000073980","article-title":"Gaze and mutual gaze","volume":"165","author":"Argyle","year":"1994","journal-title":"Br. J. Psychiatry"},{"key":"B3","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1007\/978-3-319-02714-2_14","article-title":"Nova: automated analysis of nonverbal signals in social interactions","volume-title":"Human behavior understanding: 4th international workshop, HBU 2013, barcelona, Spain, october 22, 2013. Proceedings 4","author":"Baur","year":"2013"},{"key":"B4","volume-title":"Blazeface: sub-millisecond neural face detection on mobile gpus","author":"Bazarevsky","year":"2019"},{"key":"B5","doi-asserted-by":"publisher","first-page":"3","DOI":"10.3389\/fnbot.2012.00003","article-title":"I reach faster when i see you look: gaze effects in human\u2013human and human\u2013robot face-to-face cooperation","volume":"6","author":"Boucher","year":"2012","journal-title":"Front. neurorobotics"},{"key":"B6","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1007\/bf01000738","article-title":"Body movement and emphasis in speech","volume":"9","author":"Bull","year":"1985","journal-title":"J. nonverbal Behav."},{"key":"B7","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1002\/edp.2430010106","article-title":"Mind knowledge in infancy: the emergence of intentional communication","volume":"1","author":"Camaioni","year":"1992","journal-title":"Early Dev. Parent."},{"key":"B8","doi-asserted-by":"publisher","first-page":"269","DOI":"10.2307\/3033565","article-title":"The role of gaze in the initiation of conversation","volume":"41","author":"Cary","year":"1978","journal-title":"Soc. Psychol."},{"key":"B9","doi-asserted-by":"publisher","first-page":"456","DOI":"10.21105\/joss.00456","article-title":"ros_control: a generic and simple control framework for ros","volume":"2","author":"Chitta","year":"2017","journal-title":"J. Open Source Softw."},{"key":"B10","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1145\/3029798.3038325","article-title":"How to describe interaction with a collaborative robot","volume-title":"Proceedings of the companion of the 2017 ACM\/IEEE international conference on human-robot interaction","author":"Christiernin","year":"2017"},{"key":"B11","article-title":"Reducing the barrier to entry of complex robotic software: a moveit! case study","author":"Coleman","year":"2014"},{"key":"B12","article-title":"A biological inspired cognitive model of multi-sensory joint attention in human robot collaborative tasks","author":"Eldardeer","year":"2020"},{"key":"B13","doi-asserted-by":"publisher","first-page":"e15855","DOI":"10.1371\/journal.pone.0015855","article-title":"Social requests and social affordances: how they affect the kinematics of motor sequences during interactions between conspecifics","volume":"6","author":"Ferri","year":"2011","journal-title":"PloS one"},{"key":"B14","first-page":"3","article-title":"Visual SceneMaker: a tool for authoring interactive virtual characters","volume-title":"Special issue of the journal of multimodal user interfaces: interacting with embodied conversational agents","author":"Gebhard","year":"2012"},{"key":"B15","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1109\/tpami.2023.3321337","article-title":"Automatic gaze analysis: a survey of deep learning based approaches","volume":"46","author":"Ghosh","year":"2023","journal-title":"IEEE Trans. Pattern Analysis Mach. Intell."},{"key":"B16","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1016\/0167-9457(83)90004-0","article-title":"Kinematics of head movements accompanying speech during conversation","volume":"2","author":"Hadar","year":"1983","journal-title":"Hum. Mov. Sci."},{"key":"B17","doi-asserted-by":"publisher","first-page":"20150080","DOI":"10.1098\/rstb.2015.0080","article-title":"Gazing at me: the importance of social meaning in understanding direct-gaze cues","volume":"371","author":"Hamilton","year":"2016","journal-title":"Philosophical Trans. R. Soc. B Biol. Sci."},{"key":"B18","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1109\/HRI.2016.7451737","article-title":"Anticipatory robot control for efficient human-robot collaboration","volume-title":"2016 11th ACM\/IEEE international conference on human-robot interaction (HRI)","author":"Huang","year":"2016"},{"key":"B19","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1111\/j.1744-6570.2010.01175.x","article-title":"Psychological and physiological reactions to high workloads: implications for well-being","volume":"63","author":"Ilies","year":"2010","journal-title":"Pers. Psychol."},{"key":"B20","doi-asserted-by":"publisher","first-page":"e36390","DOI":"10.1371\/journal.pone.0036390","article-title":"Gaze direction and request gesture in social interactions","volume":"7","author":"Innocenti","year":"2012","journal-title":"PLoS one"},{"key":"B21","first-page":"22","article-title":"Development of a manufacturing system for gear assembly using collaborative robots","author":"Li","year":"2021"},{"key":"B22","first-page":"247","article-title":"Exploring a model of gaze for grounding in multimodal HRI","volume-title":"Proceedings of the 16th international conference on multimodal interaction, ICMI 2014","author":"Mehlmann","year":"2014"},{"key":"B23","doi-asserted-by":"crossref","DOI":"10.1109\/ICHMS59971.2024.10555621","article-title":"Exploring the dynamics between cobot\u2019s production rhythm, locus of control and emotional state in a collaborative assembly scenario","author":"Mondellini","year":"2024"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1245857","DOI":"10.3389\/fpsyg.2023.1245857","article-title":"Behavioral patterns in robotic collaborative assembly: comparing neurotypical and autism spectrum disorder participants","volume":"14","author":"Mondellini","year":"2023","journal-title":"Front. Psychol."},{"key":"B25","first-page":"285","article-title":"A human-driven control architecture for promoting good mental health in collaborative robot scenarios","volume-title":"2021 30th IEEE international conference on robot and human interactive communication","author":"Nicora","year":"2021"},{"key":"B26","first-page":"5048","article-title":"Robot reading human gaze: why eye tracking is better than head tracking for human-robot collaboration","author":"Palinko","year":"2016"},{"key":"B27","doi-asserted-by":"crossref","DOI":"10.1145\/3594806.3594819","article-title":"Gaze-based attention recognition for human-robot collaboration","author":"Prajod","year":"2023"},{"key":"B28","volume-title":"Mindbot planetary gearbox","author":"Redaelli","year":"2021"},{"key":"B29","doi-asserted-by":"publisher","first-page":"607","DOI":"10.1007\/s12369-017-0421-z","article-title":"Assistive robotic technology to combat social isolation in acute hospital settings","volume":"10","author":"Sarabia","year":"2018","journal-title":"Int. J. Soc. Robotics"},{"key":"B30","first-page":"8615","volume-title":"Human gaze following for human-robot interaction","author":"Saran","year":"2018"},{"key":"B31","doi-asserted-by":"crossref","first-page":"1025","DOI":"10.1109\/HRI53351.2022.9889440","article-title":"It\u2019s not all bad-worker perceptions of industrial robots","volume-title":"2022 17th ACM\/IEEE international conference on human-robot interaction (HRI)","author":"Schneiders","year":"2022"},{"key":"B32","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1016\/j.tics.2008.11.009","article-title":"The eye contact effect: mechanisms and development","volume":"13","author":"Senju","year":"2009","journal-title":"Trends cognitive Sci."},{"key":"B33","doi-asserted-by":"publisher","first-page":"68","DOI":"10.3390\/robotics10020068","article-title":"Gazeemd: detecting visual intention in gaze-based human-robot interaction","volume":"10","author":"Shi","year":"2021","journal-title":"Robotics"},{"key":"B34","article-title":"Robotic operating system","year":"2018"},{"key":"B35","doi-asserted-by":"publisher","first-page":"348","DOI":"10.1037\/a0036821","article-title":"Boredom at work: proximal and distal consequences of affective work-related boredom","volume":"19","author":"van Hooff","year":"2014","journal-title":"J. Occup. health Psychol."},{"key":"B36","first-page":"831","article-title":"The social signal interpretation (ssi) framework: multimodal signal processing and recognition in real-time","author":"Wagner","year":"2013"},{"key":"B37","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1109\/THMS.2021.3092684","article-title":"Cobots in industry 4.0: a roadmap for future practice studies on human\u2013robot collaboration","volume":"51","author":"Weiss","year":"2021","journal-title":"IEEE Trans. Human-Machine Syst."},{"key":"B38","doi-asserted-by":"publisher","first-page":"530","DOI":"10.1016\/j.jmsy.2021.10.006","article-title":"Industry 4.0 and industry 5.0\u2014inception, conception and perception","volume":"61","author":"Xu","year":"2021","journal-title":"J. Manuf. Syst."},{"key":"B39","first-page":"365","article-title":"Eth-xgaze: a large scale dataset for gaze estimation under extreme head pose and gaze variation","volume-title":"European conference on computer vision","author":"Zhang","year":"2020"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1394379\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,17]],"date-time":"2024-07-17T04:10:24Z","timestamp":1721189424000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1394379\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,17]]},"references-count":39,"alternative-id":["10.3389\/frobt.2024.1394379"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2024.1394379","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,17]]},"article-number":"1394379"}}