{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T08:58:24Z","timestamp":1773392304614,"version":"3.50.1"},"reference-count":37,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T00:00:00Z","timestamp":1729123200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>This research report introduces a learning system designed to detect the object that humans are gazing at, using solely visual feedback. By incorporating face detection, human attention prediction, and online object detection, the system enables the robot to perceive and interpret human gaze accurately, thereby facilitating the establishment of joint attention with human partners. Additionally, a novel dataset collected with the humanoid robot iCub is introduced, comprising more than 22,000 images from ten participants gazing at different annotated objects. This dataset serves as a benchmark for human gaze estimation in table-top human\u2013robot interaction (HRI) contexts. In this work, we use it to assess the proposed pipeline\u2019s performance and examine each component\u2019s effectiveness. Furthermore, the developed system is deployed on the iCub and showcases its functionality. The results demonstrate the potential of the proposed approach as a first step to enhancing social awareness and responsiveness in social robotics. This advancement can enhance assistance and support in collaborative scenarios, promoting more efficient human\u2013robot collaborations.<\/jats:p>","DOI":"10.3389\/frobt.2024.1346714","type":"journal-article","created":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T04:10:53Z","timestamp":1729138253000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["A pipeline for estimating human attention toward objects with on-board cameras on the iCub humanoid robot"],"prefix":"10.3389","volume":"11","author":[{"given":"Shiva","family":"Hanifi","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elisa","family":"Maiettini","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maria","family":"Lombardi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lorenzo","family":"Natale","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2024,10,17]]},"reference":[{"key":"B1","article-title":"Predicting user intent through eye gaze for shared autonomy","volume-title":"2016 AAAI fall symposium series","author":"Admoni","year":"2016"},{"key":"B2","doi-asserted-by":"publisher","first-page":"944","DOI":"10.3390\/s22030944","article-title":"A systematic review of research on robot-assisted therapy for children with autism","volume":"22","author":"Alabdulkareem","year":"2022","journal-title":"Sensors"},{"key":"B3","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1016\/s1364-6613(00)01501-1","article-title":"Social perception from visual cues: role of the sts region","volume":"4","author":"Allison","year":"2000","journal-title":"Trends cognitive Sci."},{"key":"B4","volume-title":"One eye is all you need: lightweight ensembles for gaze estimation with single encoders","author":"Athavale","year":"2022"},{"key":"B5","doi-asserted-by":"publisher","first-page":"1485","DOI":"10.1007\/s12369-020-00730-0","article-title":"Small talk with a robot? the impact of dialog content, talk initiative, and gaze behavior of a social robot on trust, acceptance, and proximity","volume":"13","author":"Babel","year":"2021","journal-title":"Int. J. Soc. Robotics"},{"key":"B6","volume-title":"First person action-object detection with egonet","author":"Bertasius","year":"2016"},{"key":"B7","doi-asserted-by":"publisher","first-page":"e3151","DOI":"10.2196\/rehab.3151","article-title":"Therapist: towards an autonomous socially interactive robot for motor and neurorehabilitation therapies for children","volume":"1","author":"Calderita","year":"2014","journal-title":"JMIR rehabilitation assistive Technol."},{"key":"B8","first-page":"510","article-title":"The ycb object and model set: towards common benchmarks for manipulation research","author":"Calli","year":"2015"},{"key":"B9","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1109\/tpami.2019.2929257","article-title":"Openpose: realtime multi-person 2d pose estimation using part affinity fields","volume":"43","author":"Cao","year":"2019","journal-title":"IEEE Trans. pattern analysis Mach. Intell."},{"key":"B10","first-page":"7291","article-title":"Realtime multi-person 2d pose estimation using part affinity fields","author":"Cao","year":"2017"},{"key":"B11","first-page":"13581","article-title":"Fast object segmentation learning with kernel-based methods for robotics","author":"Ceola","year":"2021"},{"key":"B12","doi-asserted-by":"publisher","first-page":"5259","DOI":"10.1109\/tip.2020.2982828","article-title":"Gaze estimation by exploring two-eye asymmetry","volume":"29","author":"Cheng","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"B13","doi-asserted-by":"publisher","first-page":"217","DOI":"10.3758\/s13423-019-01689-4","article-title":"Examining joint attention with the use of humanoid robots-a new approach to study fundamental mechanisms of social cognition","volume":"27","author":"Chevalier","year":"2020","journal-title":"Psychonomic Bull. and Rev."},{"key":"B14","first-page":"5396","article-title":"Detecting attended visual targets in video","author":"Chong","year":"2020"},{"key":"B15","first-page":"77","article-title":"Head and gaze dynamics in visual attention and context learning","author":"Doshi","year":"2009"},{"key":"B16","first-page":"334","article-title":"Rt-gene: real-time eye gaze estimation in natural environments","author":"Fischer","year":"2018"},{"key":"B17","doi-asserted-by":"publisher","first-page":"401","DOI":"10.1016\/j.jvcir.2017.10.004","article-title":"Next-active-object prediction from egocentric videos","volume":"49","author":"Furnari","year":"2017","journal-title":"J. Vis. Commun. Image Represent."},{"key":"B18","first-page":"3553","article-title":"Watch where you\u2019re going! gaze and head orientation as predictors for social robot navigation","author":"Holman","year":"2021"},{"key":"B19","first-page":"200","article-title":"Using human eye gaze patterns as indicators of need for assistance from a socially assistive robot","author":"Kurylo","year":"2019"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2304.08485","article-title":"Visual instruction tuning","volume":"36","author":"Liu","year":"2024","journal-title":"Adv. neural Inf. Process. Syst."},{"key":"B21","doi-asserted-by":"publisher","first-page":"770165","DOI":"10.3389\/frobt.2022.770165","article-title":"Toward an attentive robotic architecture: learning-based mutual gaze estimation in human\u2013robot interaction","volume":"9","author":"Lombardi","year":"","journal-title":"Front. Robotics AI"},{"key":"B22","first-page":"480","article-title":"Icub knows where you look: exploiting social cues for interactive object detection learning","author":"Lombardi","year":""},{"key":"B23","doi-asserted-by":"publisher","first-page":"69","DOI":"10.5674\/jjppp1983.11.69","article-title":"Measurement of coordination of eye and head movements by sensor of terrestrial magnetism","volume":"11","author":"Maesako","year":"1993","journal-title":"Jpn. J. Physiological Psychol. Psychophysiol."},{"key":"B25","first-page":"862","article-title":"Interactive data collection for deep learning object detectors on humanoid robots","author":"Maiettini","year":"2017"},{"key":"B26","doi-asserted-by":"publisher","first-page":"739","DOI":"10.1007\/s10514-019-09894-9","article-title":"On-line object detection: a robotics challenge","volume":"44","author":"Maiettini","year":"","journal-title":"Aut. Robots"},{"key":"B27","first-page":"194","article-title":"A weakly supervised strategy for learning object detection on a humanoid robot","author":"Maiettini","year":""},{"key":"B28","first-page":"392","article-title":"Weakly-supervised object detection learning through human-robot interaction","author":"Maiettini","year":"2021"},{"key":"B29","doi-asserted-by":"publisher","first-page":"531","DOI":"10.1007\/bf00247307","article-title":"Changing patterns of eye-head coordination during 6 h of optically reversed vision","volume":"69","author":"Melvill Jones","year":"1988","journal-title":"Exp. Brain Res."},{"key":"B30","doi-asserted-by":"publisher","first-page":"8","DOI":"10.5772\/5761","article-title":"Yarp: yet another robot platform","volume":"3","author":"Metta","year":"2006","journal-title":"Int. J. Adv. Robotic Syst."},{"key":"B31","doi-asserted-by":"publisher","first-page":"1125","DOI":"10.1016\/j.neunet.2010.08.010","article-title":"The icub humanoid robot: an open-systems platform for research in cognitive development","volume":"23","author":"Metta","year":"2010","journal-title":"Neural Netw."},{"key":"B32","first-page":"318","article-title":"Eye gaze tracking for a humanoid robot","author":"Palinko","year":"2015"},{"key":"B33","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1016\/s0028-3932(02)00146-x","article-title":"Brain activation evoked by perception of gaze shifts: the influence of context","volume":"41","author":"Pelphrey","year":"2003","journal-title":"Neuropsychologia"},{"key":"B34","first-page":"1435","article-title":"Following gaze in video","author":"Recasens","year":"2017"},{"key":"B35","first-page":"8615","article-title":"Human gaze following for human-robot interaction","author":"Saran","year":"2018"},{"key":"B36","volume-title":"3dgazenet: generalizing gaze estimation with weak-supervision from synthetic views","author":"Ververas","year":"2022"},{"key":"B37","doi-asserted-by":"publisher","first-page":"332","DOI":"10.1109\/tsmcb.2002.999809","article-title":"Study on eye gaze estimation","volume":"32","author":"Wang","year":"2002","journal-title":"IEEE Trans. Syst. Man, Cybern. Part B Cybern."},{"key":"B38","article-title":"Humanoid robot as assistant tutor for autistic children","volume":"8","author":"Yousif","year":"2020","journal-title":"Int. J. Comput. Appl. Sci."}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1346714\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T04:10:59Z","timestamp":1729138259000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1346714\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,17]]},"references-count":37,"alternative-id":["10.3389\/frobt.2024.1346714"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2024.1346714","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,17]]},"article-number":"1346714"}}