{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,2]],"date-time":"2026-02-02T05:09:55Z","timestamp":1770008995564,"version":"3.49.0"},"reference-count":88,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2021,10,18]],"date-time":"2021-10-18T00:00:00Z","timestamp":1634515200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"EU H2020 Marie Sk\u0142odowska-Curie Actions Innovative Training Networks","award":["674868"],"award-info":[{"award-number":["674868"]}]},{"name":"Royal Academy of Engineering IAPP project Human-Robot Interaction Strategies for Rehabilitation based on Socially Assistive Robotics","award":["IAPP\/1516\/137"],"award-info":[{"award-number":["IAPP\/1516\/137"]}]},{"DOI":"10.13039\/100007637","name":"Colciencias","doi-asserted-by":"crossref","award":["813-2017"],"award-info":[{"award-number":["813-2017"]}],"id":[{"id":"10.13039\/100007637","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100011878","name":"Flemish Government","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100011878","id-type":"DOI","asserted-by":"crossref"}]},{"name":"EU H2020 L2TOR","award":["688014"],"award-info":[{"award-number":["688014"]}]},{"name":"EU FP7 DREAM","award":["611391"],"award-info":[{"award-number":["611391"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Hum.-Robot Interact."],"published-print":{"date-parts":[[2022,3,31]]},"abstract":"<jats:p>\n            User identification is an essential step in creating a personalised long-term interaction with robots. This requires learning the users continuously and incrementally, possibly starting from a state without any known user. In this article, we describe a multi-modal incremental Bayesian network with online learning, which is the first method that can be applied in such scenarios. Face recognition is used as the primary biometric, and it is combined with ancillary information, such as gender, age, height, and time of interaction to improve the recognition. The Multi-modal Long-term User Recognition Dataset is generated to simulate various\n            <jats:bold>human-robot interaction (HRI)<\/jats:bold>\n            scenarios and evaluate our approach in comparison to face recognition, soft biometrics, and a state-of-the-art open world recognition method (Extreme Value Machine). The results show that the proposed methods significantly outperform the baselines, with an increase in the identification rate up to 47.9% in open-set and closed-set scenarios, and a significant decrease in long-term recognition performance loss. The proposed models generalise well to new users, provide stability, improve over time, and decrease the bias of face recognition. The models were applied in HRI studies for user recognition, personalised rehabilitation, and customer-oriented service, which showed that they are suitable for long-term HRI in the real world.\n          <\/jats:p>","DOI":"10.1145\/3477963","type":"journal-article","created":{"date-parts":[[2021,10,19]],"date-time":"2021-10-19T00:28:21Z","timestamp":1634603301000},"page":"1-50","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Multi-modal Open World User Identification"],"prefix":"10.1145","volume":"11","author":[{"given":"Bahar","family":"Irfan","sequence":"first","affiliation":[{"name":"Centre for Robotics and Neural Systems, University of Plymouth, Drake Circus, Plymouth, United Kingdom"}]},{"given":"Michael Garcia","family":"Ortiz","sequence":"additional","affiliation":[{"name":"AI Lab, SoftBank Robotics Europe and City, University of London, London, United Kingdom"}]},{"given":"Natalia","family":"Lyubova","sequence":"additional","affiliation":[{"name":"Prophesee, Paris, France"}]},{"given":"Tony","family":"Belpaeme","sequence":"additional","affiliation":[{"name":"IDLab - imec, Ghent University and Centre for Robotics and Neural Systems, University of Plymouth, Plymouth, United Kingdom"}]}],"member":"320","published-online":{"date-parts":[[2021,10,18]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.3390\/app8030387"},{"key":"e_1_3_3_3_2","volume-title":"OpenFace: A General-purpose Face Recognition Library with Mobile Applications","author":"Amos Brandon","year":"2016","unstructured":"Brandon Amos, Bartosz Ludwiczuk, and Mahadev Satyanarayanan. 2016. OpenFace: A General-purpose Face Recognition Library with Mobile Applications. Technical Report. CMU-CS-16-118, CMU School of Computer Science."},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/3468.952717"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.21236\/ADA457181"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2009.5152362"},{"key":"e_1_3_3_7_2","first-page":"3","volume-title":"13th Conference on Uncertainty in Artificial Intelligence","author":"Bauer Eric","year":"1997","unstructured":"Eric Bauer, Daphne Koller, and Yoram Singer. 1997. Update rules for parameter estimation in Bayesian networks. In 13th Conference on Uncertainty in Artificial Intelligence. 3\u201313."},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00216-007-1818-6"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298799"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.173"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/BFb0016008"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.5555\/1162264"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1038\/srep19908"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-014-0245-z"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/CCRA.2018.8588133"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2021.633248"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/11608288_76"},{"key":"e_1_3_3_18_2","first-page":"176","volume-title":"International Conference on Audio and Video-based Person Authentication","author":"Choudhury Tanzeem","year":"1999","unstructured":"Tanzeem Choudhury, Brian Clarkson, Tony Jebara, and Alex Pentland. 1999. Multimodal person recognition using unconstrained audio and video. In International Conference on Audio and Video-based Person Authentication. 176\u2013181."},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3125739.3125756"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2019.00110"},{"key":"e_1_3_3_21_2","volume-title":"Online Learning of Bayesian Network Parameters","author":"Cohen Ira","year":"2001","unstructured":"Ira Cohen, Alexandre Bronstein, and Fabio G. Cozman. 2001. Online Learning of Bayesian Network Parameters. Technical Report HPL-2001-55 (R.1). HP Laboratories."},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4471-6784-6_17"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00994110"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/AFGR.2008.4813386"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2015.2480381"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-010-0635-7"},{"key":"e_1_3_3_27_2","unstructured":"Rocco De Rosa Thomas Mensink and Barbara Caputo. 2016. Online Open World Recognition. arxiv:cs.CV\/1604.02275"},{"key":"e_1_3_3_28_2","first-page":"733","volume-title":"IEEE International Conference on Data Mining","author":"Rosa Rocco D. De","year":"2015","unstructured":"Rocco D. De Rosa, Francesco Orabona, and Nicol\u00f2 Cesa-Bianchi. 2015. The ABACOC algorithm: A novel approach for nonparametric classification of data streams. In IEEE International Conference on Data Mining. IEEE, 733\u2013738. DOI:https:\/\/doi.org\/10.1109\/ICDM.2015.43"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1162\/089976698300017197"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939835"},{"key":"e_1_3_3_31_2","first-page":"465","volume-title":"9th International Conference on Rehabilitation Robotics","author":"Feil-Seifer David","year":"2005","unstructured":"David Feil-Seifer and Maja J. Matari\u0107. 2005. Defining socially assistive robotics. In 9th International Conference on Rehabilitation Robotics (ICORR). 465\u2013468. DOI:https:\/\/doi.org\/10.1109\/ICORR.2005.1501143"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2007.364080"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICAR.2013.6766498"},{"key":"e_1_3_3_34_2","unstructured":"ZongYuan Ge Sergey Demyanov Zetao Chen and Rahil Garnavi. 2017. Generative OpenMax for Multi-Class Open Set Classification. arxiv:cs.CV\/1707.07418"},{"key":"e_1_3_3_35_2","volume-title":"European Symposium on Artificial Neural Networks","author":"Gepperth A.","year":"2016","unstructured":"A. Gepperth and B. Hammer. 2016. Incremental learning algorithms and applications. In European Symposium on Artificial Neural Networks (ESANN)."},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1177\/1179546817710028"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-60045-1_20"},{"key":"e_1_3_3_38_2","volume-title":"Robust Statistics: The Approach Based on Influence Functions","author":"Hampel Frank R.","year":"1986","unstructured":"Frank R. Hampel, Elvezio M. Ronchetti, Peter J. Rousseeuw, and Werner A. Stahel. 1986. Robust Statistics: The Approach Based on Influence Functions. Wiley."},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2008.4543772"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022623210503"},{"key":"e_1_3_3_41_2","first-page":"3","volume-title":"Online Learning of a Weighted Selective Naive Bayes Classifier with Non-convex Optimization","author":"Hue Carine","year":"2017","unstructured":"Carine Hue, Marc Boull\u00e9, and Vincent Lemaire. 2017. Online Learning of a Weighted Selective Naive Bayes Classifier with Non-convex Optimization. Springer International Publishing, Cham, 3\u201317. DOI:https:\/\/doi.org\/10.1007\/978-3-319-45763-5_1"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/RO-MAN47096.2020.9223491"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3371382.3378278"},{"key":"e_1_3_3_44_2","volume-title":"ACM\/IEEE International Conference on Human-robot Interaction Social Robots in the Wild Workshop","author":"Irfan Bahar","year":"2018","unstructured":"Bahar Irfan, Natalia Lyubova, Michael Garcia Ortiz, and Tony Belpaeme. 2018. Multi-modal open-set person identification in HRI. In ACM\/IEEE International Conference on Human-robot Interaction Social Robots in the Wild Workshop. ACM."},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2005.01.012"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-25948-0_99"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2009.5413921"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-77326-1_6"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10578-9_26"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2002.1017616"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.5555\/1577069.1755843"},{"key":"e_1_3_3_52_2","first-page":"717","volume-title":"Probabilistic Graphical Models: Principles and Techniques","author":"Koller Daphne","year":"2009","unstructured":"Daphne Koller and Nir Friedman. 2009. Parameter estimation. In Probabilistic Graphical Models: Principles and Techniques. The MIT Press, 717\u2013782."},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-59745-452-0"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2020.1801172"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICORR.2017.8009382"},{"key":"e_1_3_3_56_2","first-page":"852","volume-title":"IEEE Computer Society Conference on Computer Vision and Pattern Recognition","volume":"1","author":"Lee Kuang-Chih","year":"2005","unstructured":"Kuang-Chih Lee and D. Kriegman. 2005. Online learning of probabilistic appearance manifolds for video-based recognition and tracking. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), Vol. 1. IEEE, 852\u2013859. DOI:https:\/\/doi.org\/10.1109\/CVPR.2005.260"},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2993016"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-37275-2_40"},{"key":"e_1_3_3_59_2","first-page":"1480","volume-title":"12th International Conference on Control, Automation and Systems","author":"Lin Chao-Yu","year":"2012","unstructured":"Chao-Yu Lin, Kai-Tai Song, Yi-Wen Chen, Shuo-Cheng Chien, Sin-Horng Chen, Chen-Yu Chiang, Jyh-Her Yang, Yi-Chiao Wu, and Tzu-Jui Liu. 2012. User identification design by fusion of face recognition and speaker recognition. In 12th International Conference on Control, Automation and Systems. 1480\u20131485."},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICNC.2008.651"},{"key":"e_1_3_3_61_2","unstructured":"Salvatore S. Mangiafico. 2016. Summary and Analysis of Extension Program Evaluation in R . Vol. version 1.18.8. Retrieved from: rcompanion.org\/handbook\/."},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.5555\/2447556.2447565"},{"key":"e_1_3_3_63_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.102.3.419"},{"key":"e_1_3_3_64_2","first-page":"109","volume-title":"Psychology of Learning and Motivation","author":"McCloskey Michael","year":"1989","unstructured":"Michael McCloskey and Neal J. Cohen. 1989. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of Learning and Motivation, Gordon H. Bower (Ed.). Vol. 24. Academic Press, 109\u2013165. DOI:https:\/\/doi.org\/10.1016\/S0079-7421(08)60536-8"},{"key":"e_1_3_3_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.83"},{"key":"e_1_3_3_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROMAN.2014.6926273"},{"key":"e_1_3_3_67_2","unstructured":"German Ignacio Parisi Ronald Kemker Jose L. Part Christopher Kanan and Stefan Wermter. 2018. Continual Lifelong Learning with Neural Networks: A Review. arxiv:cs.LG\/1802.07569"},{"key":"e_1_3_3_68_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2010.2049842"},{"key":"e_1_3_3_69_2","doi-asserted-by":"publisher","DOI":"10.5244\/C.29.41"},{"key":"e_1_3_3_70_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-85729-932-1_21"},{"key":"e_1_3_3_71_2","doi-asserted-by":"publisher","DOI":"10.1109\/MTS.2018.2795096"},{"key":"e_1_3_3_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2015.41"},{"key":"e_1_3_3_73_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-016-0940-3"},{"key":"e_1_3_3_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2707495"},{"key":"e_1_3_3_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISBA.2017.7947693"},{"key":"e_1_3_3_76_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aat7544"},{"key":"e_1_3_3_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2012.256"},{"key":"e_1_3_3_78_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2014.2321392"},{"key":"e_1_3_3_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCB.2011.6117490"},{"key":"e_1_3_3_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"e_1_3_3_81_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.244"},{"key":"e_1_3_3_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.220"},{"key":"e_1_3_3_83_2","volume-title":"International Symposium on Pattern Recognition","author":"Verlinde Patrick","year":"1999","unstructured":"Patrick Verlinde, Pascal Druyts, Gerard Cholet, and Marc Acheroy. 1999. Applying Bayes based classifiers for decision fusion in a multi-modal identity verification system. In International Symposium on Pattern Recognition."},{"key":"e_1_3_3_84_2","doi-asserted-by":"publisher","DOI":"10.5555\/1762222.1762327"},{"key":"e_1_3_3_85_2","doi-asserted-by":"publisher","DOI":"10.1145\/3171221.3171273"},{"key":"e_1_3_3_86_2","doi-asserted-by":"publisher","DOI":"10.5772\/62950"},{"key":"e_1_3_3_87_2","doi-asserted-by":"publisher","DOI":"10.1109\/MWSCAS.2004.1353967"},{"key":"e_1_3_3_88_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-74549-5_19"},{"key":"e_1_3_3_89_2","first-page":"523","volume-title":"18th International Conference on Pattern Recognition","volume":"1","author":"Zhou Yue","year":"2006","unstructured":"Yue Zhou and T. S. Huang. 2006. Weighted Bayesian network for visual tracking. In 18th International Conference on Pattern Recognition (ICPR\u201906), Vol. 1. IEEE, 523\u2013526. DOI:https:\/\/doi.org\/10.1109\/ICPR.2006.1188"}],"container-title":["ACM Transactions on Human-Robot Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3477963","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3477963","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:10:38Z","timestamp":1750183838000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3477963"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,18]]},"references-count":88,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,3,31]]}},"alternative-id":["10.1145\/3477963"],"URL":"https:\/\/doi.org\/10.1145\/3477963","relation":{},"ISSN":["2573-9522","2573-9522"],"issn-type":[{"value":"2573-9522","type":"print"},{"value":"2573-9522","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,18]]},"assertion":[{"value":"2020-01-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-10-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}