{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T02:57:52Z","timestamp":1773716272737,"version":"3.50.1"},"reference-count":50,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2022,5,10]],"date-time":"2022-05-10T00:00:00Z","timestamp":1652140800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000780","name":"EU Horizon 2020 Program","doi-asserted-by":"publisher","award":["780086"],"award-info":[{"award-number":["780086"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000780","name":"EU Horizon 2020 Program","doi-asserted-by":"publisher","award":["UIDB\/50009\/2020"],"award-info":[{"award-number":["UIDB\/50009\/2020"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001871","name":"ISR\/LARSyS Strategic Funding","doi-asserted-by":"publisher","award":["780086"],"award-info":[{"award-number":["780086"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001871","name":"ISR\/LARSyS Strategic Funding","doi-asserted-by":"publisher","award":["UIDB\/50009\/2020"],"award-info":[{"award-number":["UIDB\/50009\/2020"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Applied Sciences"],"abstract":"<jats:p>This paper describes the development of three main functionalities for a domestic mobile service robot and an automatic benchmarking system used for the systematic performance evaluation of the robot\u2019s functionalities. Three main robot functionalities are addressed: (1) People Perception, (2) People Following and (3) Pick and Placing, where the hardware and software systems developed for each functionality are described and demonstrated on an actual mobile service robot, with the goal of providing assistance to an elderly person inside the house. Furthermore, a set of innovative benchmarks and an automatic performance evaluation system are proposed and used to evaluate the performance of the developed functionalities. These benchmarks are now made publicly available and is part of the European Robotics League (ERL)-Consumer to systematically evaluate the performance of service robot solutions at different testbeds around Europe.<\/jats:p>","DOI":"10.3390\/app12104819","type":"journal-article","created":{"date-parts":[[2022,5,10]],"date-time":"2022-05-10T08:31:55Z","timestamp":1652171515000},"page":"4819","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Functionalities, Benchmarking System and Performance Evaluation for a Domestic Service Robot: People Perception, People Following, and Pick and Placing"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8456-6284","authenticated-orcid":false,"given":"Meysam","family":"Basiri","sequence":"first","affiliation":[{"name":"Institute for Systems and Robotics, Tecnico Lisboa, 1049-001 Lisboa, Portugal"}]},{"given":"Jo\u00e3o","family":"Pereira","sequence":"additional","affiliation":[{"name":"Institute for Systems and Robotics, Tecnico Lisboa, 1049-001 Lisboa, Portugal"}]},{"given":"Rui","family":"Bettencourt","sequence":"additional","affiliation":[{"name":"Institute for Systems and Robotics, Tecnico Lisboa, 1049-001 Lisboa, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8209-6939","authenticated-orcid":false,"given":"Enrico","family":"Piazza","sequence":"additional","affiliation":[{"name":"Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, 20133 Milan, Italy"}]},{"given":"Emanuel","family":"Fernandes","sequence":"additional","affiliation":[{"name":"Institute for Systems and Robotics, Tecnico Lisboa, 1049-001 Lisboa, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8842-8989","authenticated-orcid":false,"given":"Carlos","family":"Azevedo","sequence":"additional","affiliation":[{"name":"Institute for Systems and Robotics, Tecnico Lisboa, 1049-001 Lisboa, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8962-8050","authenticated-orcid":false,"given":"Pedro","family":"Lima","sequence":"additional","affiliation":[{"name":"Institute for Systems and Robotics, Tecnico Lisboa, 1049-001 Lisboa, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,10]]},"reference":[{"key":"ref_1","unstructured":"Wise, M., Ferguson, M., King, D., Diehr, E., and Dymesich, D. (2016, January 11). Fetch and freight: Standard platforms for service robot applications. Proceedings of the Workshop on Autonomous Mobile Service Robots, New York, NY, USA."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1007\/s11465-007-0070-2","article-title":"Current research, key performances and future development of search and rescue robots","volume":"2","author":"Liu","year":"2007","journal-title":"Front. Mech. Eng. China"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/j.robot.2018.06.005","article-title":"An optional passive\/active transformable wheel-legged mobility concept for search and rescue robots","volume":"107","author":"Bai","year":"2018","journal-title":"Robot. Auton. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"102","DOI":"10.55417\/fr.2021004","article-title":"A multipurpose mobile manipulator for autonomous firefighting and construction of outdoor structures","volume":"1","author":"Basiri","year":"2021","journal-title":"Field Robot."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Siciliano, B., and Khatib, O. (2016). Domestic Robotics. Springer Handbook of Robotics, Springer International Publishing.","DOI":"10.1007\/978-3-319-32552-1"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sinapov, J., and Stoytchev, A. (2011, January 9\u201313). Object category recognition by a humanoid robot using behavior-grounded relational learning. Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China.","DOI":"10.1109\/ICRA.2011.5980417"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1007\/s12369-008-0006-y","article-title":"Toward acceptable domestic robots: Applying insights from social psychology","volume":"1","author":"Young","year":"2009","journal-title":"Int. J. Soc. Robot."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"M\u00fcller, A.C., and Behnke, S. (June, January 31). Learning depth-sensitive conditional random fields for semantic segmentation of RGB-D images. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China.","DOI":"10.1109\/ICRA.2014.6907778"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Messias, J., Ventura, R., Lima, P., Sequeira, J., Alvito, P., Marques, C., and Carri\u00e7o, P. (2014, January 14\u201315). A robotic platform for edutainment activities in a pediatric hospital. Proceedings of the 2014 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), Espinho, Portugal.","DOI":"10.1109\/ICARSC.2014.6849785"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Muszynski, S., St\u00fcckler, J., and Behnke, S. (2012, January 9\u201313). Adjustable autonomy for mobile teleoperation of personal service robots. Proceedings of the 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication, Paris, France.","DOI":"10.1109\/ROMAN.2012.6343870"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Gu, S., Holly, E., Lillicrap, T., and Levine, S. (June, January 29). Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989385"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"358","DOI":"10.5772\/56906","article-title":"Sitlog: A programming language for service robot tasks","volume":"10","author":"Pineda","year":"2013","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Matamoros, M., Seib, V., Memmesheimer, R., and Paulus, D. (2018, January 25\u201327). RoboCup@Home: Summarizing achievements in over eleven years of competition. Proceedings of the 2018 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), Torres Vedras, Portugal.","DOI":"10.1109\/ICARSC.2018.8374181"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1109\/MRA.2015.2448871","article-title":"Competitions for Benchmarking: Task and Functionality Scoring Complete Performance Assessment","volume":"22","author":"Amigoni","year":"2015","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_15","unstructured":"(2020, October 13). European Robotics League. Available online: https:\/\/www.eu-robotics.net\/robotics_league\/."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1007\/s13218-019-00619-9","article-title":"Benchmarking Functionalities of Domestic Service Robots Through Scientific Competitions","volume":"33","author":"Basiri","year":"2019","journal-title":"KI-K\u00fcnstliche Intell."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1109\/MRA.2012.2191993","article-title":"RoboCup@Home: Demonstrating Everyday Manipulation Skills in RoboCup@Home","volume":"19","author":"Stuckler","year":"2012","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Cartucho, J., Ventura, R., and Veloso, M. (2018, January 1\u20135). Robust object recognition through symbiotic deep learning in mobile robots. Proceedings of the 2018 IEEE\/RSJ international conference on intelligent robots and systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8594067"},{"key":"ref_19","unstructured":"(2018, December 13). RoCKIn: Robot Competitions Kick Innovation in Cognitive Systems. Available online: http:\/\/rockinrobotchallenge.eu."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Lima, P.U. (2017). The RoCKIn Project. RoCKIn: Benchmarking Through Robot Competitions, IntechOpen. Chapter 2.","DOI":"10.5772\/intechopen.70011"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1675","DOI":"10.1109\/JSEN.2017.2784900","article-title":"A Multi-Type Features Method for Leg Detection in 2-D Laser Range Data","volume":"18","author":"Li","year":"2018","journal-title":"IEEE Sens. J."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Weinrich, C., Wengefeld, T., Schroeter, C., and Gross, H. (2014, January 25\u201329). People detection and distinction of their walking aids in 2D laser range data based on generic distance-invariant features. Proceedings of the 23rd IEEE International Symposium on Robot and Human Interactive Communication, Edinburgh, UK.","DOI":"10.1109\/ROMAN.2014.6926346"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Davis, J.W., and Keck, M.A. (2005, January 5\u20137). A Two-Stage Template Approach to Person Detection in Thermal Imagery. Proceedings of the 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV\/MOTION\u201905)\u2014Volume 1, Breckenridge, CO, USA.","DOI":"10.1109\/ACVMOT.2005.14"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.patcog.2015.08.027","article-title":"Human detection from images and videos: A survey","volume":"51","author":"Nguyen","year":"2016","journal-title":"Pattern Recognit."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2821","DOI":"10.1109\/TPAMI.2012.241","article-title":"Efficient Human Pose Estimation from Single Depth Images","volume":"35","author":"Shotton","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"995","DOI":"10.1016\/j.imavis.2006.07.012","article-title":"People detection and tracking using stereo vision and color","volume":"25","author":"Aguirre","year":"2007","journal-title":"Image Vis. Comput."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Martinel, N., Luca Foresti, G., and Micheloni, C. (2019, January 16\u201317). Aggregating deep pyramidal representations for person re-identification. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA.","DOI":"10.1109\/CVPRW.2019.00196"},{"key":"ref_28","first-page":"13114","article-title":"Viewpoint-Aware Loss with Angular Regularization for Person Re-Identification","volume":"34","author":"Zhu","year":"2020","journal-title":"AAAI Conf. Artif. Intell."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Haque, A., Alahi, A., and Li, F.-F. (2016, January 27\u201330). Recurrent Attention Models for Depth-Based Person Identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.138"},{"key":"ref_30","unstructured":"Vedaldi, A., Bischof, H., Brox, T., and Frahm, J.M. (2020). Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-identification. Computer Vision\u2014ECCV 2020, Springer International Publishing."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1415","DOI":"10.1109\/TSMC.2016.2616343","article-title":"A Novel Vision-Based Tracking Algorithm for a Human-Following Mobile Robot","volume":"47","author":"Gupta","year":"2017","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Xing, G., Tian, S., Sun, H., Liu, W., and Liu, H. (2013, January 25\u201327). People-following system design for mobile robots using kinect sensor. Proceedings of the 2013 25th Chinese Control and Decision Conference (CCDC), Guiyang, China.","DOI":"10.1109\/CCDC.2013.6561495"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"271","DOI":"10.5772\/56123","article-title":"RGB-D, laser and thermal sensor fusion for people following in a mobile robot","volume":"10","author":"Susperregi","year":"2013","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"558","DOI":"10.1007\/s42452-021-04506-7","article-title":"An autonomous mobile manipulator to build outdoor structures consisting of heterogeneous brick patterns","volume":"3","author":"Basiri","year":"2021","journal-title":"SN Appl. Sci."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Kumar, R., Lal, S., Kumar, S., and Chand, P. (2014, January 4\u20135). Object detection and recognition for a pick and place robot. Proceedings of the Asia-Pacific World Congress on Computer Science and Engineering, Nadi, Fiji.","DOI":"10.1109\/APWCCSE.2014.7053853"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/2.16222","article-title":"Task-level planning of pick-and-place robot motions","volume":"22","author":"Jones","year":"1989","journal-title":"Computer"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1109\/70.538972","article-title":"A tutorial on visual servo control","volume":"12","author":"Hutchinson","year":"1996","journal-title":"IEEE Trans. Robot. Autom."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"828","DOI":"10.1109\/TRO.2011.2112593","article-title":"Photometric Visual Servoing","volume":"27","author":"Collewet","year":"2011","journal-title":"IEEE Trans. Robot."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Bateux, Q., Marchand, E., Leitner, J., Chaumette, F., and Corke, P. (2018, January 21\u201325). Training Deep Neural Networks for Visual Servoing. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8461068"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1177\/0278364914549607","article-title":"Deep learning for detecting robotic grasps","volume":"34","author":"Lenz","year":"2015","journal-title":"Int. J. Robot. Res."},{"key":"ref_41","first-page":"1334","article-title":"End-to-end training of deep visuomotor policies","volume":"17","author":"Levine","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref_42","unstructured":"Kalashnikov, D., Irpan, A., Pastor, P., Ibarz, J., Herzog, A., Jang, E., Quillen, D., Holly, E., Kalakrishnan, M., and Vanhoucke, V. (2018, January 29\u201331). Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation. Proceedings of the 2018 Conference on Robot Learning, Zurich, Switzerland."},{"key":"ref_43","unstructured":"Ventura, R., Basiri, M., Mateus, A., Garcia, J., Miraldo, P., Santos, P., and Lima, P. (2016). A domestic assistive robot developed through robot competitions. Ijcai 2016 Workshop on Autonomous Mobile Service Robots, Intelligent Robots and Systems Group (IRSg)."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1007\/s13218-019-00618-w","article-title":"SocRob@ Home","volume":"33","author":"Lima","year":"2019","journal-title":"KI-K\u00fcnstliche Intell."},{"key":"ref_46","unstructured":"Geitgey, A. (2020, September 10). Face Recognition. Available online: https:\/\/github.com\/ageitgey\/face_recognition."},{"key":"ref_47","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201325). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Kazemi, V., and Sullivan, J. (2014, January 23\u201328). One Millisecond Face Alignment with an Ensemble of Regression Trees. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.241"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Schroff, F., Kalenichenko, D., and Philbin, J. (2015, January 7\u201312). FaceNet: A Unified Embedding for Face Recognition and Clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"ref_50","unstructured":"ALVAR (2020, November 05). Library for Virtual and Augmented Reality. Available online: http:\/\/virtual.vtt.fi\/virtual\/proj2\/multimedia\/alvar\/."}],"container-title":["Applied Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2076-3417\/12\/10\/4819\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:08:46Z","timestamp":1760137726000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2076-3417\/12\/10\/4819"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,10]]},"references-count":50,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["app12104819"],"URL":"https:\/\/doi.org\/10.3390\/app12104819","relation":{},"ISSN":["2076-3417"],"issn-type":[{"value":"2076-3417","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,10]]}}}