{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T01:13:56Z","timestamp":1774314836332,"version":"3.50.1"},"reference-count":47,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2024,7,17]],"date-time":"2024-07-17T00:00:00Z","timestamp":1721174400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Project Augmented Humanity","award":["POCI-01-0247-FEDER-046103"],"award-info":[{"award-number":["POCI-01-0247-FEDER-046103"]}]},{"name":"Project Augmented Humanity","award":["2021.06664.BD"],"award-info":[{"award-number":["2021.06664.BD"]}]},{"name":"Competitiveness and Internationalization Operational Program","award":["POCI-01-0247-FEDER-046103"],"award-info":[{"award-number":["POCI-01-0247-FEDER-046103"]}]},{"name":"Competitiveness and Internationalization Operational Program","award":["2021.06664.BD"],"award-info":[{"award-number":["2021.06664.BD"]}]},{"name":"Lisbon Regional Operational Program","award":["POCI-01-0247-FEDER-046103"],"award-info":[{"award-number":["POCI-01-0247-FEDER-046103"]}]},{"name":"Lisbon Regional Operational Program","award":["2021.06664.BD"],"award-info":[{"award-number":["2021.06664.BD"]}]},{"name":"European Regional Development Fund","award":["POCI-01-0247-FEDER-046103"],"award-info":[{"award-number":["POCI-01-0247-FEDER-046103"]}]},{"name":"European Regional Development Fund","award":["2021.06664.BD"],"award-info":[{"award-number":["2021.06664.BD"]}]},{"name":"Foundation for Science and Technology (FCT)","award":["POCI-01-0247-FEDER-046103"],"award-info":[{"award-number":["POCI-01-0247-FEDER-046103"]}]},{"name":"Foundation for Science and Technology (FCT)","award":["2021.06664.BD"],"award-info":[{"award-number":["2021.06664.BD"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>This paper presents a collaborative manufacturing cell implemented in a laboratory setting, focusing on developing learning-based interaction abilities to enhance versatility and ease of use. The key components of the system include 3D real-time volumetric monitoring for safety, visual recognition of hand gestures for human-to-robot communication, classification of physical-contact-based interaction primitives during handover operations, and detection of hand\u2013object interactions to anticipate human intentions. Due to the nature and complexity of perception, deep-learning-based techniques were used to enhance robustness and adaptability. The main components are integrated in a system containing multiple functionalities, coordinated through a dedicated state machine. This ensures appropriate actions and reactions based on events, enabling the execution of specific modules to complete a given multi-step task. An ROS-based architecture supports the software infrastructure among sensor interfacing, data processing, and robot and gripper controllers nodes. The result is demonstrated by a functional use case that involves multiple tasks and behaviors, paving the way for the deployment of more advanced collaborative cells in manufacturing contexts.<\/jats:p>","DOI":"10.3390\/robotics13070107","type":"journal-article","created":{"date-parts":[[2024,7,17]],"date-time":"2024-07-17T12:29:43Z","timestamp":1721219383000},"page":"107","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Human\u2013Robot Collaborative Manufacturing Cell with Learning-Based Interaction Abilities"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3308-6667","authenticated-orcid":false,"given":"Joel","family":"Baptista","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering (DEM), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9571-9808","authenticated-orcid":false,"given":"Afonso","family":"Castro","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering (DEM), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2455-1849","authenticated-orcid":false,"given":"Manuel","family":"Gomes","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering (DEM), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-1506-155X","authenticated-orcid":false,"given":"Pedro","family":"Amaral","sequence":"additional","affiliation":[{"name":"Department of Electronics, Telecommunications and Informatics (DETI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1283-7388","authenticated-orcid":false,"given":"V\u00edtor","family":"Santos","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering (DEM), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6191-0727","authenticated-orcid":false,"given":"Filipe","family":"Silva","sequence":"additional","affiliation":[{"name":"Department of Electronics, Telecommunications and Informatics (DETI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9288-5058","authenticated-orcid":false,"given":"Miguel","family":"Oliveira","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering (DEM), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, 3810-193 Aveiro, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2024,7,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1016\/j.procs.2019.09.104","article-title":"Industry 5.0 and human\u2013robot co-working","volume":"158","author":"Demir","year":"2019","journal-title":"Procedia Comput. Sci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"392","DOI":"10.1016\/j.jmsy.2022.04.007","article-title":"Evaluating quality in human\u2013robot interaction: A systematic search and classification of performance and human-centered factors, measures and metrics towards an industry 5.0","volume":"63","author":"Coronado","year":"2022","journal-title":"J. Manuf. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1016\/j.mechatronics.2018.02.009","article-title":"Survey on human\u2013robot collaboration in industrial settings: Safety, intuitive interfaces and applications","volume":"55","author":"Villani","year":"2018","journal-title":"Mechatronics"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1109\/TSMC.2020.3041231","article-title":"Survey of human\u2013robot collaboration in industrial settings: Awareness, intelligence, and compliance","volume":"51","author":"Kumar","year":"2020","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Castro, A., Silva, F., and Santos, V. (2021). Trends of human\u2013robot collaboration in industry contexts: Handover, learning, and metrics. Sensors, 21.","DOI":"10.3390\/s21124113"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"102208","DOI":"10.1016\/j.rcim.2021.102208","article-title":"Human\u2013robot collaboration in industrial environments: A literature review on non-destructive disassembly","volume":"73","author":"Hjorth","year":"2022","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"101998","DOI":"10.1016\/j.rcim.2020.101998","article-title":"Emerging research fields in safety and ergonomics in industrial collaborative robotics: A systematic literature review","volume":"67","author":"Gualtieri","year":"2021","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.jmsy.2021.11.007","article-title":"Designing human\u2013robot collaboration (HRC) workspaces in industrial settings: A systematic literature review","volume":"62","author":"Pinto","year":"2022","journal-title":"J. Manuf. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Lorenzini, M., Lagomarsino, M., Fortini, L., Gholami, S., and Ajoudani, A. (2023). Ergonomic human\u2013robot collaboration in industry: A review. Front. Robot. AI, 9.","DOI":"10.3389\/frobt.2022.813907"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1016\/j.procir.2015.08.014","article-title":"Design considerations for safe human\u2013robot collaborative workplaces","volume":"37","author":"Michalos","year":"2015","journal-title":"Procedia CIrP"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1016\/j.promfg.2018.06.095","article-title":"Safe layout design and evaluation of a human\u2013robot collaborative application cell through risk assessment\u2013a computer aided approach","volume":"25","author":"Wadekar","year":"2018","journal-title":"Procedia Manuf."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Forlini, M., Neri, F., Scoccia, C., Carbonari, L., and Palmieri, G. (2023, January 14\u201316). Collision Avoidance in Collaborative Robotics Based on Real-Time Skeleton Tracking. Proceedings of the International Conference on Robotics in Alpe-Adria Danube Region, Bled, Slovenia.","DOI":"10.1007\/978-3-031-32606-6_10"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1109\/MRA.2018.2890460","article-title":"Adaptable workstations for human\u2013robot collaboration: A reconfigurable framework for improving worker ergonomics and productivity","volume":"26","author":"Kim","year":"2019","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Gualtieri, L., Palomba, I., Merati, F.A., Rauch, E., and Vidoni, R. (2020). Design of human-centered collaborative assembly workstations for the improvement of operators\u2019 physical ergonomics and production efficiency: A case study. Sustainability, 12.","DOI":"10.3390\/su12093606"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1928","DOI":"10.1109\/TCDS.2022.3182811","article-title":"Pick the right co-worker: Online assessment of cognitive ergonomics in human\u2013robot collaborative assembly","volume":"15","author":"Lagomarsino","year":"2022","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1729881417716043","DOI":"10.1177\/1729881417716043","article-title":"Natural multimodal communication for human\u2013robot collaboration","volume":"14","author":"Maurtua","year":"2017","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Su, H., Qi, W., Chen, J., Yang, C., Sandoval, J., and Laribi, M.A. (2023). Recent advancements in multimodal human\u2013robot interaction. Front. Neurorobot., 17.","DOI":"10.3389\/fnbot.2023.1084000"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1016\/j.robot.2019.03.003","article-title":"Cobot programming for collaborative industrial tasks: An overview","volume":"116","author":"Marei","year":"2019","journal-title":"Robot. Auton. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.rcim.2015.12.007","article-title":"Collaborative manufacturing with physical human\u2013robot interaction","volume":"40","author":"Cherubini","year":"2016","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1888","DOI":"10.1109\/TCYB.2019.2947532","article-title":"Physical human\u2013robot collaboration: Robotic systems, learning methods, collaborative strategies, sensors, and actuators","volume":"51","author":"Ogenyi","year":"2019","journal-title":"IEEE Trans. Cybern."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"10355","DOI":"10.1109\/JSEN.2020.2995271","article-title":"Progress and prospects of multimodal fusion methods in physical human\u2013robot interaction: A review","volume":"20","author":"Xue","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Huang, C.M., and Mutlu, B. (2016, January 7\u201310). Anticipatory robot control for efficient human\u2013robot collaboration. Proceedings of the 2016 11th ACM\/IEEE International Conference on Human-Robot Interaction (HRI), Christchurch, New Zealand.","DOI":"10.1109\/HRI.2016.7451737"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1109\/LRA.2017.2729666","article-title":"Anticipatory robot assistance for the prevention of human static joint overloading in human\u2013robot collaboration","volume":"3","author":"Kim","year":"2017","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"4132","DOI":"10.1109\/LRA.2018.2861569","article-title":"Action anticipation: Reading the intentions of humans and robots","volume":"3","author":"Duarte","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"B\u00fctepage, J., Kjellstr\u00f6m, H., and Kragic, D. (2018, January 21\u201326). Anticipating many futures: Online human motion prediction and generation for human\u2013robot interaction. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8460651"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1855","DOI":"10.1109\/TRO.2021.3075365","article-title":"Object handovers: A review for robotics","volume":"37","author":"Ortenzi","year":"2021","journal-title":"IEEE Trans. Robot."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"4378","DOI":"10.1109\/LRA.2023.3280752","article-title":"Maximising Coefficiency of Human-Robot Handovers through Reinforcement Learning","volume":"8","author":"Lagomarsino","year":"2023","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"106368","DOI":"10.1016\/j.infsof.2020.106368","article-title":"Large-scale machine learning systems in real-world industrial settings: A review of challenges and solutions","volume":"127","author":"Lwakatare","year":"2020","journal-title":"Inf. Softw. Technol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"102231","DOI":"10.1016\/j.rcim.2021.102231","article-title":"A survey of robot learning strategies for human\u2013robot collaboration in industrial settings","volume":"73","author":"Mukherjee","year":"2022","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3489","DOI":"10.1007\/s11831-023-09903-2","article-title":"A Review on the Effectiveness of Machine Learning and Deep Learning Algorithms for Collaborative Robot","volume":"30","author":"Sajwan","year":"2023","journal-title":"Arch. Comput. Methods Eng."},{"key":"ref_31","unstructured":"Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., and Ng, A.Y. (2024, July 10). ROS: An open-source Robot Operating System. In ICRA Workshop on Open Source Software, Kobe, Japan, 2009; Volume 3, p. 5. Available online: http:\/\/robotics.stanford.edu\/~ang\/papers\/icraoss09-ROS.pdf."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"118000","DOI":"10.1016\/j.eswa.2022.118000","article-title":"ATOM: A general calibration framework for multi-modal, multi-sensor systems","volume":"207","author":"Oliveira","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1016\/j.jmsy.2022.07.006","article-title":"A Sensor-to-Pattern Calibration Framework for Multi-Modal Industrial Collaborative Cells","volume":"64","author":"Rato","year":"2022","journal-title":"J. Manuf. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Baptista, J., Santos, V., Silva, F., and Pinho, D. (2023). Domain Adaptation with Contrastive Simultaneous Multi-Loss Training for Hand Gesture Recognition. Sensors, 23.","DOI":"10.3390\/s23063332"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1016\/j.jmsy.2023.03.010","article-title":"Classification of handover interaction primitives in a COBOT\u2013human context with a deep neural network","volume":"68","author":"Castro","year":"2023","journal-title":"J. Manuf. Syst."},{"key":"ref_36","unstructured":"Gomes, M., Oliveira, M., and Santos, V. (2023). Volumetric Occupancy Detection: A Comparative Analysis of Mapping Algorithms. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Amaral, P., Silva, F., and Santos, V. (2023). Recognition of Grasping Patterns Using Deep Learning for Human\u2013Robot Collaboration. Sensors, 23.","DOI":"10.3390\/s23218989"},{"key":"ref_38","unstructured":"Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C., Yong, M.G., and Lee, J. (2019). MediaPipe: A Framework for Building Perception Pipelines. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2015). Rethinking the Inception Architecture for Computer Vision. arXiv.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Scibilia, A., Laghi, M., De Momi, E., Peternel, L., and Ajoudani, A. (2018, January 6\u20139). A Self-Adaptive Robot Control Framework for Improved Tracking and Interaction Performances in Low-Stiffness Teleoperation. Proceedings of the 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids), Beijing, China.","DOI":"10.1109\/HUMANOIDS.2018.8625062"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"12219","DOI":"10.1109\/LRA.2022.3211493","article-title":"Safety-Aware Time-Optimal Motion Planning With Uncertain Human State Estimation","volume":"7","author":"Faroni","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1007\/s10514-012-9321-0","article-title":"OctoMap: An Efficient Probabilistic 3D Mapping Framework Based on Octrees","volume":"34","author":"Hornung","year":"2013","journal-title":"Auton. Robot."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Deans, C. (2021). Biological Prescience: The Role of Anticipation in Organismal Processes. Front. Physiol., 12.","DOI":"10.3389\/fphys.2021.672457"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1007\/s10514-017-9648-7","article-title":"Using probabilistic movement primitives in robotics","volume":"42","author":"Paraschos","year":"2018","journal-title":"Auton. Robot."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Huang, C.M., Andrist, S., Saupp\u00e9, A., and Mutlu, B. (2015). Using gaze patterns to predict task intent in collaboration. Front. Psychol., 6.","DOI":"10.3389\/fpsyg.2015.01049"},{"key":"ref_46","first-page":"3","article-title":"Reducing the Barrier to Entry of Complex Robotic Software: A MoveIt! Case Study","volume":"5","author":"Coleman","year":"2014","journal-title":"J. Softw. Eng. Robot."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/MRA.2010.938836","article-title":"The SMACH High-Level Executive [ROS News]","volume":"17","author":"Bohren","year":"2010","journal-title":"IEEE Robot. Autom. Mag."}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/13\/7\/107\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:18:21Z","timestamp":1760109501000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/13\/7\/107"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,17]]},"references-count":47,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,7]]}},"alternative-id":["robotics13070107"],"URL":"https:\/\/doi.org\/10.3390\/robotics13070107","relation":{},"ISSN":["2218-6581"],"issn-type":[{"value":"2218-6581","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,17]]}}}