{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T17:07:19Z","timestamp":1791911239415,"version":"4.3.4"},"reference-count":48,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2022,9,14]],"date-time":"2022-09-14T00:00:00Z","timestamp":1663113600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"French National Research Agency","award":["ANR-15-IDEX-02"],"award-info":[{"award-number":["ANR-15-IDEX-02"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Industry 4.0 transforms classical industrial systems into more human-centric and digitized systems. Close human\u2013robot collaboration is becoming more frequent, which means security and efficiency issues need to be carefully considered. In this paper, we propose to equip robots with exteroceptive sensors and online motion generation so that the robot is able to perceive and predict human trajectories and react to the motion of the human in order to reduce the occurrence of the collisions. The dataset for training is generated in a real environment in which a human and a robot are sharing their workspace. An Encoder\u2013Decoder based network is proposed to predict the human hand trajectories. A Model Predictive Control (MPC) framework is also proposed, which is able to plan a collision-free trajectory in the shared workspace based on this human motion prediction. The proposed framework is validated in a real environment that ensures collision free collaboration between humans and robots in a shared workspace.<\/jats:p>","DOI":"10.3390\/s22186951","type":"journal-article","created":{"date-parts":[[2022,9,14]],"date-time":"2022-09-14T23:16:36Z","timestamp":1663197396000},"page":"6951","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Human Arm Motion Prediction for Collision Avoidance in a Shared Workspace"],"prefix":"10.3390","volume":"22","author":[{"given":"Pu","family":"Zheng","sequence":"first","affiliation":[{"name":"The Laboratoire d\u2019Informatique de Grenoble, University of Grenoble Alpes, 38000 Grenoble, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pierre-Brice","family":"Wieber","sequence":"additional","affiliation":[{"name":"Inria Centre at the University Grenoble Alpes, 38000 Grenoble, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7517-6858","authenticated-orcid":false,"given":"Junaid","family":"Baber","sequence":"additional","affiliation":[{"name":"The Laboratoire d\u2019Informatique de Grenoble, University of Grenoble Alpes, 38000 Grenoble, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Olivier","family":"Aycard","sequence":"additional","affiliation":[{"name":"The Laboratoire d\u2019Informatique de Grenoble, University of Grenoble Alpes, 38000 Grenoble, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"102231","DOI":"10.1016\/j.rcim.2021.102231","article-title":"A Survey of Robot Learning Strategies for Human-Robot Collaboration in Industrial Settings","volume":"73","author":"Mukherjee","year":"2022","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"ref_2","unstructured":"Romero, D., Stahre, J., Wuest, T., Noran, O., Bernus, P., Fast-Berglund, \u00c5., and Gorecky, D. (2016, January 29\u201331). Towards an operator 4.0 typology: A human-centric perspective on the fourth industrial revolution technologies. Proceedings of the International Conference on Computers and Industrial Engineering (CIE46), Tianjin, China."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1108\/01439910710774386","article-title":"The DLR lightweight robot: Design and control concepts for robots in human environments","volume":"34","author":"Loughlin","year":"2007","journal-title":"Ind. Robot Int. J."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"De Luca, A., and Flacco, F. (2012, January 24\u201327). Integrated control for pHRI: Collision avoidance, detection, reaction and collaboration. Proceedings of the 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), Rome, Italy.","DOI":"10.1109\/BioRob.2012.6290917"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1177\/0018720814565188","article-title":"Analyzing the effects of human-aware motion planning on close-proximity human\u2013Robot collaboration","volume":"57","author":"Lasota","year":"2015","journal-title":"Hum. Factors"},{"key":"ref_6","unstructured":"Zheng, P., Wieber, P.B., and Aycard, O. (August, January 31). Online optimal motion generation with guaranteed safety in shared workspace. Proceedings of the ICRA, Paris, France."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Osokin, D. (2018). Real-time 2d multi-person pose estimation on cpu: Lightweight openpose. arXiv.","DOI":"10.5220\/0007555407440748"},{"key":"ref_8","unstructured":"Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.L., Yong, M.G., and Lee, J. (2019). Mediapipe: A framework for building perception pipelines. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wang, Y., Ye, X., Yang, Y., and Zhang, W. (2017, January 15\u201317). Collision-free trajectory planning in human\u2013robot interaction through hand movement prediction from vision. Proceedings of the 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), Birmingham, UK.","DOI":"10.1109\/HUMANOIDS.2017.8246890"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"103241","DOI":"10.1016\/j.ergon.2021.103241","article-title":"Fostering short-term human anticipatory behavior in human\u2013robot collaboration","volume":"87","author":"Psarakis","year":"2022","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Liu, Q., Li, M., Yin, C., Qian, G., Meng, W., Ai, Q., and Hu, J. (2022). CNN-Based Hand Grasping Prediction and Control via Postural Synergy Basis Extraction. Sensors, 22.","DOI":"10.3390\/s22030831"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Widmann, D., and Karayiannidis, Y. (2018, January 12\u201315). Human Motion Prediction in Human-Robot Handovers based on Dynamic Movement Primitives. Proceedings of the 2018 European Control Conference (ECC), Limassol, Cyprus.","DOI":"10.23919\/ECC.2018.8550170"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"103225","DOI":"10.1016\/j.cviu.2021.103225","article-title":"Deep 3D human pose estimation: A review","volume":"210","author":"Wang","year":"2021","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.cviu.2015.05.006","article-title":"Kinect range sensing: Structured-light versus Time-of-Flight Kinect","volume":"139","author":"Sarbolandi","year":"2015","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Xu, T., An, D., Jia, Y., and Yue, Y. (2021). A review: Point cloud-based 3d human joints estimation. Sensors, 21.","DOI":"10.3390\/s21051684"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1362","DOI":"10.1016\/j.cviu.2009.11.005","article-title":"Kinematic self retargeting: A framework for human pose estimation","volume":"114","author":"Zhu","year":"2010","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ye, M., and Yang, R. (2014, January 23\u201328). Real-time simultaneous pose and shape estimation for articulated objects using a single depth camera. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, USA.","DOI":"10.1109\/CVPR.2014.301"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"677","DOI":"10.1109\/JAS.2017.7510625","article-title":"Automatic feature point detection and tracking of human actions in time-of-flight videos","volume":"4","author":"Yuan","year":"2017","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"178900","DOI":"10.1109\/ACCESS.2020.3027892","article-title":"3D Joints Estimation of the Human Body in Single-Frame Point Cloud","volume":"8","author":"Xu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2821","DOI":"10.1109\/TPAMI.2012.241","article-title":"Efficient human pose estimation from single depth images","volume":"35","author":"Shotton","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"12334","DOI":"10.1109\/JSEN.2020.2999849","article-title":"Learning to estimate 3d human pose from point cloud","volume":"20","author":"Zhou","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1061","DOI":"10.1109\/LRA.2019.2894005","article-title":"HMFP-DBRNN: Real-Time Hand Motion Filtering and Prediction via Deep Bidirectional RNN","volume":"4","author":"Shahtalebi","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Haque, A., Peng, B., Luo, Z., Alahi, A., Yeung, S., and Fei-Fei, L. (2016, January 11\u201314). Towards viewpoint invariant 3d human pose estimation. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_10"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wang, K., Xie, J., Zhang, G., Liu, L., and Yang, J. (2020, January 13\u201319). Sequential 3D human pose and shape estimation from point clouds. Proceedings of the Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00730"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Li, S., and Chan, A.B. (2014, January 1\u20135). 3d human pose estimation from monocular images with deep convolutional neural network. Proceedings of the Asian Conference on Computer Vision, Singapore.","DOI":"10.1007\/978-3-319-16808-1_23"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Martinez, J., Hossain, R., Romero, J., and Little, J.J. (2017, January 22\u201329). A simple yet effective baseline for 3d human pose estimation. Proceedings of the Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.288"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"532","DOI":"10.1038\/nrn1427","article-title":"Optimal feedback control and the neural basis of volitional motor control","volume":"5","author":"Scott","year":"2004","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1688","DOI":"10.1523\/JNEUROSCI.05-07-01688.1985","article-title":"The coordination of arm movements: An experimentally confirmed mathematical model","volume":"5","author":"Flash","year":"1985","journal-title":"J. Neurosci."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1007\/BF00204593","article-title":"Formation and control of optimal trajectory in human multijoint arm movement","volume":"61","author":"Uno","year":"1989","journal-title":"Biol. Cybern."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Sylla, N., Bonnet, V., Venture, G., Armande, N., and Fraisse, P. (2014, January 12\u201315). Human arm optimal motion analysis in industrial screwing task. Proceedings of the 5th IEEE RAS\/EMBS International Conference on Biomedical Robotics and Biomechatronics, Sao Paulo, Brazil.","DOI":"10.1109\/BIOROB.2014.6913905"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"818","DOI":"10.1109\/TASE.2017.2707129","article-title":"Overapproximative human arm occupancy prediction for collision avoidance","volume":"15","author":"Pereira","year":"2017","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_32","unstructured":"Bishop, C.M., and Nasrabadi, N.M. (2006). Pattern Recognition and Machine Learning, Springer."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Luo, R., and Mai, L. (2019, January 4\u20138). Human Intention Inference and On-Line Human Hand Motion Prediction for Human-Robot Collaboration. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Venetian Macao, Macau.","DOI":"10.1109\/IROS40897.2019.8968192"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Wang, J., Fang, Z., Shen, L., and He, C. (2021, January 24\u201326). Prediction of Human Motion with Motion Optimization and Neural Networks. Proceedings of the 2021 3rd International Symposium on Robotics &Intelligent Manufacturing Technology (ISRIMT), Changzhou, China.","DOI":"10.1109\/ISRIMT53730.2021.9596968"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Mainprice, J., and Berenson, D. (2013, January 3\u20137). Human-robot collaborative manipulation planning using early prediction of human motion. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696368"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ding, H., Rei\u00dfig, G., Wijaya, K., Bortot, D., Bengler, K., and Stursberg, O. (2011, January 9\u201313). Human arm motion modeling and long-term prediction for safe and efficient human\u2013robot-interaction. Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China.","DOI":"10.1109\/ICRA.2011.5980248"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.cirp.2020.04.077","article-title":"Recurrent neural network for motion trajectory prediction in human\u2013robot collaborative assembly","volume":"69","author":"Zhang","year":"2020","journal-title":"CIRP Ann."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1007\/s10514-011-9258-8","article-title":"Provably safe navigation for mobile robots with limited field-of-views in dynamic environments","volume":"32","author":"Bouraine","year":"2012","journal-title":"Auton. Robot."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1109\/TPAMI.2019.2929257","article-title":"OpenPose: Realtime multi-person 2D pose estimation using Part Affinity Fields","volume":"43","author":"Cao","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1477","DOI":"10.1007\/s11548-022-02637-9","article-title":"Robust hand tracking for surgical telestration","volume":"17","author":"Petersen","year":"2022","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Shin, J., Matsuoka, A., Hasan, M.A.M., and Srizon, A.Y. (2021). American Sign Language Alphabet Recognition by Extracting Feature from Hand Pose Estimation. Sensors, 21.","DOI":"10.3390\/s21175856"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Nowak, J., Fraisse, P., Cherubini, A., and Daures, J.P. (2021, January 23\u201325). Point Clouds With Color: A Simple Open Library for Matching RGB and Depth Pixels from an Uncalibrated Stereo Pair. Proceedings of the 2021 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Karlsruhe, Germany.","DOI":"10.1109\/MFI52462.2021.9591200"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"108252","DOI":"10.1016\/j.patcog.2021.108252","article-title":"Pedestrian trajectory prediction with convolutional neural networks","volume":"121","author":"Zamboni","year":"2022","journal-title":"Pattern Recognit."},{"key":"ref_44","unstructured":"Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., and Isard, M. (2016, January 2\u20134). {TensorFlow}: A System for {Large-Scale} Machine Learning. Proceedings of the 12th USENIX symposium on operating systems design and implementation (OSDI 16), Savannah, GA, USA."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Li, C., and Chen, X. (2022). Video prediction for driving scenes with a memory differential motion network model. Appl. Intell.","DOI":"10.1007\/s10489-022-03813-9"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"117986","DOI":"10.1016\/j.eswa.2022.117986","article-title":"StockNet\u2014GRU based stock index prediction","volume":"207","author":"Gupta","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"103742","DOI":"10.1016\/j.trc.2022.103742","article-title":"Using CNN-LSTM to predict signal phasing and timing aided by High-Resolution detector data","volume":"141","author":"Islam","year":"2022","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1016\/j.neucom.2022.04.047","article-title":"A dual attention model based on probabilistically mask for 3D human motion prediction","volume":"493","author":"Cao","year":"2022","journal-title":"Neurocomputing"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/18\/6951\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:31:09Z","timestamp":1760142669000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/18\/6951"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,14]]},"references-count":48,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["s22186951"],"URL":"https:\/\/doi.org\/10.3390\/s22186951","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,14]]}}}