{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T12:16:30Z","timestamp":1780575390927,"version":"3.54.1"},"reference-count":56,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T00:00:00Z","timestamp":1766102400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>Collaboration between humans and robots is essential for optimizing the performance of complex tasks in industrial environments, reducing worker strain, and improving safety. This paper presents an integrated human-robot collaboration (HRC) system that leverages advanced intention recognition for real-time task sharing and interaction. By utilizing state-of-the-art human pose estimation combined with deep learning models, we developed a robust framework for detecting and predicting worker intentions. Specifically, we employed LSTM-based and transformer-based neural networks with convolutional and pooling layers to classify human hand trajectories, achieving higher accuracy compared to previous approaches. Additionally, our system integrates dynamic movement primitives (DMPs) for smooth robot motion transitions, collision prevention, and automatic motion onset\/cessation detection. We validated the system in a real-world industrial assembly task, demonstrating its effectiveness in enhancing the fluency, safety, and efficiency of human-robot collaboration. The proposed method shows promise in improving real-time decision-making in collaborative environments, offering a safer and more intuitive interaction between humans and robots.<\/jats:p>","DOI":"10.3389\/frobt.2025.1708987","type":"journal-article","created":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T05:17:40Z","timestamp":1766121460000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Human intention recognition by deep LSTM and transformer networks for real-time human-robot collaboration"],"prefix":"10.3389","volume":"12","author":[{"given":"Matija","family":"Mavsar","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mihael","family":"Simoni\u010d","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ale\u0161","family":"Ude","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,12,19]]},"reference":[{"key":"B1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/IJCNN48605.2020.9207082","article-title":"Adaptive pooling is all you need: an empirical study on hyperparameter-insensitive human action recognition using wearable sensors","volume-title":"2020 international joint conference on neural networks (IJCNN)","author":"Abdu-Aguye","year":"2020"},{"key":"B2","article-title":"Adaptable human intention and trajectory prediction for human-robot collaboration","author":"Abuduweili","year":"2019"},{"key":"B3","doi-asserted-by":"publisher","first-page":"957","DOI":"10.1007\/s10514-017-9677-2","article-title":"Progress and prospects of the human\u2013robot collaboration","volume":"42","author":"Ajoudani","year":"2018","journal-title":"Aut. Robots"},{"key":"B4","first-page":"292","article-title":"Comparing apples to oranges: Llm-powered multimodal intention prediction in an object categorization task","volume-title":"International conference on social robotics","author":"Ali","year":"2024"},{"key":"B5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/tits.2025.3529299","article-title":"Reinforcement learning for dynamic optimization of lane change intention recognition for transportation networks","author":"Byeon","year":"2025","journal-title":"IEEE Trans. Intelligent Transp. Syst."},{"key":"B6","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1016\/j.rcim.2018.11.002","article-title":"Dynamic speed and separation monitoring for collaborative robot applications\u2013concepts and performance","volume":"58","author":"Byner","year":"2019","journal-title":"Robotics Computer-Integrated Manuf."},{"key":"B7","doi-asserted-by":"publisher","first-page":"8327","DOI":"10.1016\/j.ifacol.2023.10.1022","article-title":"A probabilistic dynamic movement primitives framework on human hand motion prediction for an object transfer scenario","volume":"56","author":"Cai","year":"2023","journal-title":"IFAC-PapersOnLine"},{"key":"B8","doi-asserted-by":"publisher","first-page":"5151","DOI":"10.1109\/lra.2020.3005892","article-title":"A framework for recognition and prediction of human motions in human-robot collaboration using probabilistic motion models","volume":"5","author":"Callens","year":"2020","journal-title":"IEEE Robotics Automation Lett."},{"key":"B9","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1186\/s12984-023-01268-8","article-title":"Review of electromyography onset detection methods for real-time control of robotic exoskeletons","volume":"20","author":"Carvalho","year":"2023","journal-title":"J. Neuroengineering Rehabilitation"},{"key":"B10","doi-asserted-by":"publisher","first-page":"7157","DOI":"10.1109\/tpami.2022.3222784","article-title":"Alphapose: whole-body regional multi-person pose estimation and tracking in real-time","volume":"45","author":"Fang","year":"2022","journal-title":"IEEE Trans. Pattern Analysis Mach. Intell."},{"key":"B11","first-page":"509","article-title":"Coupling movement primitives: interaction with the environment and bimanual tasks","volume-title":"2014 IEEE-RAS international conference on humanoid robots","author":"Gams","year":"2014"},{"key":"B12","doi-asserted-by":"publisher","first-page":"1578","DOI":"10.1109\/tcyb.2021.3106543","article-title":"Hybrid recurrent neural network architecture-based intention recognition for human\u2013robot collaboration","volume":"53","author":"Gao","year":"2021","journal-title":"IEEE Trans. Cybern."},{"key":"B13","first-page":"10335","article-title":"Transformer networks for trajectory forecasting","volume-title":"25th international conference on pattern recognition (ICPR)","author":"Giuliari","year":"2021"},{"key":"B14","doi-asserted-by":"crossref","first-page":"666","DOI":"10.1109\/RO-MAN53752.2022.9900533","article-title":"Recognizing motion onset during robot-assisted body-weight unloading is challenging but seems feasible","volume-title":"2022 31st IEEE international conference on robot and human interactive communication (RO-MAN)","author":"Hassani","year":"2022"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"B16","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1109\/thms.2019.2904558","article-title":"Evaluating fluency in human\u2013robot collaboration","volume":"49","author":"Hoffman","year":"2019","journal-title":"IEEE Trans. Human-Machine Syst."},{"key":"B17","doi-asserted-by":"publisher","first-page":"328","DOI":"10.1162\/neco_a_00393","article-title":"Dynamical movement primitives: learning attractor models for motor behaviors","volume":"25","author":"Ijspeert","year":"2013","journal-title":"Neural Comput."},{"key":"B18","doi-asserted-by":"publisher","first-page":"0271","DOI":"10.34133\/space.0271","article-title":"Intention recognition of space noncooperative targets using large language models","volume":"5","author":"Jing","year":"2025","journal-title":"Space Sci. Technol."},{"key":"B19","article-title":"Adam: a method for stochastic optimization","volume-title":"3rd international conference for learning representations (ICLR)","author":"Kingma","year":"2015"},{"key":"B20","first-page":"1229","article-title":"Trajformer: efficient trajectory classification with transformers","volume-title":"Proceedings of the 31st ACM international conference on information and knowledge management","author":"Liang","year":"2022"},{"key":"B21","first-page":"1228","article-title":"Spatio-temporal GRU for trajectory classification","volume-title":"IEEE international conference on data mining (ICDM)","author":"Liu","year":"2019"},{"key":"B22","article-title":"MediaPipe: a framework for building perception pipelines","author":"Lugaresi","year":"2019"},{"key":"B23","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1016\/j.rcim.2016.08.001","article-title":"Implementing speed and separation monitoring in collaborative robot workcells","volume":"44","author":"Marvel","year":"2017","journal-title":"Robotics Computer-Integrated Manuf."},{"key":"B24","doi-asserted-by":"publisher","first-page":"100","DOI":"10.3390\/robotics8040100","article-title":"Human\u2013robot collaboration in manufacturing applications: a review","volume":"8","author":"Matheson","year":"2019","journal-title":"Robotics"},{"key":"B25","first-page":"208","article-title":"Intention recognition with recurrent neural networks for dynamic human-robot collaboration","volume-title":"International conference on advanced robotics (ICAR)","author":"Mavsar","year":"2021"},{"key":"B26","doi-asserted-by":"publisher","first-page":"494","DOI":"10.1109\/tnnls.2022.3175720","article-title":"Simulation-aided handover prediction from video using recurrent image-to-motion networks","volume":"35","author":"Mavsar","year":"2024","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"B27","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1109\/WHC.2019.8816157","article-title":"Prediction of human trajectory following a haptic robotic guide using recurrent neural networks","volume-title":"2019 IEEE world haptics conference (WHC)","author":"Moon","year":"2019"},{"key":"B28","doi-asserted-by":"publisher","first-page":"837","DOI":"10.1017\/s0263574711001056","article-title":"Action sequencing using dynamic movement primitives","volume":"30","author":"Nemec","year":"2012","journal-title":"Robotica"},{"key":"B29","first-page":"8024","article-title":"PyTorch: an imperative style, high-performance deep learning library","volume":"32","author":"Paszke","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"B30","doi-asserted-by":"publisher","first-page":"1157957","DOI":"10.3389\/fnbot.2023.1157957","article-title":"Comparison of LSTM, transformers, and MLP-mixer neural networks for gaze based human intention prediction","volume":"17","author":"Pettersson","year":"2023","journal-title":"Front. Neurorobotics"},{"key":"B31","doi-asserted-by":"crossref","first-page":"1168","DOI":"10.1109\/IROS.2013.6696498","article-title":"Dynamic movement primitives for human-robot interaction: comparison with human behavioral observation","volume-title":"2013 IEEE\/RSJ international conference on intelligent robots and systems","author":"Prada","year":"2013"},{"key":"B32","doi-asserted-by":"crossref","first-page":"7533","DOI":"10.1109\/ICRA.2018.8463211","article-title":"Intent-aware multi-agent reinforcement learning","volume-title":"2018 IEEE international conference on robotics and automation (ICRA)","author":"Qi","year":"2018"},{"key":"B33","doi-asserted-by":"publisher","first-page":"613981","DOI":"10.3389\/fcomp.2021.613981","article-title":"Fusionnet: a deep fully residual convolutional neural network for image segmentation in connectomics","volume":"3","author":"Quan","year":"2021","journal-title":"Front. Comput. Sci."},{"key":"B34","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1007\/11008941_60","article-title":"Learning movement primitives","volume-title":"Robotics research; the eleventh international symposium","author":"Schaal","year":"2005"},{"key":"B35","first-page":"5909","article-title":"Anticipation in human-robot cooperation: a recurrent neural network approach for multiple action sequences prediction","volume-title":"IEEE international conference on robotics and automation (ICRA)","author":"Schydlo","year":"2018"},{"key":"B36","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1007\/s10846-024-02051-0","article-title":"Dynamic via-points and improved spatial generalization for online trajectory generation with dynamic movement primitives","volume":"110","author":"Sidiropoulos","year":"2024","journal-title":"J. Intelligent Robotic Syst."},{"key":"B37","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/s10846-021-01328-y","article-title":"Analysis of methods for incremental policy refinement by kinesthetic guidance","volume":"102","author":"Simoni\u010d","year":"2021","journal-title":"J. Intelligent Robotic Syst."},{"key":"B38","volume-title":"Robot modeling and control","author":"Spong","year":"2006"},{"key":"B39","first-page":"7082","article-title":"Joint intention and trajectory prediction based on transformer","volume-title":"IEEE\/RSJ international conference on intelligent robots and systems (IROS)","author":"Sui","year":"2021"},{"key":"B40","doi-asserted-by":"crossref","first-page":"10712","DOI":"10.1109\/IROS47612.2022.9982160","article-title":"Sociable and ergonomic human-robot collaboration through action recognition and augmented hierarchical quadratic programming","volume-title":"2022 IEEE\/RSJ international conference on intelligent robots and systems (IROS)","author":"Tassi","year":"2022"},{"key":"B41","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1109\/CIS-RAM47153.2019.9095779","article-title":"Fast human motion prediction for human-robot collaboration with wearable interface","volume-title":"2019 IEEE international conference on cybernetics and intelligent systems (CIS) and IEEE conference on robotics, automation and mechatronics (RAM)","author":"Tortora","year":"2019"},{"key":"B42","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1109\/Humanoids53995.2022.10000097","article-title":"Whole-body control for velocity-controlled mobile collaborative robots using coupling dynamic movement primitives","volume-title":"2022 IEEE-RAS 21st international conference on humanoid robots (humanoids)","author":"Tu","year":"2022"},{"key":"B43","first-page":"2997","article-title":"Orientation in Cartesian space dynamic movement primitives","volume-title":"International conference on robotics and automation (ICRA)","author":"Ude","year":"2014"},{"key":"B44","doi-asserted-by":"publisher","first-page":"800","DOI":"10.1109\/tro.2010.2065430","article-title":"Task-specific generalization of discrete and periodic dynamic movement primitives","volume":"26","author":"Ude","year":"2020","journal-title":"IEEE Trans. Robotics"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1706.03762","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"B46","first-page":"1675","article-title":"Recurrent convolutional networks based intention recognition for human-robot collaboration tasks","volume-title":"IEEE international conference on systems, man, and cybernetics (SMC)","author":"Wang","year":"2017"},{"key":"B47","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1109\/thms.2018.2883176","article-title":"Controlling object hand-over in human\u2013robot collaboration via natural wearable sensing","volume":"49","author":"Wang","year":"2019","journal-title":"IEEE Trans. Human-Machine Syst."},{"key":"B48","first-page":"88","article-title":"Towards data-efficient detection transformers","volume-title":"European conference on computer vision (ECCV)","author":"Wang","year":"2022"},{"key":"B49","first-page":"2089","article-title":"Optimizing deeper transformers on small datasets","volume-title":"Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers)","author":"Xu","year":"2021"},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.12328","article-title":"Spatial temporal graph convolutional networks for skeleton-based action recognition","volume":"32","author":"Yan","year":"2018","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"B51","doi-asserted-by":"crossref","first-page":"1390","DOI":"10.1109\/FPM45753.2019.9035907","article-title":"Human-robot collaboration by intention recognition using deep LSTM neural network","volume-title":"2019 IEEE 8th international conference on fluid power and mechatronics (FPM)","author":"Yan","year":"2019"},{"key":"B52","first-page":"3118","article-title":"Reactive human-to-robot handovers of arbitrary objects","volume-title":"International conference on robotics and automation (ICRA)","author":"Yang","year":"2021"},{"key":"B53","doi-asserted-by":"publisher","first-page":"1686","DOI":"10.1109\/lra.2021.3058917","article-title":"A scalable approach to predict multi-agent motion for human-robot collaboration","volume":"6","author":"Yasar","year":"2021","journal-title":"IEEE Robotics Automation Lett."},{"key":"B54","first-page":"507","article-title":"Spatio-temporal graph transformer networks for pedestrian trajectory prediction","volume-title":"European conference on computer vision (ECCV)","author":"Yu","year":"2020"},{"key":"B55","doi-asserted-by":"publisher","first-page":"1330","DOI":"10.1109\/34.888718","article-title":"A flexible new technique for camera calibration","volume":"22","author":"Zhang","year":"2000","journal-title":"IEEE Trans. Pattern Analysis Mach. Intell."},{"key":"B56","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.cirp.2020.04.077","article-title":"Recurrent neural network for motion trajectory prediction in human-robot collaborative assembly","volume":"69","author":"Zhang","year":"2020","journal-title":"CIRP Ann."}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1708987\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T05:17:46Z","timestamp":1766121466000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1708987\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,19]]},"references-count":56,"alternative-id":["10.3389\/frobt.2025.1708987"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2025.1708987","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,19]]},"article-number":"1708987"}}