{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T18:37:45Z","timestamp":1775155065154,"version":"3.50.1"},"reference-count":59,"publisher":"ASME International","issue":"5","license":[{"start":{"date-parts":[[2023,12,12]],"date-time":"2023-12-12T00:00:00Z","timestamp":1702339200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.asme.org\/publications-submissions\/publishing-information\/legal-policies"}],"funder":[{"DOI":"10.13039\/100000084","name":"Directorate for Engineering","doi-asserted-by":"publisher","award":["2026533"],"award-info":[{"award-number":["2026533"]}],"id":[{"id":"10.13039\/100000084","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,5,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Human intention prediction plays a critical role in human\u2013robot collaboration, as it helps robots improve efficiency and safety by accurately anticipating human intentions and proactively assisting with tasks. While current applications often focus on predicting intent once human action is completed, recognizing human intent in advance has received less attention. This study aims to equip robots with the capability to forecast human intent before completing an action, i.e., early intent prediction. To achieve this objective, we first extract features from human motion trajectories by analyzing changes in human joint distances. These features are then utilized in a Hidden Markov Model (HMM) to determine the state transition times from uncertain intent to certain intent. Second, we propose two models including a Transformer and a Bi-LSTM for classifying motion intentions. Then, we design a human\u2013robot collaboration experiment in which the operator reaches multiple targets while the robot moves continuously following a predetermined path. The data collected through the experiment were divided into two groups: full-length data and partial data before state transitions detected by the HMM. Finally, the effectiveness of the suggested framework for predicting intentions is assessed using two different datasets, particularly in a scenario when motion trajectories are similar but underlying intentions vary. The results indicate that using partial data prior to the motion completion yields better accuracy compared to using full-length data. Specifically, the transformer model exhibits a 2% improvement in accuracy, while the Bi-LSTM model demonstrates a 6% increase in accuracy.<\/jats:p>","DOI":"10.1115\/1.4064258","type":"journal-article","created":{"date-parts":[[2023,12,13]],"date-time":"2023-12-13T17:21:55Z","timestamp":1702488115000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":17,"title":["Early Prediction of Human Intention for Human\u2013Robot Collaboration Using Transformer Network"],"prefix":"10.1115","volume":"24","author":[{"given":"Xinyao","family":"Zhang","sequence":"first","affiliation":[{"name":"University of Florida Department of Environmental Engineering Sciences, , Gainesville, FL 32611"}]},{"given":"Sibo","family":"Tian","sequence":"additional","affiliation":[{"name":"University at Buffalo Department of Mechanical and Aerospace Engineering, , Buffalo, NY 14260"}]},{"given":"Xiao","family":"Liang","sequence":"additional","affiliation":[{"name":"University at Buffalo Department of Civil, Structural, and Environmental Engineering, , Buffalo, NY 14260"}]},{"given":"Minghui","family":"Zheng","sequence":"additional","affiliation":[{"name":"University at Buffalo Department of Mechanical and Aerospace Engineering, , Buffalo, NY 14260"}]},{"given":"Sara","family":"Behdad","sequence":"additional","affiliation":[{"name":"University of Florida Department of Environmental Engineering Sciences, , Gainesville, FL 32611"}]}],"member":"33","published-online":{"date-parts":[[2024,1,8]]},"reference":[{"issue":"3","key":"2025010801015173300_CIT0001","doi-asserted-by":"publisher","first-page":"031004","DOI":"10.1115\/1.4039061","article-title":"Design of Hybrid Cells to Facilitate Safe and Efficient Human\u2013Robot Collaboration During Assembly Operations","volume":"18","author":"Kaipa","year":"2018","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"2","key":"2025010801015173300_CIT0002","doi-asserted-by":"publisher","first-page":"021006","DOI":"10.1115\/1.4052083","article-title":"Automated Planning for Robotic Multi-Resolution Additive Manufacturing","volume":"22","author":"Bhatt","year":"2021","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"key":"2025010801015173300_CIT0003","first-page":"1249","article-title":"Facilitating Intention Prediction for Humans by Optimizing Robot Motions","author":"Stulp","year":"2015"},{"key":"2025010801015173300_CIT0004","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1016\/j.rcim.2018.11.004","article-title":"Human Intention Estimation Based on Hidden Markov Model Motion Validation for Safe Flexible Robotized Warehouses","volume":"57","author":"Petkovi\u0107","year":"2019","journal-title":"Rob. Comput. Integr. Manuf."},{"issue":"1","key":"2025010801015173300_CIT0005","doi-asserted-by":"crossref","DOI":"10.1115\/1.4039145","article-title":"A Review of Intent Detection, Arbitration, and Communication Aspects of Shared Control for Physical Human\u2013Robot Interaction","volume":"70","author":"Losey","year":"2018","journal-title":"ASME Appl. Mech. Rev."},{"key":"2025010801015173300_CIT0006","doi-asserted-by":"publisher","first-page":"924","DOI":"10.1016\/j.procir.2021.11.155","article-title":"Identifying Human Intention During Assembly Operations Using Wearable Motion Capturing Systems Including Eye Focus","volume":"104","author":"Manns","year":"2021","journal-title":"Proc. CIRP"},{"issue":"10","key":"2025010801015173300_CIT0007","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1016\/j.ifacol.2022.09.420","article-title":"Human Intention and Workspace Recognition for Collaborative Assembly","volume":"55","author":"Gajjar","year":"2022","journal-title":"IFAC-PapersOnLine"},{"issue":"16","key":"2025010801015173300_CIT0008","doi-asserted-by":"publisher","first-page":"4371","DOI":"10.3390\/su11164371","article-title":"Industry 5.0-A Human-Centric Solution","volume":"11","author":"Nahavandi","year":"2019","journal-title":"Sustainability"},{"issue":"7","key":"2025010801015173300_CIT0009","doi-asserted-by":"publisher","first-page":"3033","DOI":"10.1109\/TCYB.2019.2905157","article-title":"Making Sense of Spatio-Temporal Preserving Representations for EEG-Based Human Intention Recognition","volume":"50","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Cybern."},{"issue":"1","key":"2025010801015173300_CIT0010","first-page":"1","article-title":"Probabilistic Human Intent Recognition for Shared Autonomy in Assistive Robotics","volume":"9","author":"Jain","year":"2019","journal-title":"ACM Trans. Human-Rob. Interact. (THRI)"},{"issue":"3","key":"2025010801015173300_CIT0011","doi-asserted-by":"publisher","first-page":"599","DOI":"10.26599\/TST.2020.9010061","article-title":"Spatial-Temporal ConvLSTM for Vehicle Driving Intention Prediction","volume":"27","author":"Huang","year":"2021","journal-title":"Tsinghua Sci. Technol."},{"issue":"2","key":"2025010801015173300_CIT0012","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1109\/TIV.2022.3162719","article-title":"Predicting Pedestrian Crossing Intention With Feature Fusion and Spatio-Temporal Attention","volume":"7","author":"Yang","year":"2022","journal-title":"IEEE Trans. Intell. Vehicles"},{"issue":"3","key":"2025010801015173300_CIT0013","doi-asserted-by":"publisher","first-page":"279","DOI":"10.3390\/e22030279","article-title":"Information Entropy-Based Intention Prediction of Aerial Targets Under Uncertain and Incomplete Information","volume":"22","author":"Zhou","year":"2020","journal-title":"Entropy"},{"key":"2025010801015173300_CIT0014","first-page":"279","article-title":"Human Intention Prediction in Human-Robot Collaborative Tasks","author":"Wang","year":"2018"},{"issue":"3","key":"2025010801015173300_CIT0015","doi-asserted-by":"publisher","first-page":"2339","DOI":"10.1109\/TASE.2021.3074873","article-title":"Predicting Human Intentions in Human\u2013Robot Hand-Over Tasks Through Multimodal Learning","volume":"19","author":"Wang","year":"2022","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"issue":"4","key":"2025010801015173300_CIT0016","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1002\/hfm.20703","article-title":"Effects of Movement Speed and Predictability in Human\u2013Robot Collaboration","volume":"27","author":"Koppenborg","year":"2017","journal-title":"Human Factors Ergon. Manuf. Service Ind."},{"issue":"4","key":"2025010801015173300_CIT0017","doi-asserted-by":"publisher","first-page":"040905","DOI":"10.1115\/1.4054000","article-title":"Efficient Joining Sequence Variation Analysis of Stochastic Batch Assemblies","volume":"22","author":"Tabar","year":"2022","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"3","key":"2025010801015173300_CIT0018","doi-asserted-by":"publisher","first-page":"030901","DOI":"10.1115\/1.4056255","article-title":"Upper Extremity Joint Torque Estimation Through an Electromyography-Driven Model","volume":"23","author":"Tahmid","year":"2023","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"key":"2025010801015173300_CIT0019","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2023.3264284","article-title":"Unsupervised Human Activity Recognition Learning for Disassembly Tasks","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Ind. Inform."},{"issue":"1","key":"2025010801015173300_CIT0020","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1016\/j.cirp.2018.04.066","article-title":"Deep Learning-Based Human Motion Recognition for Predictive Context-Aware Human-Robot Collaboration","volume":"67","author":"Wang","year":"2018","journal-title":"CIRP Ann."},{"key":"2025010801015173300_CIT0021","doi-asserted-by":"publisher","first-page":"605","DOI":"10.1016\/j.jmsy.2020.04.007","article-title":"Transferable Two-Stream Convolutional Neural Network for Human Action Recognition","volume":"56","author":"Xiong","year":"2020","journal-title":"J. Manuf. Syst."},{"issue":"6","key":"2025010801015173300_CIT0022","doi-asserted-by":"publisher","first-page":"138","DOI":"10.3390\/robotics11060138","article-title":"A Narrative Review on Wearable Inertial Sensors for Human Motion Tracking in Industrial Scenarios","volume":"11","author":"Digo","year":"2022","journal-title":"Robotics"},{"key":"2025010801015173300_CIT0023","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1016\/j.jmsy.2017.04.009","article-title":"Human Motion Prediction for Human-Robot Collaboration","volume":"44","author":"Liu","year":"2017","journal-title":"J. Manuf. Syst."},{"key":"2025010801015173300_CIT0024","first-page":"674","article-title":"Comparison of Kinect and Vicon Motion Capture of Upper-Body Joint Angle Tracking","author":"Schlagenhauf","year":"2018"},{"key":"2025010801015173300_CIT0025","first-page":"3356","article-title":"An Optimization-Based Human Behavior Modeling and Prediction for Human-Robot Collaborative Disassembly","author":"Tian","year":"2023"},{"key":"2025010801015173300_CIT0026","doi-asserted-by":"publisher","first-page":"28","DOI":"10.3389\/frobt.2015.00028","article-title":"A Review of Human Activity Recognition Methods","volume":"2","author":"Vrigkas","year":"2015","journal-title":"Front. Rob. AI"},{"issue":"12","key":"2025010801015173300_CIT0027","doi-asserted-by":"publisher","first-page":"31314","DOI":"10.3390\/s151229858","article-title":"Physical Human Activity Recognition Using Wearable Sensors","volume":"15","author":"Attal","year":"2015","journal-title":"Sensors"},{"issue":"9","key":"2025010801015173300_CIT0028","doi-asserted-by":"publisher","first-page":"3109","DOI":"10.3390\/s18093109","article-title":"Human Motion Recognition by Textile Sensors Based on Machine Learning Algorithms","volume":"18","author":"Vu","year":"2018","journal-title":"Sensors"},{"issue":"4","key":"2025010801015173300_CIT0029","doi-asserted-by":"publisher","first-page":"1822","DOI":"10.1109\/TCYB.2019.2940276","article-title":"Bayesian Estimation of Human Impedance and Motion Intention for Human\u2013Robot Collaboration","volume":"51","author":"Yu","year":"2019","journal-title":"IEEE Trans. Cybern."},{"key":"2025010801015173300_CIT0030","first-page":"1326","article-title":"Human Intention Estimation Based on Neural Networks for Enhanced Collaboration With Robots","author":"Nicolis","year":"2018"},{"key":"2025010801015173300_CIT0031","first-page":"208","article-title":"Intention Recognition With Recurrent Neural Networks for Dynamic Human-Robot Collaboration","author":"Mavsar","year":"2021"},{"key":"2025010801015173300_CIT0032","first-page":"665","article-title":"Recurrent Neural Networks for Inferring Intentions in Shared Tasks for Industrial Collaborative Robots","author":"Maceira","year":"2020"},{"issue":"7","key":"2025010801015173300_CIT0033","doi-asserted-by":"publisher","first-page":"1461","DOI":"10.1007\/s00607-021-00928-8","article-title":"Multi-Input CNN-GRU Based Human Activity Recognition Using Wearable Sensors","volume":"103","author":"Dua","year":"2021","journal-title":"Computing"},{"key":"2025010801015173300_CIT0034","first-page":"3595","article-title":"Dependency Exploitation: A Unified CNN-RNN Approach for Visual Emotion Recognition","author":"Zhu","year":"2017"},{"issue":"2","key":"2025010801015173300_CIT0035","doi-asserted-by":"publisher","first-page":"3485","DOI":"10.1109\/LRA.2020.2976305","article-title":"Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction","volume":"5","author":"Liu","year":"2020","journal-title":"IEEE Rob. Autom. Lett."},{"key":"2025010801015173300_CIT0036","first-page":"1390","article-title":"Human-Robot Collaboration by Intention Recognition Using Deep LSTM Neural Network","author":"Yan","year":"2019"},{"issue":"9","key":"2025010801015173300_CIT0037","doi-asserted-by":"publisher","first-page":"2892","DOI":"10.3390\/s18092892","article-title":"Feature Representation and Data Augmentation for Human Activity Classification Based on Wearable IMU Sensor Data Using a Deep LSTM Neural Network","volume":"18","author":"Steven Eyobu","year":"2018","journal-title":"Sensors"},{"key":"2025010801015173300_CIT0038","first-page":"1441","article-title":"Intention-Aware Long Horizon Trajectory Prediction of Surrounding Vehicles Using Dual LSTM Networks","author":"Xin","year":"2018"},{"key":"2025010801015173300_CIT0039","doi-asserted-by":"publisher","first-page":"103414","DOI":"10.1016\/j.trc.2021.103414","article-title":"An Improved Learning-Based LSTM Approach for Lane Change Intention Prediction Subject to Imbalanced Data","volume":"133","author":"Shi","year":"2021","journal-title":"Transp. Res. Part C: Emerg. Technol."},{"key":"2025010801015173300_CIT0040","first-page":"327","article-title":"Intent Prediction of Vulnerable Road Users From Motion Trajectories Using Stacked LSTM Network","author":"Saleh","year":"2017"},{"key":"2025010801015173300_CIT0041","first-page":"7082","article-title":"Joint Intention and Trajectory Prediction Based on Transformer","author":"Sui","year":"2021"},{"key":"2025010801015173300_CIT0042","doi-asserted-by":"publisher","first-page":"1157957","DOI":"10.3389\/fnbot.2023.1157957","article-title":"Comparison of LSTM, Transformers, and MLP-Mixer Neural Networks for Gaze Based Human Intention Prediction","volume":"17","author":"Pettersson","year":"2023","journal-title":"Front. Neurorob."},{"key":"2025010801015173300_CIT0043","article-title":"ConveRT: Efficient and Accurate Conversational Representations From Transformers","author":"Henderson","year":"2019","journal-title":"arXiv preprint arXiv:1911.03688"},{"issue":"3","key":"2025010801015173300_CIT0044","doi-asserted-by":"publisher","first-page":"1578","DOI":"10.1109\/TCYB.2021.3106543","article-title":"Hybrid Recurrent Neural Network Architecture-Based Intention Recognition for Human\u2013Robot Collaboration","volume":"53","author":"Gao","year":"2023","journal-title":"IEEE Trans. Cybern."},{"key":"2025010801015173300_CIT0045","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1016\/j.jmsy.2023.01.007","article-title":"An Attention-Based Deep Learning Approach for Inertial Motion Recognition and Estimation in Human-Robot Collaboration","volume":"67","author":"Zhou","year":"2023","journal-title":"J. Manuf. Syst."},{"key":"2025010801015173300_CIT0046","first-page":"1805","article-title":"Extended Kalman Filter for State Estimation and Trajectory Prediction of a Moving Object Detected by an Unmanned Aerial Vehicle","author":"Prevost","year":"2007"},{"key":"2025010801015173300_CIT0047","first-page":"1","article-title":"Operator Intent Prediction With Subgoal Transition Probability Learning for Shared Control Applications","author":"Jin","year":"2020"},{"key":"2025010801015173300_CIT0048","author":"Linderman","year":"2020"},{"issue":"1","key":"2025010801015173300_CIT0049","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1109\/TIV.2021.3065933","article-title":"A Review of HMM-Based Approaches of Driving Behaviors Recognition and Prediction","volume":"7","author":"Deng","year":"2021","journal-title":"IEEE Trans. Intell. Vehicles"},{"key":"2025010801015173300_CIT0050","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1016\/j.comcom.2020.04.021","article-title":"A Driving Intention Prediction Method Based on Hidden Markov Model for Autonomous Driving","volume":"157","author":"Liu","year":"2020","journal-title":"Comput. Commun."},{"key":"2025010801015173300_CIT0051","first-page":"5738","article-title":"A Data-Driven Framework for Proactive Intention-Aware Motion Planning of a Robot in a Human Environment","author":"Peddi","year":"2020"},{"key":"2025010801015173300_CIT0052","first-page":"367","article-title":"Understanding Human Intentions via Hidden Markov Models in Autonomous Mobile Robots","author":"Kelley","year":"2008"},{"issue":"3","key":"2025010801015173300_CIT0053","doi-asserted-by":"publisher","first-page":"1429","DOI":"10.1007\/s11831-020-09422-4","article-title":"A Systematic Review of Hidden Markov Models and Their Applications","volume":"28","author":"Mor","year":"2021","journal-title":"Archiv. Comput. Methods Eng."},{"key":"2025010801015173300_CIT0054","first-page":"1","article-title":"A Survey of HMM-Based Algorithms in Machinery Fault Prediction","author":"Ramezani","year":"2021"},{"key":"2025010801015173300_CIT0055","first-page":"30","article-title":"Attention Is All You Need","author":"Vaswani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"2025010801015173300_CIT0056","first-page":"10335","article-title":"Transformer Networks for Trajectory Forecasting","author":"Giuliari","year":"2021"},{"key":"2025010801015173300_CIT0057","article-title":"TENER: Adapting Transformer Encoder for Named Entity Recognition","author":"Yan","year":"2019","journal-title":"arXiv preprint arXiv:1911.04474"},{"issue":"10","key":"2025010801015173300_CIT0058","doi-asserted-by":"publisher","first-page":"4334","DOI":"10.1109\/TII.2018.2789925","article-title":"Distilling the Knowledge From Handcrafted Features for Human Activity Recognition","volume":"14","author":"Chen","year":"2018","journal-title":"IEEE Trans. Ind. Inform."},{"issue":"1","key":"2025010801015173300_CIT0059","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1109\/MASSP.1986.1165342","article-title":"An Introduction to Hidden Markov Models","volume":"3","author":"Rabiner","year":"1986","journal-title":"IEEE ASSP Mag."}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/24\/5\/051003\/7329329\/jcise_24_5_051003.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/24\/5\/051003\/7329329\/jcise_24_5_051003.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,8]],"date-time":"2025-01-08T01:02:02Z","timestamp":1736298122000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/24\/5\/051003\/1192601\/Early-Prediction-of-Human-Intention-for-Human"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,8]]},"references-count":59,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,5,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4064258","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"value":"1530-9827","type":"print"},{"value":"1944-7078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,8]]},"article-number":"051003"}}