{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T16:30:55Z","timestamp":1779294655643,"version":"3.51.4"},"reference-count":57,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2024,7,5]],"date-time":"2024-07-05T00:00:00Z","timestamp":1720137600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"DST, Government of India","award":["TDP\/BDTD\/21\/2019"],"award-info":[{"award-number":["TDP\/BDTD\/21\/2019"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Grasp classification is pivotal for understanding human interactions with objects, with wide-ranging applications in robotics, prosthetics, and rehabilitation. This study introduces a novel methodology utilizing a multisensory data glove to capture intricate grasp dynamics, including finger posture bending angles and fingertip forces. Our dataset comprises data collected from 10 participants engaging in grasp trials with 24 objects using the YCB object set. We evaluate classification performance under three scenarios: utilizing grasp posture alone, utilizing grasp force alone, and combining both modalities. We propose Glove-Net, a hybrid CNN-BiLSTM architecture for classifying grasp patterns within our dataset, aiming to harness the unique advantages offered by both CNNs and BiLSTM networks. This model seamlessly integrates CNNs\u2019 spatial feature extraction capabilities with the temporal sequence learning strengths inherent in BiLSTM networks, effectively addressing the intricate dependencies present within our grasping data. Our study includes findings from an extensive ablation study aimed at optimizing model configurations and hyperparameters. We quantify and compare the classification accuracy across these scenarios: CNN achieved 88.09%, 69.38%, and 93.51% testing accuracies for posture-only, force-only, and combined data, respectively. LSTM exhibited accuracies of 86.02%, 70.52%, and 92.19% for the same scenarios. Notably, the hybrid CNN-BiLSTM proposed model demonstrated superior performance with accuracies of 90.83%, 73.12%, and 98.75% across the respective scenarios. Through rigorous numerical experimentation, our results underscore the significance of multimodal grasp classification and highlight the efficacy of the proposed hybrid Glove-Net architectures in leveraging multisensory data for precise grasp recognition. These insights advance understanding of human\u2013machine interaction and hold promise for diverse real-world applications.<\/jats:p>","DOI":"10.3390\/s24134378","type":"journal-article","created":{"date-parts":[[2024,7,5]],"date-time":"2024-07-05T12:30:59Z","timestamp":1720182659000},"page":"4378","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Glove-Net: Enhancing Grasp Classification with Multisensory Data and Deep Learning Approach"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9904-4497","authenticated-orcid":false,"given":"Subhash","family":"Pratap","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering, Indian Institute of Technology Guwahati, Guwahati 781039, India"},{"name":"Department of Mechanical Engineering, Gifu University, Gifu 501-1193, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2499-6039","authenticated-orcid":false,"given":"Jyotindra","family":"Narayan","sequence":"additional","affiliation":[{"name":"Department of Computing, Imperial College London, London SW7 2RH, UK"},{"name":"Chair of Digital Health, Universit\u00e4t Bayreuth, 95445 Bayreuth, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yoshiyuki","family":"Hatta","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Gifu University, Gifu 501-1193, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8977-3709","authenticated-orcid":false,"given":"Kazuaki","family":"Ito","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Gifu University, Gifu 501-1193, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4547-6013","authenticated-orcid":false,"given":"Shyamanta M.","family":"Hazarika","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Indian Institute of Technology Guwahati, Guwahati 781039, India"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,7,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Chen, W., Yu, C., Tu, C., Lyu, Z., Tang, J., Ou, S., Fu, Y., and Xue, Z. (2020). A survey on hand pose estimation with wearable sensors and computer-vision-based methods. Sensors, 20.","DOI":"10.3390\/s20041074"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2509014","DOI":"10.1109\/TIM.2021.3077967","article-title":"Dynamic hand gesture recognition based on signals from specialized data glove and deep learning algorithms","volume":"70","author":"Dong","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"6450","DOI":"10.1109\/JSEN.2020.2965580","article-title":"Static and dynamic human arm\/hand gesture capturing and recognition via multiinformation fusion of flexible strain sensors","volume":"20","author":"Zhang","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"10824","DOI":"10.1109\/JSEN.2021.3059028","article-title":"Grasp classification with weft knit data glove using a convolutional neural network","volume":"21","author":"Ayodele","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4002515","DOI":"10.1109\/TIM.2023.3243614","article-title":"State-of-the-art in data gloves: A review of hardware, algorithms, and applications","volume":"72","author":"Pan","year":"2023","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1142\/S0219622022300026","article-title":"A review on vision-based hand gesture recognition targeting RGB-Depth sensors","volume":"22","author":"Rawat","year":"2023","journal-title":"Int. J. Inf. Technol. Decis. Mak."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1109\/RBME.2021.3078190","article-title":"Emerging wearable interfaces and algorithms for hand gesture recognition: A survey","volume":"15","author":"Jiang","year":"2021","journal-title":"IEEE Rev. Biomed. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1016\/j.eng.2023.01.009","article-title":"A Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps","volume":"32","author":"Liu","year":"2023","journal-title":"Engineering"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Pan, M., Tang, Y., and Li, H. (2022, January 18\u201322). Review of the State-of-the-Art of Data Gloves. Proceedings of the 2022 IEEE International Symposium on Smart Electronic Systems (iSES), Warangal, India.","DOI":"10.1109\/iSES54909.2022.00088"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"eaay4663","DOI":"10.1126\/scirobotics.aay4663","article-title":"A tale of two explanations: Enhancing human trust by explaining robot behavior","volume":"4","author":"Edmonds","year":"2019","journal-title":"Sci. Robot."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Liu, H., Xie, X., Millar, M., Edmonds, M., Gao, F., Zhu, Y., Santos, V.J., Rothrock, B., and Zhu, S.C. (2017, January 24\u201328). A glove-based system for studying hand-object manipulation via joint pose and force sensing. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8206575"},{"key":"ref_12","unstructured":"Schlesinger, G. (1919). The mechanical structure of the artificial limbs. Replacement Links and Work Aids: For War Disabled and Accident Victims, Springer."},{"key":"ref_13","first-page":"902","article-title":"The prehensile movements of the human hand","volume":"38","author":"Napier","year":"1956","journal-title":"Bone Jt. J."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1109\/70.34763","article-title":"On grasp choice, grasp models, and the design of hands for manufacturing tasks","volume":"5","author":"Cutkosky","year":"1989","journal-title":"IEEE Trans. Robot. Autom."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1109\/TOH.2012.53","article-title":"A hand-centric classification of human and robot dexterous manipulation","volume":"6","author":"Bullock","year":"2012","journal-title":"IEEE Trans. Haptics"},{"key":"ref_16","unstructured":"Lyons, D. (1985, January 25\u201328). A simple set of grasps for a dextrous hand. Proceedings of the 1985 IEEE International Conference on Robotics and Automation, St. Louis, MO, USA."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1109\/THMS.2015.2470657","article-title":"The grasp taxonomy of human grasp types","volume":"46","author":"Feix","year":"2015","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wang, S., Zheng, J., Zheng, B., and Jiang, X. (2022). Phase-based grasp classification for prosthetic hand control using sEMG. Biosensors, 12.","DOI":"10.3390\/bios12020057"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Sufri, N.A.J., As\u2019Ari, M.A., and Jie, L.Y.J. (2022, January 15\u201316). Vision Based Grasp Type Selection in Hand Prosthesis Using Convolutional Neural Networks. Proceedings of the 2022 2nd International Conference on Intelligent Cybernetics Technology & Applications (ICICyTA), Online.","DOI":"10.1109\/ICICyTA57421.2022.10038013"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Pratap, S., Narayan, J., Hatta, Y., Ito, K., and Hazarika, S.M. (2024, January 14\u201316). From Tactile Signals to Grasp Classification: Exploring Patterns with Machine Learning. Proceedings of the 2024 IEEE International Conference on Interdisciplinary Approaches in Technology and Management for Social Innovation (IATMSI), Gwalior, India.","DOI":"10.1109\/IATMSI60426.2024.10502560"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/j.mejo.2018.01.014","article-title":"Wearable technologies for hand joints monitoring for rehabilitation: A survey","volume":"88","author":"Rashid","year":"2019","journal-title":"Microelectron. J."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Pratap, S., Ito, K., and Hazarika, S.M. (2023, January 5\u20138). Grasp Synergies in Activities of Daily Living: A Cross-sectional Study using a Multi-sensory Data Glove. Proceedings of the 2023 6th International Conference on Advances in Robotics, Ropar, India.","DOI":"10.1145\/3610419.3610431"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"106472","DOI":"10.1016\/j.compag.2021.106472","article-title":"Dexterous robotic grasping of delicate fruits aided with a multi-sensory e-glove and manual grasping analysis for damage-free manipulation","volume":"190","author":"Zheng","year":"2021","journal-title":"Comput. Electron. Agric."},{"key":"ref_24","first-page":"539","article-title":"Hand gesture recognition using deep feature fusion network based on wearable sensors","volume":"21","author":"Yuan","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"107383","DOI":"10.1016\/j.compeleceng.2021.107383","article-title":"A pattern recognition model for static gestures in malaysian sign language based on machine learning techniques","volume":"95","author":"Alrubayi","year":"2021","journal-title":"Comput. Electr. Eng."},{"key":"ref_26","first-page":"1624","article-title":"A wireless multi-channel capacitive sensor system for efficient glove-based gesture recognition with AI at the edge","volume":"67","author":"Pan","year":"2020","journal-title":"IEEE Trans. Circuits Syst. II Express Briefs"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"943","DOI":"10.1109\/TII.2020.3010369","article-title":"A wearable hand rehabilitation system with soft gloves","volume":"17","author":"Chen","year":"2020","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1104","DOI":"10.1109\/TII.2016.2550528","article-title":"Hand body language gesture recognition based on signals from specialized glove and machine learning algorithms","volume":"12","author":"Tabor","year":"2016","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2513809","DOI":"10.1109\/TIM.2021.3109732","article-title":"Toward the minimum number of wearables to recognize signer-independent Italian sign language with machine-learning algorithms","volume":"70","author":"Calado","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"945","DOI":"10.1109\/TBME.2021.3110432","article-title":"Poststroke grasp ability assessment using an intelligent data glove based on action research arm test: Development, algorithms, and experiments","volume":"69","author":"Dutta","year":"2021","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Thanh Noi, P., and Kappas, M. (2017). Comparison of random forest, k-nearest neighbor, and support vector machine classifiers for land cover classification using Sentinel-2 imagery. Sensors, 18.","DOI":"10.3390\/s18010018"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Awad, M., Khanna, R., Awad, M., and Khanna, R. (2015). Support vector machines for classification. Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers, Springer.","DOI":"10.1007\/978-1-4302-5990-9"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"12972","DOI":"10.1109\/JSEN.2020.3001982","article-title":"A robust data-driven soft sensory glove for human hand motions identification and replication","volume":"20","author":"Nassour","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1016\/j.procs.2019.04.018","article-title":"Basic daily activity recognition with a data glove","volume":"151","author":"Maitre","year":"2019","journal-title":"Procedia Comput. Sci."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1626","DOI":"10.1109\/TNSRE.2019.2928719","article-title":"An assessment system for post-stroke manual dexterity using principal component analysis and logistic regression","volume":"27","author":"Lin","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"63373","DOI":"10.1109\/ACCESS.2019.2916887","article-title":"Deep multimodal representation learning: A survey","volume":"7","author":"Guo","year":"2019","journal-title":"Ieee Access"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Noreen, I., Hamid, M., Akram, U., Malik, S., and Saleem, M. (2021). Hand pose recognition using parallel multi stream CNN. Sensors, 21.","DOI":"10.3390\/s21248469"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Lee, S., Choi, Y., Sung, M., Bae, J., and Choi, Y. (2021). A knitted sensing glove for human hand postures pattern recognition. Sensors, 21.","DOI":"10.3390\/s21041364"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"698","DOI":"10.1038\/s41586-019-1234-z","article-title":"Learning the signatures of the human grasp using a scalable tactile glove","volume":"569","author":"Sundaram","year":"2019","journal-title":"Nature"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"6000704","DOI":"10.1109\/LSENS.2018.2864963","article-title":"Sensor-based continuous hand gesture recognition by long short-term memory","volume":"2","author":"Tai","year":"2018","journal-title":"IEEE Sensors Lett."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"219923","DOI":"10.1109\/ACCESS.2020.3039401","article-title":"Deep learning based real-time recognition of dynamic finger gestures using a data glove","volume":"8","author":"Lee","year":"2020","journal-title":"IEEE Access"},{"key":"ref_42","first-page":"200352","article-title":"CNN-LSTM and post-processing for EMG-based hand gesture recognition","volume":"22","author":"Ferri","year":"2024","journal-title":"Intell. Syst. Appl."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1016\/j.bbe.2022.02.005","article-title":"EMGHandNet: A hybrid CNN and Bi-LSTM architecture for hand activity classification using surface EMG signals","volume":"42","author":"Karnam","year":"2022","journal-title":"Biocybern. Biomed. Eng."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Wu, J., Ren, P., Song, B., Zhang, R., Zhao, C., and Zhang, X. (2023). Data glove-based gesture recognition using CNN-BiLSTM model with attention mechanism. PLoS ONE, 18.","DOI":"10.1371\/journal.pone.0294174"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/j.patrec.2021.04.017","article-title":"Object recognition in performed basic daily activities with a handcrafted data glove prototype","volume":"147","author":"Maitre","year":"2021","journal-title":"Pattern Recognit. Lett."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Narayan, J., Johri, S., and Dwivedy, S.K. (2022). LSTM neural network-based classification of sensory signals for healthy and unhealthy gait assessment. Predictive Modeling in Biomedical Data Mining and Analysis, Elsevier.","DOI":"10.1016\/B978-0-323-99864-2.00010-X"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Mishra, S., Abbas, M., Jindal, K., Narayan, J., and Dwivedy, S.K. (2022). Artificial intelligence-based technological advancements in clinical healthcare applications: A systematic review. Revolutions in Product Design for Healthcare: Advances in Product Design and Design Methods for Healthcare, Springer.","DOI":"10.1007\/978-981-16-9455-4_11"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Abbasi, B., Noohi, E., Parastegari, S., and \u017defran, M. (2016, January 26\u201331). Grasp taxonomy based on force distribution. Proceedings of the 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), New York, NY, USA.","DOI":"10.1109\/ROMAN.2016.7745245"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1080\/09638280410001704278","article-title":"A comparison of the grip force distribution in natural hands and in prosthetic hands","volume":"26","author":"Kargov","year":"2004","journal-title":"Disabil. Rehabil."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1706","DOI":"10.1152\/jn.1999.81.4.1706","article-title":"Control of fingertip forces in multidigit manipulation","volume":"81","author":"Flanagan","year":"1999","journal-title":"J. Neurophysiol."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"582","DOI":"10.1109\/TRO.2019.2959445","article-title":"Making sense of vision and touch: Learning multimodal representations for contact-rich tasks","volume":"36","author":"Lee","year":"2020","journal-title":"IEEE Trans. Robot."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Zandigohar, M., Han, M., Sharif, M., G\u00fcnay, S.Y., Furmanek, M.P., Yarossi, M., Bonato, P., Onal, C., Pad\u0131r, T., and Erdo\u011fmu\u015f, D. (2024). Multimodal fusion of emg and vision for human grasp intent inference in prosthetic hand control. Front. Robot. AI, 11.","DOI":"10.3389\/frobt.2024.1312554"},{"key":"ref_53","unstructured":"Pratap, S., Hatta, Y., Ito, K., and Hazarika, S.M. (2024). Understanding Grasp Synergies during Reach-to-grasp using an Instrumented Data Glove. arXiv."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"103259","DOI":"10.1016\/j.robot.2019.103259","article-title":"The anthropomorphic hand assessment protocol (AHAP)","volume":"121","author":"Starke","year":"2019","journal-title":"Robot. Auton. Syst."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Calli, B., Walsman, A., Singh, A., Srinivasa, S., Abbeel, P., and Dollar, A.M. (2015). Benchmarking in manipulation research: The ycb object and model set and benchmarking protocols. arXiv.","DOI":"10.1109\/MRA.2015.2448951"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1080\/105294199277860","article-title":"The international conference on harmonization good clinical practice guideline","volume":"6","author":"Dixon","year":"1999","journal-title":"Qual. Assur."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Gohil, P., Thoduka, S., and Pl\u00f6ger, P.G. (2022, January 21\u201325). Sensor fusion and multimodal learning for robotic grasp verification using neural networks. Proceedings of the 2022 26th International Conference on Pattern Recognition (ICPR), Montreal, QC, Canada.","DOI":"10.1109\/ICPR56361.2022.9955646"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/13\/4378\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:10:54Z","timestamp":1760109054000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/13\/4378"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,5]]},"references-count":57,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2024,7]]}},"alternative-id":["s24134378"],"URL":"https:\/\/doi.org\/10.3390\/s24134378","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,5]]}}}