{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T20:58:42Z","timestamp":1791406722562,"version":"4.3.3"},"reference-count":49,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2025,3,26]],"date-time":"2025-03-26T00:00:00Z","timestamp":1742947200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Tateishi Science and Technology Foundation"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This study introduces a novel system for evaluating hand motor function through synergy-based analysis during object manipulation in virtual and mixed-reality environments. Conventional assessments of hand function are often subjective, relying on visual observation by therapists or patient-reported outcomes. To address these limitations, we developed a system that utilizes the leap motion controller (LMC) to capture finger motion data without the constraints of glove-type devices. Spatial synergies were extracted using principal component analysis (PCA) and Varimax rotation, providing insights into finger motor coordination with the sparse decomposition. Additionally, we incorporated the HoloLens 2 to create a mixed-reality object manipulation task that enhances spatial awareness for the user, improving natural interaction with virtual objects. Our results demonstrate that synergy-based analysis allows for the systematic detection of hand movement abnormalities that are not captured through traditional task performance metrics. This system demonstrates promise in advancing rehabilitation by enabling more objective and detailed evaluations of finger motor function, facilitating personalized therapy, and potentially contributing to the early detection of motor impairments in the future.<\/jats:p>","DOI":"10.3390\/s25072080","type":"journal-article","created":{"date-parts":[[2025,3,28]],"date-time":"2025-03-28T03:35:28Z","timestamp":1743132928000},"page":"2080","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Synergy-Based Evaluation of Hand Motor Function in Object Handling Using Virtual and Mixed Realities"],"prefix":"10.3390","volume":"25","author":[{"given":"Yuhei","family":"Sorimachi","sequence":"first","affiliation":[{"name":"Neuro-Robotics Lab, Department of Robotics, Graduate School of Engineering, Tohoku University, Sendai 980-8579, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hiroki","family":"Akaida","sequence":"additional","affiliation":[{"name":"Neuro-Robotics Lab, Department of Robotics, Graduate School of Engineering, Tohoku University, Sendai 980-8579, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5326-7847","authenticated-orcid":false,"given":"Kyo","family":"Kutsuzawa","sequence":"additional","affiliation":[{"name":"Neuro-Robotics Lab, Department of Robotics, Graduate School of Engineering, Tohoku University, Sendai 980-8579, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1217-3892","authenticated-orcid":false,"given":"Dai","family":"Owaki","sequence":"additional","affiliation":[{"name":"Neuro-Robotics Lab, Department of Robotics, Graduate School of Engineering, Tohoku University, Sendai 980-8579, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6179-5706","authenticated-orcid":false,"given":"Mitsuhiro","family":"Hayashibe","sequence":"additional","affiliation":[{"name":"Neuro-Robotics Lab, Department of Robotics, Graduate School of Engineering, Tohoku University, Sendai 980-8579, Japan"},{"name":"Department of Biomedical Engineering, Graduate School of Biomedical Engineering, Tohoku University, Sendai 980-8579, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"604","DOI":"10.46234\/ccdcw2021.160","article-title":"Major trends in population growth around the world","volume":"3","author":"Gu","year":"2021","journal-title":"China CDC Wkly."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"ii37","DOI":"10.1093\/ageing\/afl084","article-title":"Falls in older people: Epidemiology, risk factors and strategies for prevention","volume":"35","author":"Rubenstein","year":"2006","journal-title":"Age Ageing"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1056\/NEJMcp020719","article-title":"Preventing falls in elderly persons","volume":"348","author":"Tinetti","year":"2003","journal-title":"N. Engl. J. Med."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Carment, L., Abdellatif, A., Lafuente-Lafuente, C., Pariel, S., Maier, M.A., Belmin, J., and Lindberg, P.G. (2018). Manual dexterity and aging: A pilot study disentangling sensorimotor from cognitive decline. Front. Neurol., 9.","DOI":"10.3389\/fneur.2018.00910"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"909","DOI":"10.1002\/gps.5492","article-title":"Psychomotor processing and functional decline in Parkinson\u2019s disease predicted by the Purdue Pegboard test","volume":"36","author":"Hinkle","year":"2021","journal-title":"Int. J. Geriatr. Psychiatry"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"949","DOI":"10.1161\/01.STR.30.5.949","article-title":"Impaired dexterity of the ipsilateral hand after stroke and the relationship to cognitive deficit","volume":"30","author":"Sunderland","year":"1999","journal-title":"Stroke"},{"key":"ref_7","first-page":"61","article-title":"Functional evaluation: The Barthel Index: A simple index of independence useful in scoring improvement in the rehabilitation of the chronically ill","volume":"14","author":"Mahoney","year":"1965","journal-title":"Md. State Med. J."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Gummesson, C., Atroshi, I., and Ekdahl, C. (2003). The disabilities of the arm, shoulder and hand (DASH) outcome questionnaire: Longitudinal construct validity and measuring self-rated health change after surgery. BMC Musculoskelet. Disord., 4.","DOI":"10.1186\/1471-2474-4-11"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1161\/STROKEAHA.110.592766","article-title":"Fugl-Meyer assessment of sensorimotor function after stroke: Standardized training procedure for clinical practice and clinical trials","volume":"42","author":"Sullivan","year":"2011","journal-title":"Stroke"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Wade, E., Parnandi, A.R., and Mataric, M.J. (2010, January 22\u201325). Automated administration of the Wolf Motor Function Test for post-stroke assessment. Proceedings of the 4th International Conference on Pervasive Computing Technologies for Healthcare, Munich, Germany.","DOI":"10.4108\/ICST.PERVASIVEHEALTH2010.8903"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"7001350010p1","DOI":"10.5014\/ajot.2016.015487","article-title":"Range of motion requirements for upper-limb activities of daily living","volume":"70","author":"Gates","year":"2016","journal-title":"Am. J. Occup. Ther."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1016\/j.jht.2016.08.001","article-title":"Functional range of motion of the hand joints in activities of the International Classification of Functioning, Disability and Health","volume":"30","author":"Vergara","year":"2017","journal-title":"J. Hand Ther."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.clinbiomech.2015.10.002","article-title":"A novel method for in-vivo evaluation of finger kinematics including definition of healthy motion patterns","volume":"31","author":"Coupier","year":"2016","journal-title":"Clin. Biomech."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Mottet, D., van Dokkum, L.E.H., Froger, J., Goua\u00efch, A., and Laffont, I. (2017). Trajectory formation principles are the same after mild or moderate stroke. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0173674"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.apergo.2016.03.003","article-title":"Using kinematic reduction for studying grasping postures. An application to power and precision grasp of cylinders","volume":"56","author":"Vergara","year":"2016","journal-title":"Appl. Ergon."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"661","DOI":"10.1016\/j.humov.2003.10.001","article-title":"Quantitative analysis of finger motion coordination in hand manipulative and gestic acts","volume":"22","author":"Braido","year":"2004","journal-title":"Hum. Mov. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1186\/1743-0003-11-113","article-title":"Analysis of hand synergies in healthy subjects during bimanual manipulation of various objects","volume":"11","author":"Ribeiro","year":"2014","journal-title":"J. Neuroeng. Rehabil."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Liu, Y., Jiang, L., Yang, D., and Liu, H. (2016). Analysis of hand and wrist postural synergies in tolerance grasping of various objects. PLoS ONE, 11.","DOI":"10.1371\/journal.pone.0161772"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2896","DOI":"10.1152\/jn.2001.86.6.2896","article-title":"Hand synergies during reach-to-grasp","volume":"86","author":"Mason","year":"2001","journal-title":"J. Neurophysiol."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Patel, V., Thukral, P., Burns, M.K., Florescu, I., Chandramouli, R., and Vinjamuri, R. (2017). Hand grasping synergies as biometrics. Front. Bioeng. Biotechnol., 5.","DOI":"10.3389\/fbioe.2017.00026"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"10105","DOI":"10.1523\/JNEUROSCI.18-23-10105.1998","article-title":"Postural hand synergies for tool use","volume":"18","author":"Santello","year":"1998","journal-title":"J. Neurosci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1271","DOI":"10.1523\/JNEUROSCI.4512-07.2008","article-title":"Multidigit movement synergies of the human hand in an unconstrained haptic exploration task","volume":"28","author":"Thakur","year":"2008","journal-title":"J. Neurosci."},{"key":"ref_23","unstructured":"Todorov, E., and Ghahramani, Z. (2004, January 1\u20135). Analysis of the synergies underlying complex hand manipulation. Proceedings of the 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Francisco, CA, USA."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Fukunishi, A., Kutsuzawa, K., Owaki, D., and Hayashibe, M. (2024). Synergy quality assessment of muscle modules for determining learning performance using a realistic musculoskeletal model. Front. Comput. Neurosci., 18.","DOI":"10.3389\/fncom.2024.1355855"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"41474","DOI":"10.1109\/JSEN.2024.3486294","article-title":"Muscle control analysis of human walking and cycling in speed and load variations with time-varying synergy","volume":"24","author":"Matsumura","year":"2024","journal-title":"IEEE Sens. J."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1253","DOI":"10.1109\/TNSRE.2019.2915816","article-title":"Evaluation of finger force control ability in terms of multi-finger synergy","volume":"27","author":"Lee","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1556","DOI":"10.1109\/TNSRE.2020.2998642","article-title":"Hand kinematics characterization while performing activities of daily living through kinematics reduction","volume":"28","author":"Vergara","year":"2020","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Camardella, C., Tse, K.C., Frisoli, A., and Tong, R.K.-Y. (2020, January 20\u201324). Investigating muscle synergies changes after rehabilitation robotics training on stroke survivors: A pilot study. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada,.","DOI":"10.1109\/EMBC44109.2020.9175726"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1109\/OJEMB.2023.3263123","article-title":"Optimal Identification of Muscle Synergies From Typical Sit-to-Stand Clinical Tests","volume":"4","author":"Ranaldi","year":"2023","journal-title":"IEEE Open J. Eng. Med. Biol."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Tsatsis, C.G., Rice, K.E., Protopopova, V., Ramos, D., Jadav, J., Coppola, J.F., Broderick, M., and Putrino, D. (2017, January 5). Lateropulsion rehabilitation using virtual reality for stroke patients. Proceedings of the 2017 IEEE Long Island Systems, Applications and Technology Conference (LISAT), Farmingdale, NY, USA.","DOI":"10.1109\/LISAT.2017.8001960"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Norouzi-Gheidari, N., Levin, M.F., Fung, J., and Archambault, P. (2013, January 26\u201329). Interactive virtual reality game-based rehabilitation for stroke patients. Proceedings of the 2013 International Conference on Virtual Rehabilitation (ICVR), Philadelphia, PA, USA.","DOI":"10.1109\/ICVR.2013.6662126"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Liao, K.-L., Huang, M., Wang, Y., Wu, Z., Yang, R., Zhang, J., and Wang, L. (2021, January 4\u20136). A virtual reality serious game design for upper limb rehabilitation. Proceedings of the 2021 IEEE 9th International Conference on Serious Games and Applications for Health (SeGAH), Dubai, United Arab Emirates.","DOI":"10.1109\/SEGAH52098.2021.9551913"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Deterding, S., Dixon, D., Khaled, R., and Nacke, L. (2011, January 29\u201330). From game design elements to gamefulness: Defining \u201cgamification\u201d. Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, Tampere, Finland.","DOI":"10.1145\/2181037.2181040"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Song, X., Ding, L., Zhao, J., Jia, J., and Shull, P. (2019, January 19\u201322). Cellphone augmented reality game-based rehabilitation for improving motor function and mental state after stroke. Proceedings of the 2019 IEEE 16th International Conference on Wearable and Implantable Body Sensor Networks (BSN), Chicago, IL, USA.","DOI":"10.1109\/BSN.2019.8771093"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1141","DOI":"10.1016\/j.apmr.2006.04.010","article-title":"Video game-based exercises for balance rehabilitation: A single-subject design","volume":"87","author":"Betker","year":"2006","journal-title":"Arch. Phys. Med. Rehabil."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"626","DOI":"10.1109\/THMS.2018.2860579","article-title":"Projection-based AR: Effective visual feedback in gait rehabilitation","volume":"48","author":"Sekhavat","year":"2018","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Garzotto, F., Torelli, E., Vona, F., and Aruanno, B. (2018, January 10\u201312). HoloLearn: Learning through mixed reality for people with cognitive disability. Proceedings of the 2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), Taichung, Taiwan.","DOI":"10.1109\/AIVR.2018.00042"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Condino, S., Turini, G., Viglialoro, R., Gesi, M., and Ferrari, V. (2019). Wearable augmented reality application for shoulder rehabilitation. Electronics, 8.","DOI":"10.3390\/electronics8101178"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Tada, K., Sorimachi, Y., Kutsuzawa, K., Owaki, D., and Hayashibe, M. (2024). Integrated quantitative evaluation of spatial cognition and motor function with HoloLens mixed reality. Sensors, 24.","DOI":"10.3390\/s24020528"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Pezzera, M., Chitti, E., and Borghese, N.A. (2020, January 12\u201314). MIRARTS: A mixed reality application to support postural rehabilitation. Proceedings of the 2020 IEEE 8th International Conference on Serious Games and Applications for Health (SeGAH), Vancouver, BC, Canada.","DOI":"10.1109\/SeGAH49190.2020.9201694"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Aruanno, B., Garzotto, F., and Rodriguez, M.C. (2017, January 18\u201320). HoloLens-based mixed reality experiences for subjects with Alzheimer\u2019s disease. Proceedings of the 12th Biannual Conference on Italian SIGCHI Chapter, Cagliari, Italy.","DOI":"10.1145\/3125571.3125589"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1152\/jn.00245.2013","article-title":"A unifying model of concurrent spatial and temporal modularity in muscle activity","volume":"111","author":"Delis","year":"2014","journal-title":"J. Neurophysiol."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1007\/BF02289233","article-title":"The varimax criterion for analytic rotation in factor analysis","volume":"23","author":"Kaiser","year":"1958","journal-title":"Psychometrika"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"6380","DOI":"10.3390\/s130506380","article-title":"Analysis of the accuracy and robustness of the leap motion controller","volume":"13","author":"Weichert","year":"2013","journal-title":"Sensors"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1109\/THMS.2015.2470657","article-title":"The grasp taxonomy of human grasp types","volume":"46","author":"Feix","year":"2015","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_46","unstructured":"Ungureanu, D., Bogo, F., Galliani, S., Sama, P., Duan, X., Meekhof, C., St\u00fchmer, J., Cashman, T.J., Tekin, B., and Sch\u00f6nberger, J.L. (2020). Hololens 2 research mode as a tool for computer vision research. arXiv."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Barradas, V.R., Cho, W., and Koike, Y. (2023). EMG space similarity feedback promotes learning of expert-like muscle activation patterns in a complex motor skill. Front. Hum. Neurosci., 16.","DOI":"10.3389\/fnhum.2022.805867"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1002\/ca.980080604","article-title":"Extensor tendons of the fingers: Arrangement and variations-II","volume":"8","author":"Butt","year":"1995","journal-title":"Clin. Anat."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Prevete, R., Donnarumma, F., d\u2019Avella, A., and Pezzulo, G. (2018). Evidence for sparse synergies in grasping actions. Sci. Rep., 8.","DOI":"10.1038\/s41598-017-18776-y"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/7\/2080\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:02:40Z","timestamp":1760029360000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/7\/2080"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,26]]},"references-count":49,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2025,4]]}},"alternative-id":["s25072080"],"URL":"https:\/\/doi.org\/10.3390\/s25072080","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,26]]}}}