{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,17]],"date-time":"2025-11-17T13:50:40Z","timestamp":1763387440219,"version":"3.44.0"},"reference-count":68,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,8,22]],"date-time":"2024-08-22T00:00:00Z","timestamp":1724284800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62202335"],"award-info":[{"award-number":["62202335"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"?Sino-German Cooperation 2.0 Funding Program of Tongji University","award":["ZD2023029"],"award-info":[{"award-number":["ZD2023029"]}]},{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2023YFC3603502"],"award-info":[{"award-number":["2023YFC3603502"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2024,8,22]]},"abstract":"<jats:p>Integrated speech and hand-motor training is an effective post-stroke rehabilitation method. However, few interactive systems and assistive technologies were developed in this field. Driven by this challenge, we leverage Mixed Reality technology, which merges immersive virtual scenarios with physical hands-on tools in the real world, to provide patients with multi-modal interactions and engaging training experiences. Following a user-centered design approach, we first interviewed seven therapists to identify user requirements and design considerations. We further designed MRehab, an interactive rehabilitation system that allows patients to regain speech and hand skills through MR scenarios that depict daily living activities. We conducted a preliminary user test with 12 patients and 5 therapists to validate the feasibility and understand the user experience with MRehab. The results confirmed its feasibility for hand-motor training. Additionally, the patients expressed high motivation, engagement, and a positive attitude toward using MRehab. Our findings demonstrate the potential of MR technology in integrated speech and hand function rehabilitation training.<\/jats:p>","DOI":"10.1145\/3678532","type":"journal-article","created":{"date-parts":[[2024,9,9]],"date-time":"2024-09-09T14:36:21Z","timestamp":1725892581000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["MRehab: A Mixed Reality Rehabilitation System Supporting Integrated Speech and Hand Training"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2688-8306","authenticated-orcid":false,"given":"Qi","family":"Wang","sequence":"first","affiliation":[{"name":"College of Design and Innovation, Tongji University, Shanghai, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-1361-0987","authenticated-orcid":false,"given":"Can","family":"Luo","sequence":"additional","affiliation":[{"name":"College of Design and Innovation, Tongji University, Shanghai, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-6090-8778","authenticated-orcid":false,"given":"Shuai","family":"Yang","sequence":"additional","affiliation":[{"name":"College of Design and Innovation, Tongji University, Shanghai, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9347-0882","authenticated-orcid":false,"given":"Shuo","family":"Xu","sequence":"additional","affiliation":[{"name":"Zhangzhou Municipal Hospital of Fujian Medical University, Zhangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-4431-453X","authenticated-orcid":false,"given":"Yibo","family":"Yang","sequence":"additional","affiliation":[{"name":"College of Design and Innovation, Tongji University, Shanghai, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4516-4629","authenticated-orcid":false,"given":"Jie","family":"Jia","sequence":"additional","affiliation":[{"name":"Huashan hospital, Fudan university, Shanghai, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3128-7441","authenticated-orcid":false,"given":"Bin","family":"Yu","sequence":"additional","affiliation":[{"name":"Nyenrode Business University, Breukelen, Netherlands"}]}],"member":"320","published-online":{"date-parts":[[2024,9,9]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1163\/22134808-00002530"},{"key":"e_1_2_2_2_1","volume-title":"Debora Duarte Macea, and Felipe Fregni","author":"Adeyemo Bamidele O","year":"2012","unstructured":"Bamidele O Adeyemo, Marcel Simis, Debora Duarte Macea, and Felipe Fregni. 2012. Systematic review of parameters of stimulation, clinical trial design characteristics, and motor outcomes in non-invasive brain stimulation in stroke. Frontiers in psychiatry 3 (2012), 88."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1088\/1757-899X\/979\/1\/012004"},{"key":"e_1_2_2_4_1","volume-title":"Language as a predictor of motor recovery: the case for a more global approach to stroke rehabilitation. Neurorehabilitation and neural repair 33, 3","author":"Anderlini Deanna","year":"2019","unstructured":"Deanna Anderlini, Guy Wallis, and Welber Marinovic. 2019. Language as a predictor of motor recovery: the case for a more global approach to stroke rehabilitation. Neurorehabilitation and neural repair 33, 3 (2019), 167--178."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-018-7089-8"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-018-1100-9"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2016.02.018"},{"volume-title":"Evaluating object manipulation interaction techniques in mixed reality: Tangible user interfaces and gesture. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR)","author":"Bozgeyikli Evren","key":"e_1_2_2_8_1","unstructured":"Evren Bozgeyikli and Lal Lila Bozgeyikli. 2021. Evaluating object manipulation interaction techniques in mixed reality: Tangible user interfaces and gesture. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). IEEE, 778--787."},{"key":"e_1_2_2_9_1","volume-title":"Speech and language therapy for aphasia following stroke. Cochrane database of systematic reviews 6","author":"Brady Marian C","year":"2016","unstructured":"Marian C Brady, Helen Kelly, Jon Godwin, Pam Enderby, and Pauline Campbell. 2016. Speech and language therapy for aphasia following stroke. Cochrane database of systematic reviews 6 (2016)."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.2196\/30196"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.5014\/ajot.57.5.499"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1080\/24750573.2019.1699738"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12984-016-0153-6"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376613"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0005-7916(00)00012-4"},{"key":"e_1_2_2_16_1","volume-title":"Abdelkader Gouaich, and Isabelle Laffont.","author":"Loreto Ines Di","year":"2011","unstructured":"Ines Di Loreto, Liesjet Van Dokkum, Abdelkader Gouaich, and Isabelle Laffont. 2011. Mixed reality as a means to strengthen post-stroke rehabilitation. In Virtual and Mixed Reality-Systems and Applications: International Conference, Virtual and Mixed Reality 2011, Held as Part of HCI International 2011, Orlando, FL, USA, July 9-14, 2011, Proceedings, Part II 4. Springer, 11--19."},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNSRE.2010.2055061"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2023.105235"},{"volume-title":"Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. 1114--1119","author":"Georgiou Theodoros","key":"e_1_2_2_19_1","unstructured":"Theodoros Georgiou, Simon Holland, and Janet van der Linden. 2016. Wearable haptic devices for post-stroke gait rehabilitation. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. 1114--1119."},{"key":"e_1_2_2_20_1","volume-title":"Interaction between recovery of motor and language abilities after stroke. Archives of physical medicine and rehabilitation 101, 8","author":"Ginex Valeria","year":"2020","unstructured":"Valeria Ginex, Giulia Gilardone, Mauro Vigan\u00f2, Alessia Monti, Elda Judica, Ilaria Passaro, Marco Gilardone, Nicola Vanacore, and Massimo Corbo. 2020. Interaction between recovery of motor and language abilities after stroke. Archives of physical medicine and rehabilitation 101, 8 (2020), 1367--1376."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3118842"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/s42235-022-00214-z"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR.2012.6180940"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1080\/2326263X.2018.1493073"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR-Adjunct.2019.00047"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1002\/hipo.22840"},{"key":"e_1_2_2_27_1","volume-title":"Three approaches to qualitative content analysis. Qualitative health research 15, 9","author":"Hsieh Hsiu-Fang","year":"2005","unstructured":"Hsiu-Fang Hsieh and Sarah E Shannon. 2005. Three approaches to qualitative content analysis. Qualitative health research 15, 9 (2005), 1277--1288."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.3389\/fneur.2021.634065"},{"key":"e_1_2_2_29_1","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1002\/ana.21198","article-title":"Mirror neuron system: basic findings and clinical applications","volume":"62","author":"Iacoboni Marco","year":"2007","unstructured":"Marco Iacoboni and John C Mazziotta. 2007. Mirror neuron system: basic findings and clinical applications. Annals of Neurology: Official Journal of the American Neurological Association and the Child Neurology Society 62, 3 (2007), 213--218.","journal-title":"Annals of Neurology: Official Journal of the American Neurological Association and the Child Neurology Society"},{"key":"e_1_2_2_30_1","doi-asserted-by":"crossref","unstructured":"Marco Iosa Stefan Hesse Antonio Oliviero Stefano Paolucci et al. 2013. New technologies for stroke rehabilitation.","DOI":"10.1155\/2013\/815814"},{"key":"e_1_2_2_31_1","volume-title":"Multisensory stimulation in stroke rehabilitation. Frontiers in human neuroscience 6","author":"Johansson Barbro Birgitta","year":"2012","unstructured":"Barbro Birgitta Johansson. 2012. Multisensory stimulation in stroke rehabilitation. Frontiers in human neuroscience 6 (2012), 60."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3369817"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICORR.2007.4428498"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3546118.3546147"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12984-022-01071-x"},{"key":"e_1_2_2_36_1","volume-title":"What do motor \"recovery\" and \"compensation\" mean in patients following stroke? Neurorehabilitation and neural repair 23, 4","author":"Levin Mindy F","year":"2009","unstructured":"Mindy F Levin, Jeffrey A Kleim, and Steven L Wolf. 2009. What do motor \"recovery\" and \"compensation\" mean in patients following stroke? Neurorehabilitation and neural repair 23, 4 (2009), 313--319."},{"key":"e_1_2_2_37_1","first-page":"19","article-title":"Feedback and virtual environments for motor learning and rehabilitation","volume":"1","author":"Levin Mindy F","year":"2010","unstructured":"Mindy F Levin, Heidi Sveistrup, and S Subramanian. 2010. Feedback and virtual environments for motor learning and rehabilitation. Schedae 1 (2010), 19--36.","journal-title":"Schedae"},{"key":"e_1_2_2_38_1","doi-asserted-by":"crossref","unstructured":"Jianren Ling Zhuochao Peng Lu Yin and Xiaojing Yuan. 2020. How efficiency and naturalness change in multimodal interaction in mobile navigation apps. In Advances in Usability User Experience Wearable and Assistive Technology: Proceedings of the AHFE 2020 Virtual Conferences on Usability and User Experience Human Factors and Assistive Technology Human Factors and Wearable Technologies and Virtual Environments and Game Design July 16-20 2020 USA. Springer 196--207.","DOI":"10.1007\/978-3-030-51828-8_26"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.3390\/app12094540"},{"key":"e_1_2_2_40_1","volume-title":"Augmented reality-based training system for hand rehabilitation. Multimedia tools and applications 76","author":"Liu Jia","year":"2017","unstructured":"Jia Liu, Jianhui Mei, Xiaorui Zhang, Xiong Lu, and Jing Huang. 2017. Augmented reality-based training system for hand rehabilitation. Multimedia tools and applications 76 (2017), 14847--14867."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3563657.3596098"},{"key":"e_1_2_2_42_1","volume-title":"Minh Ha Nguyen, et al","author":"Lui Siew Kwaon","year":"2018","unstructured":"Siew Kwaon Lui, Minh Ha Nguyen, et al. 2018. Elderly stroke rehabilitation: overcoming the complications and its associated challenges. Current gerontology and geriatrics research 2018 (2018)."},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12984-019-0612-y"},{"key":"e_1_2_2_44_1","volume-title":"Psychometric properties of the Intrinsic Motivation Inventory in a competitive sport setting: A confirmatory factor analysis. Research quarterly for exercise and sport 60, 1","author":"McAuley Edward","year":"1989","unstructured":"Edward McAuley, Terry Duncan, and Vance V Tammen. 1989. Psychometric properties of the Intrinsic Motivation Inventory in a competitive sport setting: A confirmatory factor analysis. Research quarterly for exercise and sport 60, 1 (1989), 48--58."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1113\/JP281314"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0104487"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1474-4422(19)30192-9"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.3390\/s22207709"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ConfTELE50222.2021.9435458"},{"key":"e_1_2_2_50_1","volume-title":"2022 IEEE International Symposium on Medical Measurements and Applications (MeMeA). IEEE, 1--5.","author":"Pascucci Simona","year":"2022","unstructured":"Simona Pascucci, Mariano Serrao, Franco Marinozzi, Fabiano Bini, et al. 2022. Exergaming in mixed reality for the rehabilitation of ataxic patients. In 2022 IEEE International Symposium on Medical Measurements and Applications (MeMeA). IEEE, 1--5."},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/SeGAH49190.2020.9201789"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-66562-7_25"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.3390\/app10020636"},{"key":"e_1_2_2_54_1","volume-title":"Reciprocal influence of mobility and speech-language: Advancing physical therapy and speech therapy cotreatment and collaboration for adults with neurological conditions. Physical therapy 101, 11","author":"Schwab Sarah M","year":"2021","unstructured":"Sarah M Schwab, Sarah Dugan, and Michael A Riley. 2021. Reciprocal influence of mobility and speech-language: Advancing physical therapy and speech therapy cotreatment and collaboration for adults with neurological conditions. Physical therapy 101, 11 (2021), pzab196."},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNSRE.2022.3156387"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300767"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1002\/pri.1612"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41582-019-0282-1"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3569499"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2021.3111891"},{"key":"e_1_2_2_61_1","volume-title":"Romeo Salemme, Eric Koun, Martin L\u00f6vden, V\u00e9ronique Boulenger, Alice C Roy, and Claudio Brozzoli.","author":"Thibault Simon","year":"2021","unstructured":"Simon Thibault, Rapha\u00ebl Py, Angelo Mattia Gervasi, Romeo Salemme, Eric Koun, Martin L\u00f6vden, V\u00e9ronique Boulenger, Alice C Roy, and Claudio Brozzoli. 2021. Tool use and language share syntactic processes and neural patterns in the basal ganglia. Science 374, 6569 (2021), eabe0874."},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jocn.2021.07.053"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3161195"},{"key":"e_1_2_2_64_1","volume-title":"User acceptance of information technology: Toward a unified view. MIS quarterly","author":"Venkatesh Viswanath","year":"2003","unstructured":"Viswanath Venkatesh, Michael G Morris, Gordon B Davis, and Fred D Davis. 2003. User acceptance of information technology: Toward a unified view. MIS quarterly (2003), 425--478."},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3399715.3399845"},{"key":"e_1_2_2_66_1","volume-title":"Poststroke aphasia rehabilitation: why all talk and no action? Neurorehabilitation and neural repair 33, 4","author":"Wortman-Jutt Susan","year":"2019","unstructured":"Susan Wortman-Jutt and Dylan Edwards. 2019. Poststroke aphasia rehabilitation: why all talk and no action? Neurorehabilitation and neural repair 33, 4 (2019), 235--244."},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1161\/01.STR.0000162715.43417.91"},{"key":"e_1_2_2_68_1","doi-asserted-by":"crossref","unstructured":"Zan Yue Xue Zhang Jing Wang et al. 2017. Hand rehabilitation robotics on poststroke motor recovery. Behavioural neurology 2017 (2017).","DOI":"10.1155\/2017\/3908135"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3678532","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3678532","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T14:42:18Z","timestamp":1755787338000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3678532"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,22]]},"references-count":68,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,8,22]]}},"alternative-id":["10.1145\/3678532"],"URL":"https:\/\/doi.org\/10.1145\/3678532","relation":{},"ISSN":["2474-9567"],"issn-type":[{"type":"electronic","value":"2474-9567"}],"subject":[],"published":{"date-parts":[[2024,8,22]]},"assertion":[{"value":"2024-09-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}